Detailed explanation of PHP regularity and data collection

• PHP regular expressions are mainly used for pattern segmentation, matching, search and replacement operations of strings. Using regular expressions may not be efficient in some simple environments,
  Therefore, how to better use PHP regular expressions needs to be considered comprehensively.
  Definition of PHP regular expression:
  A grammatical rule used to describe character arrangement and matching patterns.
  Regular functions in PHP:
  There are two sets of regular functions in PHP, both of which have similar functions:
  One set is provided by the PCRE (Perl Compatible Regular Expression) library. Functions named with the prefix "preg_";
  A set provided by POSIX (Portable Operating System Interface of Unix) extensions. Use functions named with the prefix "ereg_";
  (POSIX regular function library is no longer recommended for use since PHP 5.3, and will be removed from PHP 6)
  Since POSIX regular function library is about to be released, history Stage, and the forms of PCRE and perl are similar, which is more convenient for us to switch between perl and php, so here we focus on the use of PCRE regularity.
  PCRE Regular Expression
  PCRE stands for Perl Compatible Regular Expression, which means Perl compatible regular expression.
  In PCRE, the pattern expression (ie regular expression) is usually included between two backslashes "/", such as "/apple/".
  Several important concepts in regular expressions include: metacharacters, escapes, pattern units (repetitions), antonyms, references, and assertions. These concepts can be easily understood and mastered in JavaScript.
  preg_filter — Regular expression search and replacement
  preg_filter("regular", "replacement", "target");
  preg_grep ( string $pattern , array $input [, int $flags = 0 ] )
  preg_match — Perform a regular expression match
  preg_match ( string $pattern , string $subject [, array &$matches [, int $flags = 0 [, int $offset = 0 ]]] )
  preg_match_all — Global regular expression matching
  preg_replace — Perform a regular expression search and replace
  preg_split — Separate strings by a regular expression
  Commonly used meta-characters:
  Metacharacter Description
  \A Matches the atom at the beginning of the string
  \Z Matches the atom at the end of the string
  \b Matches the boundary of the word /\bis/ Matches the string whose head is is /is\ b/ Matches strings ending in is /\bis\b/ Delimitation
  \B Matches any character except word boundaries /\Bis/ Matches "is" in the word "This"
  \d Matches a number; equivalent to [0-9]
  \D Matches any character except numbers; equivalent to [^0-9]
  \w Matches an English letter, number or underscore; equivalent In [0-9a-zA-Z_]
  \W Matches any character except English letters, numbers and underscores; equivalent to [^0-9a-zA-Z_]
  \s Matches a blank character ; Equivalent to [\f\t\v]
  \S Matches any character except whitespace characters; Equivalent to [^\f\t\v]
  \f Matches a form feed character Matches a newline character in \x0c or \cL
  ; matches a carriage return character in \x0a or \cJ
  ; matches a tab character in \x0d or \cM
  \t ; etc. Equivalent to \x09\ or \cl
  \v Matches a vertical tab character; Equivalent to \x0b or \ck
  \oNN Matches an octal digit
  \xNN Matches a hexadecimal digit
  \cC Matches a control character
  Pattern Modifiers:
  Pattern modifiers are particularly used in ignoring case and matching multiple lines. Mastering this modifier can often solve our problems. Got a lot of questions.
  i ——Can match both upper and lower case letters
  M ——Treat the string as multiple lines
  S ——Treat the string as a single line, and treat newlines as ordinary characters, so that "." matches any character
  ” etc., ignoring case. /i
  Matches
  characters Description
  \ Marks the next character as a special character, text, backreference, or octal escape character. For example, "n" matches the character "n". "\n" matches a newline character. The sequence "\\" matches "\", and "\(" matches "(".
  ^ Matches the beginning of the input string. If the Multiline property of the RegExp object is set, ^ will also match "\n" or " Matches the position after \r".
  $ Matches the end of the input string. If the RegExp object's Multiline property is set, $ also matches the position before "\n" or "\r".
  * Matches the preceding character or subexpression zero or more times. For example, zo* matches "z" and "zoo". * Equivalent to {0,}.
  + Matches the previous character or subexpression one or more times. For example, "zo+" matches "zo" and "zoo" but not "z". + Equivalent to {1,}.
  ? Matches the preceding character or subexpression zero or once times. For example, "do(es)?" matches the "do" in "do" or "does". ? Equivalent to {0,1}.
  {n} n is a non-negative integer. Matches exactly n times. For example, "o{2}" does not match the "o" in "Bob" but does match both "o"s in "food".
  {n,} n is a non-negative integer. Match at least n times. For example, "o{2,}" does not match the "o" in "Bob" but matches all o's in "foooood". "o{1,}" is equivalent to "o+". "o{0,}" is equivalent to "o*".
  {n,m} M and n are non-negative integers, where n ?    When this character immediately follows any other qualifier (*, +, ?, {n}, {n,}, {n,m}), the matching pattern is “non-greedy”. The "non-greedy" pattern matches the shortest possible string that is searched for, while the default "greedy" pattern matches the longest possible string that is searched for. For example, in the string "oooo", "o+?" matches only a single "o", while "o+" matches all "o"s.
  . Matches any single character except "\n". To match any character including "\n", use a pattern such as "[\s\S]".
  [] Matches any one in []
  [^] Matches any one not in []
  1. What is data collection
  A few years ago, except for a few large portals, Basically they are personal websites. The information is scattered and there is not much content.
  In a few years, there will be more and more commercial websites, and information needs to be concentrated in large quantities. Even if there are enough financial resources to hire a large number of copy editors,
  may not be able to meet the ever-changing information resources.
  Information collection has become a technology favored by all website operators.
  2. The idea of ​​​​data collection:
  The idea of ​​​​the collection program is very simple and can be roughly divided into the following steps:
  1. Obtain the remote file source code (file_get_contents or use fopen or use fsocket to implement the http protocol get, or PHP's curl extension or PHP's other third-party open source classes)
  Note: It is best not to use file_get_contents and fopen, you will encounter problems you can't imagine. Use fsocket to implement the get of the http protocol. Now most All websites use anti-collection technology and use information extraction technology when collecting content.
  2. Analyze the code to get what you want (use regular matching here).
  　 3. Carry out downloading and warehousing operations based on the obtained content.
    4. Display data according to your web page layout
  3. Share your personal collection experience:
  1. Do not collect sites that are protected against hotlinking. In fact, you can fake the origin, but such sites collect The cost is too high
  2. For sites that collect as quickly as possible, it is best to collect locally
  3. When collecting, there are many times when you can store part of the data in the database first, and then proceed to the next step of processing.
  4. You must handle errors when collecting. I usually skip it if the collection fails three times.
  In the past, I would often get stuck picking out a piece of content just because I couldn’t pick it up.
  5. Before entering the database, you must make a good judgment, check the legality of the content, and filter unnecessary strings.
  4. Share a link address: Regular expression online test
  http://tool.chinaz.com/regex
  5. Share some commonly used PHP regular expressions
  Regular expressions that match Chinese characters : [\u4e00-\u9fa5]
  Matching double-byte characters (including Chinese characters): [^\x00-\xff]
  Regular expression matching HTML tags: /<(.*)> ;.*<\/>|<(.*) \/>/
  Regular expression matching IP address /(\d+)\.(\d+)\.(\d+)\. (\d+)/g
  Regular expression matching email addresses: \w+([-+.]\w+)*@\w+([-.]\w+)*\.\w+([-.] \w+)*
  Regular expression matching URL: http://([\w-]+\.)+[\w-]+(/[\w- ./?%&=]* )?
  Image link in matching information: /(s|S)(r|R)(c|C)*=*('|")?(w||/|.)+('|" | *|>)?/
  Use regular expressions to extract file names from URL addresses: /(.*\/)([^\.]+).*/ig
  China phone number verification/ ((d{3,4})|d{3,4}-)?d{7,8}(-d{3})*/
  China Postal Code Verification/d{6}/
  Email verification/w+([-+.]w+)*@w+([-.]w+)*.w+([-.]w+)*/
  ID card verification/d{18}|d{15 }/
  Commonly used number verification /d{n}/ n is the specified length /d{n,m}/ The length range from n to m
  Illegal character verification
  Match illegal characters such as: < > & / ' |
  Regular expression [^<>&/|'\]+
  Date verification
  The matching form is: 20030718,030718
  Range: 1900--2099
  Regular expression (( ((19){1}|(20){1})d{2})|d{2})[01]{1}d{1}[0-3]{1}d{1}

Related recommendations:

Example of PHP regular method to determine whether a string contains Chinese characters

PHP regular expression Summary of the formula

php regular expression processing method

The above is the detailed content of Detailed explanation of PHP regularity and data collection. For more information, please follow other related articles on the PHP Chinese website!