Home  >  Article  >  Backend Development  >  Discussion: Several methods of crawling pages with php

Discussion: Several methods of crawling pages with php

WBOY
WBOYOriginal
2016-07-25 08:58:341031browse
  1. $url='http://t.qq.com';
  2. $lines_array=file($url);
  3. $lines_string=implode('',$lines_array);
  4. echo htmlspecialchars($lines_string);
Copy code

2. file_get_contents() function Using file_get_contents and fopen must enable allow_url_fopen. Method: Edit php.ini and set allow_url_fopen = On. When allow_url_fopen is turned off, neither fopen nor file_get_contents can open remote files.

  1. $url='http://t.qq.com';
  2. $lines_string=file_get_contents($url);
  3. echo htmlspecialchars($lines_string);
Copy code

3. fopen()->fread()->fclose() mode

  1. $url='http://t.qq.com';
  2. $handle=fopen($url,"rb");
  3. $lines_string="";
  4. do{
  5. $data=fread($handle,1024);
  6. if(strlen($data)==0) {
  7. break;
  8. }
  9. $lines_string.=$data;
  10. }while(true);
  11. fclose($handle );
  12. echo htmlspecialchars($lines_string);
Copy code

4. curl method Using curl requires space to enable curl. Method: Modify php.ini under Windows, remove the semicolon in front of extension=php_curl.dll, and copy ssleay32.dll and libeay32.dll to C:WINDOWSsystem32; install the curl extension under Linux.

  1. $url='http://t.qq.com';
  2. $ch=curl_init();
  3. $timeout=5;
  4. curl_setopt($ch, CURLOPT_URL, $url );
  5. curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
  6. curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
  7. $lines_string=curl_exec($ch);
  8. curl_close($ch);
  9. echo htmlspecialchars($lines_string);
Copy code

5. fsockopen() function socket mode Whether the socket mode can be executed correctly is also related to the server settings. Specifically, you can use phpinfo to check which communication protocols are enabled on the server. For example, my local php socket does not enable http, so I can only use udp to test it.

  1. $fp = fsockopen("udp://127.0.0.1", 13, $errno, $errstr);
  2. if (!$fp) {
  3. echo "ERROR: $errno - $errstr
    n"
  4. } else {
  5. fwrite($fp, "n")
  6. echo fread($fp, 26)
  7. fclose($fp)
  8. }
Copy code


Statement:
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn