Introduction to php curl function and crawling 163 mailing list_PHP tutorial

If you see this, then you need to set up your php tutorial and enable this library. If you are on the Windows platform, it is very simple. You need to change the settings of your php.ini file, find php_curl.dll, and cancel the previous semicolon comment. As shown below:
//Uncomment the following
extension=php_curl.dll

If you are under Linux, then you need to recompile your php when editing. , you need to turn on the compilation parameters - add the "-with-curl" parameter to the configure command.
　A small example
If everything is ready, here is a small routine:
Copy the code as follows:

// Initialize a curl object
$curl = curl_init();
// Set the url you need to crawl
curl_setopt($curl, curlopt_url, 'http://jb51.net');
// Set header
curl_setopt($curl, curlopt_header, 1);
//Set the curl parameters and ask whether the results are saved in a string or output to the screen.
curl_setopt($curl, curlopt_returntransfer, 1);
// Run curl and request the web page
$data = curl_exec($curl);
// Close url request
curl_close($curl ; Suppose we have a form processing URL http://www.example.com/sendsms.php, which can accept two form fields, one is a phone number, and the other is text message content.
Copy the code as follows:

$phonenumber　='13912345678';
$message　='thisMessagewasgeneratedbycurlandphp';
$curlpost = 'pnumber=' .urlencode($phonenumber) .'&message=' .urlencode($message) .'&submit=send'; $ch = curl_init();chain link fencing

curl_setopt($ch, curlopt_url,'http://www.example.com/sendsms.php');
curl_setopt($ch, curlopt_header, 1);
curl_setopt($ch, curlopt_returntransfer, 1);
curl_setopt( $ch, curlopt_post, 1);
curl_setopt($ch, curlopt_postfields, $curlpost);
$data = curl_exec();
curl_close($ch);
From the above program, we can see that curlopt_post is used to set the post method of the http protocol instead of the get method, and then curlopt_postfields is used to set the post data.
　About proxy server
　The following is an example of how to use a proxy server. Please pay attention to the highlighted code. The code is very simple, so I don’t need to say more.
Copy the code as follows:

$ch　= curl_init();

curl_setopt($ch, curlopt_url, 'http://www.example.com') ;
curl_setopt($ch, curlopt_header, 1);
curl_setopt($ch, curlopt_returntransfer, 1);
curl_setopt($ch, curlopt_httpproxytunnel, 1); 'fakeproxy.com:1080');

curl_setopt($ch, curlopt_proxyuserpwd, 'user:password');

$data = curl_exec();
curl_close($ch);
?> ;



About ssl and cookies
Regarding ssl, which is the https tutorial protocol, for gas generators, you only need to change http:// in the curlopt_url connection to https:// That's it. Of course, there is also a parameter called curlopt_ssl_verifyhost that can be set to the verification site.
Regarding cookies, you need to know the following three parameters:
curlopt_cookie, set a cookie in the face-to-face session
curlopt_cookiejar, save a cookie when the session ends

curlopt_cookiefile, cookie file.

HTTP server authentication
Finally, let’s take a look at http server authentication.
Copy the code as follows:

$ch = curl_init();
curl_setopt($ch, curlopt_url, 'http://www.example.com') ;
curl_setopt($ch, curlopt_returntransfer, 1);
curl_setopt($ch, curlopt_httpauth, curlauth_basic);
curl_setopt(curlopt_userpwd, '[username]:[password]')
$data = curl_exec();

curl_close($ch);

?>


Look at a code that uses curl to grab a list of 163 email addresses

To put it bluntly, curl technology is Simulate browser actions to achieve page crawling or form submission. Many interesting functions can be achieved through this technology.
Copy the code The code is as follows:

error_reporting(0);
//Email username (without @163.com suffix)
$user = 'papatata_test';
//Email password
$pass = '000000';
//Target email address
//$mail_addr = uenucom@163.com';
//Login
$url = 'http://reg .163.com/logins.jsp tutorial?type=1&url=http://entry.mail.163.com/coremail/fcg/ntesdoor2?lightweight%3d1%26verifycookie%3d1%26language%3d-1%26style%3d- 1';
$ch = curl_init($url);
//Create a temporary file to store cookie information
$cookie = tempnam('.','~');
$referer_login = 'http://mail.163.com';
//The return result is stored in a variable instead of the default direct output
curl_setopt($ch, curlopt_returntransfer, true);
curl_setopt ($ch, curlopt_header, true);
curl_setopt($ch, curlopt_connecttimeout, 120);
curl_setopt($ch, curlopt_post, true);
curl_setopt($ch, curlopt_referer, $referer_login);
$fields_post = array(
'username'=> $user,
'password'=> $pass,
'verifycookie'=>1,
'style'=> ;-1,
'product'=> 'mail163',
'seltype'=>-1,
'secure'=>'on'
);
$ headers_login = array(
'user-agent' => 'mozilla/5.0 (windows; u; windows nt 5.1; zh-cn; rv:1.9) gecko/2008052906 firefox/3.0',
'referer' => 'http://www.163.com'
);
$fields_string = '';
foreach($fields_post as $key => $value)
{
$fields_string .= $key . '=' . $value . '&';
}
$fields_string = rtrim($fields_string , '&');
curl_setopt($ch, curlopt_cookiesession, true ;
curl_setopt($ch, curlopt_post, count($fields));
curl_setopt($ch, curlopt_postfields, $fields_string);
$result= curl_exec($ch);
curl_close($ch );
//Jump
$url='http://entry.mail.163.com/coremail/fcg/ntesdoor2?lightweight=1&verifycookie=1&language=-1&style=-1&username=loki_wuxi';
$ch = curl_init($url);
$headers = array(
'user-agent' => 'mozilla/5.0 (windows; u; windows nt 5.1; zh-cn; rv:1.9 ) gecko/2008052906 firefox/3.0'
);
curl_setopt($ch, curlopt_returntransfer, true);
curl_setopt($ch, curlopt_header, true);
curl_setopt($ch, curlopt_connecttimeout, 120 );
curl_setopt($ch, curlopt_post, true);
curl_setopt($ch, curlopt_httpheader, $headers);
//Send the previously saved cookie information to the server together
curl_setopt ($ch, curlopt_cookiefile, $cookie);
curl_setopt($ch, curlopt_cookiejar, $cookie);
$result = curl_exec($ch);
curl_close($ch);
// Get sid
preg_match('/sid=[^"].*/', $result, $location);
$sid = substr($location[0], 4, -1);
//file_put_contents('./result.txt', $sid);
//Address book address
$url='http://g4a30.mail.163.com/jy3/address/addrlist.jsp ?sid='.$sid.'&gid=all';
$ch = curl_init($url);
$headers = array(
'user-agent' => 'mozilla/5.0 ( windows; u; windows nt 5.1; zh-cn; rv:1.9) gecko/2008052906 firefox/3.0'
);
curl_setopt($ch, curlopt_returntransfer, true);
curl_setopt($ch, curlopt_header , true);
curl_setopt($ch, curlopt_connecttimeout, 120);
curl_setopt($ch, curlopt_post, true);
curl_setopt($ch, curlopt_httpheader, $headers);
curl_setopt($ ch, curlopt_cookiefile, $cookie);
curl_setopt($ch, curlopt_cookiejar, $cookie);
$result = curl_exec($ch);
curl_close($ch);
//file_put_contents( './result.txt', $result);
unlink($cookie);
//Start crawling content
preg_match_all('/]*>(.*?) ]*>(.*?)< ;/a>/i', $result,$infos,preg_set_order);
//1: Name 2: Email
print_r($infos);
?>

http://www.bkjia.com/PHPjc/444781.html

www.bkjia.comtruehttp: //www.bkjia.com/PHPjc/444781.htmlTechArticleIf you see this, then you need to set up your php tutorial and enable this library. If you are on the Windows platform, it is very simple. You need to change the settings of your php.ini file...