欢迎访问个人原创地址: http://www.phpthinking.com/archives/468
使用PHP的cURL库可以简单和有效地去抓网页。你只需要运行一个脚本,然后分析一下你所抓取的网页,然后就可以以程序的方式得到你想要的数据了。无论是你想从从一个链接上取部分数据,或是取一个XML文件并把其导入数据库,那怕就是简单的获取网页内容,cURL 是一个功能强大的PHP库。本文主要讲述如果使用这个PHP库。
启用 cURL 设置
首先,我们得先要确定我们的PHP是否开启了这个库,你可以通过使用php_info()函数来得到这一信息。
|
phpinfo(); ?> |
如果你可以在网页上看到下面的输出,那么表示cURL库已被开启。
如果你看到的话,那么你需要设置你的PHP并开启这个库。如果你是在Windows平台下,那么非常简单,你需要改一改你的php.ini文件的设置,找到php_curl.dll,并取消前面的分号注释就行了。如下所示:
|
//取消下在的注释 extension=php_curl.dll |
如果你是在Linux下面,那么,你需要重新编译你的PHP了,编辑时,你需要打开编译参数??在configure命令上加上“?with-curl” 参数。
一个小示例
如果一切就绪,下面是一个小例程:
|
// 初始化一个 cURL 对象 $curl = curl_init();
// 设置你需要抓取的URL curl_setopt( $curl , CURLOPT_URL, ' http://coolshell.cn' );
// 设置header curl_setopt( $curl , CURLOPT_HEADER, 1);
// 设置cURL 参数,要求结果保存到字符串中还是输出到屏幕上。 curl_setopt( $curl , CURLOPT_RETURNTRANSFER, 1);
// 运行cURL,请求网页 $data = curl_exec( $curl );
// 关闭URL请求 curl_close( $curl );
// 显示获得的数据 var_dump( $data ); |
如何POST数据
上面是抓取网页的代码,下面则是向某个网页POST数据。假设我们有一个处理表单的网址http://www.example.com/sendSMS.php,其可以接受两个表单域,一个是电话号码,一个是短信内容。
|
$phoneNumber = '13912345678' ; $message = 'This message was generated by curl and php' ; $curlPost = 'pNUMBER=' . urlencode( $phoneNumber ) . '&MESSAGE=' . urlencode( $message ) . '&SUBMIT=Send' ; $ch = curl_init(); curl_setopt( $ch , CURLOPT_URL, ' http://www.example.com/sendSMS.php' ); curl_setopt( $ch , CURLOPT_HEADER, 1); curl_setopt( $ch , CURLOPT_RETURNTRANSFER, 1); curl_setopt( $ch , CURLOPT_POST, 1); curl_setopt( $ch , CURLOPT_POSTFIELDS, $curlPost ); $data = curl_exec(); curl_close( $ch ); ?> |
从上面的程序我们可以看到,使用CURLOPT_POST设置HTTP协议的POST方法,而不是GET方法,然后以CURLOPT_POSTFIELDS设置POST的数据。
关于代理服务器
下面是一个如何使用代理服务器的示例。请注意其中高亮的代码,代码很简单,我就不用多说了。
|
$ch = curl_init(); curl_setopt( $ch , CURLOPT_URL, ' http://www.example.com' ); curl_setopt( $ch , CURLOPT_HEADER, 1); curl_setopt( $ch , CURLOPT_RETURNTRANSFER, 1); curl_setopt( $ch , CURLOPT_HTTPPROXYTUNNEL, 1); curl_setopt( $ch , CURLOPT_PROXY, 'fakeproxy.com:1080' ); curl_setopt( $ch , CURLOPT_PROXYUSERPWD, 'user:password' ); $data = curl_exec(); curl_close( $ch ); ?> |
关于SSL和Cookie
关于SSL也就是HTTPS协议,你只需要把CURLOPT_URL连接中的http://变成https://就可以了。当然,还有一个参数叫CURLOPT_SSL_VERIFYHOST可以设置为验证站点。
关于Cookie,你需要了解下面三个参数:
HTTP服务器认证
最后,我们来看一看HTTP服务器认证的情况。
|
$ch = curl_init(); curl_setopt( $ch , CURLOPT_URL, ' http://www.example.com' ); curl_setopt( $ch , CURLOPT_RETURNTRANSFER, 1); curl_setopt( $ch , CURLOPT_HTTPAUTH, CURLAUTH_BASIC); curl_setopt(CURLOPT_USERPWD, '[username]:[password]' )
$data = curl_exec(); curl_close( $ch ); ?> |

Calculating the total number of elements in a PHP multidimensional array can be done using recursive or iterative methods. 1. The recursive method counts by traversing the array and recursively processing nested arrays. 2. The iterative method uses the stack to simulate recursion to avoid depth problems. 3. The array_walk_recursive function can also be implemented, but it requires manual counting.

In PHP, the characteristic of a do-while loop is to ensure that the loop body is executed at least once, and then decide whether to continue the loop based on the conditions. 1) It executes the loop body before conditional checking, suitable for scenarios where operations need to be performed at least once, such as user input verification and menu systems. 2) However, the syntax of the do-while loop can cause confusion among newbies and may add unnecessary performance overhead.

Efficient hashing strings in PHP can use the following methods: 1. Use the md5 function for fast hashing, but is not suitable for password storage. 2. Use the sha256 function to improve security. 3. Use the password_hash function to process passwords to provide the highest security and convenience.

Implementing an array sliding window in PHP can be done by functions slideWindow and slideWindowAverage. 1. Use the slideWindow function to split an array into a fixed-size subarray. 2. Use the slideWindowAverage function to calculate the average value in each window. 3. For real-time data streams, asynchronous processing and outlier detection can be used using ReactPHP.

The __clone method in PHP is used to perform custom operations when object cloning. When cloning an object using the clone keyword, if the object has a __clone method, the method will be automatically called, allowing customized processing during the cloning process, such as resetting the reference type attribute to ensure the independence of the cloned object.

In PHP, goto statements are used to unconditionally jump to specific tags in the program. 1) It can simplify the processing of complex nested loops or conditional statements, but 2) Using goto may make the code difficult to understand and maintain, and 3) It is recommended to give priority to the use of structured control statements. Overall, goto should be used with caution and best practices are followed to ensure the readability and maintainability of the code.

In PHP, data statistics can be achieved by using built-in functions, custom functions, and third-party libraries. 1) Use built-in functions such as array_sum() and count() to perform basic statistics. 2) Write custom functions to calculate complex statistics such as medians. 3) Use the PHP-ML library to perform advanced statistical analysis. Through these methods, data statistics can be performed efficiently.

Yes, anonymous functions in PHP refer to functions without names. They can be passed as parameters to other functions and as return values of functions, making the code more flexible and efficient. When using anonymous functions, you need to pay attention to scope and performance issues.


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

VSCode Windows 64-bit Download
A free and powerful IDE editor launched by Microsoft

Notepad++7.3.1
Easy-to-use and free code editor

SAP NetWeaver Server Adapter for Eclipse
Integrate Eclipse with SAP NetWeaver application server.

SublimeText3 Mac version
God-level code editing software (SublimeText3)

ZendStudio 13.5.1 Mac
Powerful PHP integrated development environment
