Home >Backend Development >PHP Tutorial >php stream_context_create 无法作用于 stream_socket_client

php stream_context_create 无法作用于 stream_socket_client

WBOY
WBOYOriginal
2016-06-06 20:45:581367browse

想通过本地代理,抓取远程网页内容,代码如下:

<code class="lang-php"><?php $options = array(
    'http'=>array(
        'proxy'=>'tcp://192.168.1.108:8087',
        'request_fulluri '=>true,
        "method"  => "GET", 
        "timeout" => 2,
    ),
);

$context = stream_context_create($options);

$fp = stream_socket_client("tcp://www.bigxu.com:80", $errno, $errstr, 30,STREAM_CLIENT_CONNECT,$context);
// print_r(stream_context_get_options($fp)); exit;

if (!$fp) {
    echo "$errstr ($errno)<br>\n";
} else {
    fwrite($fp, "GET / HTTP/1.0\r\nHost: www.bigxu.com\r\nAccept: */*\r\n\r\n");
    while (!feof($fp)) {
        echo fgets($fp, 1024);
    }
    fclose($fp);
}
?>
</code>

php file.php

nginx访问日志是:
15.196.206.102 [26/Apr/2014:12:04:45 +0800] http://www.bigxu.com/ 200 20630 0.241 "-" "-"

15.196.206.102 是我的本机IP.
$context没有起到作用。
代理是绝对可以用的。
因为通过下面代码,$context会起作用

<code class="lang-php">$options = array(
     'http'=>array(
         'proxy'=>'tcp://192.168.1.108:8087',
         'request_fulluri '=>true,
         "method"  => "GET",
         "timeout" => 2,
     ),
 );
 $context = stream_context_create($options);

 if ( $fp = fopen("http://www.bigxu.com", 'r', false, $context) ) {
     print "well done";
    while (!feof($fp)) {
         echo fgets($fp, 1024);
     }
 }

</code>

php file.php
bigxu.com nginx访问日志是:
8.35.201.32 [26/Apr/2014:12:03:03 +0800] http://www.bigxu.com/ 200 7070 0.122 "-" "AppEngine-Google; (+http://code.google.com/appengine; appid: s~goagent0527)"
8.35.201.32 是我的代理IP

比较大型的爬虫项目,我肯定会用stream_socket_client来连接,大家帮我看一下,这个函数,我怎么用错了呢?

回复内容:

想通过本地代理,抓取远程网页内容,代码如下:

<code class="lang-php"><?php $options = array(
    'http'=>array(
        'proxy'=>'tcp://192.168.1.108:8087',
        'request_fulluri '=>true,
        "method"  => "GET", 
        "timeout" => 2,
    ),
);

$context = stream_context_create($options);

$fp = stream_socket_client("tcp://www.bigxu.com:80", $errno, $errstr, 30,STREAM_CLIENT_CONNECT,$context);
// print_r(stream_context_get_options($fp)); exit;

if (!$fp) {
    echo "$errstr ($errno)<br>\n";
} else {
    fwrite($fp, "GET / HTTP/1.0\r\nHost: www.bigxu.com\r\nAccept: */*\r\n\r\n");
    while (!feof($fp)) {
        echo fgets($fp, 1024);
    }
    fclose($fp);
}
?>
</code>

php file.php

nginx访问日志是:
15.196.206.102 [26/Apr/2014:12:04:45 +0800] http://www.bigxu.com/ 200 20630 0.241 "-" "-"

15.196.206.102 是我的本机IP.
$context没有起到作用。
代理是绝对可以用的。
因为通过下面代码,$context会起作用

<code class="lang-php">$options = array(
     'http'=>array(
         'proxy'=>'tcp://192.168.1.108:8087',
         'request_fulluri '=>true,
         "method"  => "GET",
         "timeout" => 2,
     ),
 );
 $context = stream_context_create($options);

 if ( $fp = fopen("http://www.bigxu.com", 'r', false, $context) ) {
     print "well done";
    while (!feof($fp)) {
         echo fgets($fp, 1024);
     }
 }

</code>

php file.php
bigxu.com nginx访问日志是:
8.35.201.32 [26/Apr/2014:12:03:03 +0800] http://www.bigxu.com/ 200 7070 0.122 "-" "AppEngine-Google; (+http://code.google.com/appengine; appid: s~goagent0527)"
8.35.201.32 是我的代理IP

比较大型的爬虫项目,我肯定会用stream_socket_client来连接,大家帮我看一下,这个函数,我怎么用错了呢?

解决了,很难想像,我居然在一个日本的网站找到了答案,所以不要随意抵制日本知识,哈哈!经过测试完全可以实现,你看看:

<code><?php $sock = stream_socket_client('tcp://127.0.0.1:8123', $errno, $errstr, 10, STREAM_CLIENT_CONNECT);  
  
if(!$sock) exit();  
  
$request = array();  
$request[] = 'GET http://www.example.com HTTP/1.1';  
$request[] = 'Host: www.example.com';  
$request[] = 'User-Agent: TestClient';  
  
if(!fwrite($sock, implode("\r\n", $request)."\r\n\r\n")){  
    exit();  
}  
  
$response = '';  
while(!feof($sock)){  
    $response .= fgets($sock, 4096);  
}  
  
echo $response;  
  
fclose($sock);  
  
?></code>

原文地址:http://pe5974.sakura.ne.jp/contents/proxy-https.php

说到采集,首先应该想到的是php的curl函数
最好的办法就是模拟爬虫(比如:百度蜘蛛爬虫或者google蜘蛛爬虫),同样也支持代理配置
通过爬虫模拟浏览器的head请求,没有什么抓去不到(理论上只要能通过浏览器请求到的数据,不管是要登录还是不要登录都是可以抓去到内容的)

不确定是不是这个代理有问题,配置个自己的代理看一下?
或者跟进一下? stream_socket_client 空了继续弄吧

Statement:
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn