Home >Backend Development >PHP Tutorial > curl多线程采撷不是采集不到就是采不全

curl多线程采撷不是采集不到就是采不全

WBOY
WBOYOriginal
2016-06-13 12:47:081100browse

curl多线程采集不是采集不到就是采不全?
如题,代码如下,请问如何解决?

<br>
/** <br>
     * curl 多线程 <br>
     *  <br>
     * @param array $array 并行网址 <br>
     * @param int $timeout 超时时间<br>
     * @return array <br>
     */<br>
 function Curl_http($array,$timeout){<br>
    $res = array();<br>
    $mh = curl_multi_init();//创建多个curl语柄<br>
    $startime = getmicrotime();<br>
    foreach($array as $k=>$url){<br>
        $conn[$k]=curl_init($url);<br>
        curl_setopt($conn[$k], CURLOPT_TIMEOUT, $timeout);//设置超时时间<br>
        curl_setopt($conn[$k], CURLOPT_USERAGENT, 'Mozilla/5.0 (compatible; MSIE 5.01; Windows NT 5.0)');<br>
        curl_setopt($conn[$k], CURLOPT_MAXREDIRS, 7);//HTTp定向级别<br>
        curl_setopt($conn[$k], CURLOPT_HEADER, 0);//这里不要header,加块效率<br>
        curl_setopt($conn[$k], CURLOPT_FOLLOWLOCATION, 1); // 302 redirect<br>
        curl_setopt($conn[$k],CURLOPT_RETURNTRANSFER,1);<br>
        curl_multi_add_handle ($mh,$conn[$k]);<br>
    }<br>
     //防止死循环耗死cpu 这段是根据网上的写法<br>
        do {<br>
            $mrc = curl_multi_exec($mh,$active);//当无数据,active=true<br>
        } while ($mrc == CURLM_CALL_MULTI_PERFORM);//当正在接受数据时<br>
        while ($active and $mrc == CURLM_OK) {//当无数据时或请求暂停时,active=true<br>
            if (curl_multi_select($mh) != -1) {<br>
                do {<br>
                    $mrc = curl_multi_exec($mh, $active);<br>
                } while ($mrc == CURLM_CALL_MULTI_PERFORM);<br>
            }<br>
        }<br>
    foreach ($array as $k => $url) {<br>
          curl_error($conn[$k]);<br>
          $res[$k]=curl_multi_getcontent($conn[$k]);//获得返回信息<br>
          $header[$k]=curl_getinfo($conn[$k]);//返回头信息<br>
          curl_close($conn[$k]);//关闭语柄<br>
          curl_multi_remove_handle($mh  , $conn[$k]);   //释放资源  <br>
        }<br>
        curl_multi_close($mh);<br>
        $endtime = getmicrotime();<br>
        $diff_time = $endtime - $startime; <div class="clear">
                 
              
              
        
            </div>
Statement:
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn