首頁  >  文章  >  後端開發  >  使用file_get_content系列函數和使用curl系列函數來擷取圖片的效能對比

使用file_get_content系列函數和使用curl系列函數來擷取圖片的效能對比

WBOY
WBOY原創
2016-08-08 09:27:441138瀏覽

  由於公司的一個汽車網站的後台的汽車內容都是主要是來自與汽車之家的,編輯的同事們必須天天手動去對著汽車之家來添加汽車,實在是太蛋疼了。於是乎,為了改變這個狀況,身為一個開發碼農,我的任務就來了。 。 。那就是準備做一個功能,只要貼上對應的汽車之家的網址url就能對這些數據進行自動填充到我們後台的表單中,目前基本的填充都實現了,但是還是沒有能夠把對應的汽車相冊採集進來。

  採集圖片的功能我以前也做過,但是汽車之家大部分的汽車都有挺多圖片的,開始的時候,我打算使用以前的採集圖片的辦法,也就是使用file_get_content獲取url對應的內容,然後再配對到圖片的位址,再使用file_get_content取得這些圖片url的內容,再載入到本地去,程式碼如下:

<?<span>php
</span><span>header</span>('Content-type:text/html;charset=utf-8'<span>);
</span><span>set_time_limit</span>(0<span>);

</span><span>class</span><span> runtime  
{  
    </span><span>var</span> <span>$StartTime</span> = 0<span>;  
    </span><span>var</span> <span>$StopTime</span> = 0<span>;  
   
    </span><span>function</span><span> get_microtime()  
    {  
        </span><span>list</span>(<span>$usec</span>, <span>$sec</span>) = <span>explode</span>(' ', <span>microtime</span><span>());  
        </span><span>return</span> ((<span>float</span>)<span>$usec</span> + (<span>float</span>)<span>$sec</span><span>);  
    }  
   
    </span><span>function</span><span> start()  
    {  
        </span><span>$this</span>->StartTime = <span>$this</span>-><span>get_microtime();  
    }  
   
    </span><span>function</span><span> stop()  
    {  
        </span><span>$this</span>->StopTime = <span>$this</span>-><span>get_microtime();  
    }  
   
    </span><span>function</span><span> spent()  
    {  
        </span><span>return</span> <span>round</span>((<span>$this</span>->StopTime - <span>$this</span>->StartTime) * 1000, 1<span>);  
    }  
   
}  

</span><span>$runtime</span>= <span>new</span><span> runtime();  
</span><span>$runtime</span>-><span>start();  

</span><span>$url</span> = 'http://car.autohome.com.cn/pic/series-s15306/289.html#pvareaid=102177'<span>;
</span><span>$rs</span> = <span>file_get_contents</span>(<span>$url</span><span>);
</span><span>//</span><span> echo $rs;exit;</span>
<span>preg_match_all</span>('/(\/pic\/series-s15306\/289-\d+\.html)/', <span>$rs</span>, <span>$urlArr</span><span>);

</span><span>$avalie</span> = <span>array_unique</span>(<span>$urlArr</span>[0<span>]);
</span><span>$count</span> = <span>array</span><span>();
</span><span>foreach</span> (<span>$avalie</span> <span>as</span> <span>$key</span> => <span>$ul</span><span>) {
   </span><span>$pattern</span> = '/<img src="(http:\/\/car1\.autoimg\.cn\/upload\/\d+\/\d+\/\d+\/.*?\.jpg)"/'<span>;
   </span><span>preg_match_all</span>(<span>$pattern</span>, <span>file_get_contents</span>('http://car.autohome.com.cn'.<span>$ul</span>), <span>$imgSrc</span><span>);
   </span><span>$count</span> = <span>array_merge</span>(<span>$count</span>, <span>$imgSrc</span>[1<span>]);
}


</span><span>foreach</span>(<span>$count</span> <span>as</span> <span>$k</span>=><span>$v</span><span>) {
  </span><span>$data</span>[<span>$k</span>] = <span>file_get_contents</span>(<span>$v</span><span>);
}

</span><span>foreach</span>(<span>$data</span> <span>as</span> <span>$k</span>=><span>$v</span><span>) {
  </span><span>file_put_contents</span>('./pic2/'.<span>time</span>().'_'.<span>rand</span>(1, 10000).'.jpg', <span>$v</span><span>);
}

</span><span>$runtime</span>-><span>stop();  
</span><span>echo</span> "页面执行时间: ".<span>$runtime</span>->spent()." 毫秒"; 

  結果發現,這種方法少圖片還好,圖片多了,那是相當的卡。 。就本地測試也比較難跑,更不如說到時候上線了。百度之後,我採用了curl的辦法來下載圖片,經過測試後的確有所改善,但是感覺還是有點慢,要是php有多線程那有多好。 。 。

  又經過一番折騰和找資料,發現php的curl庫其實還是可以模擬多線程的,那就是使用curl_multi_*系列的函數,經過改寫,程式碼又變成了這樣:

  

<?<span>php
</span><span>header</span>('Content-type:text/html;charset=utf-8'<span>);
</span><span>set_time_limit</span>(0<span>);

</span><span>class</span><span> runtime  
{  
    </span><span>var</span> <span>$StartTime</span> = 0<span>;  
    </span><span>var</span> <span>$StopTime</span> = 0<span>;  
   
    </span><span>function</span><span> get_microtime()  
    {  
        </span><span>list</span>(<span>$usec</span>, <span>$sec</span>) = <span>explode</span>(' ', <span>microtime</span><span>());  
        </span><span>return</span> ((<span>float</span>)<span>$usec</span> + (<span>float</span>)<span>$sec</span><span>);  
    }  
   
    </span><span>function</span><span> start()  
    {  
        </span><span>$this</span>->StartTime = <span>$this</span>-><span>get_microtime();  
    }  
   
    </span><span>function</span><span> stop()  
    {  
        </span><span>$this</span>->StopTime = <span>$this</span>-><span>get_microtime();  
    }  
   
    </span><span>function</span><span> spent()  
    {  
        </span><span>return</span> <span>round</span>((<span>$this</span>->StopTime - <span>$this</span>->StartTime) * 1000, 1<span>);  
    }  
   
}  

</span><span>$runtime</span>= <span>new</span><span> runtime();  
</span><span>$runtime</span>-><span>start();  


</span><span>$url</span> = 'http://car.autohome.com.cn/pic/series-s15306/289.html#pvareaid=102177'<span>;
</span><span>$rs</span> = <span>file_get_contents</span>(<span>$url</span><span>);
</span><span>preg_match_all</span>('/(\/pic\/series-s15306\/289-\d+\.html)/', <span>$rs</span>, <span>$urlArr</span><span>);

</span><span>$avalie</span> = <span>array_unique</span>(<span>$urlArr</span>[0<span>]);
</span><span>$count</span> = <span>array</span><span>();
</span><span>foreach</span> (<span>$avalie</span> <span>as</span> <span>$key</span> => <span>$ul</span><span>) {
   </span><span>$pattern</span> = '/<img src="(http:\/\/car1\.autoimg\.cn\/upload\/\d+\/\d+\/\d+\/.*?\.jpg)"/'<span>;
   </span><span>preg_match_all</span>(<span>$pattern</span>, <span>file_get_contents</span>('http://car.autohome.com.cn'.<span>$ul</span>), <span>$imgSrc</span><span>);
   </span><span>$count</span> = <span>array_merge</span>(<span>$count</span>, <span>$imgSrc</span>[1<span>]);
}

</span><span>$handle</span> =<span> curl_multi_init();

</span><span>foreach</span>(<span>$count</span> <span>as</span> <span>$k</span> => <span>$v</span><span>) {
  </span><span>$curl</span>[<span>$k</span>] = curl_init(<span>$v</span><span>);
  curl_setopt(</span><span>$curl</span>[<span>$k</span>], CURLOPT_RETURNTRANSFER, 1<span>);
  curl_setopt(</span><span>$curl</span>[<span>$k</span>], CURLOPT_HEADER, 0<span>);
  curl_setopt(</span><span>$curl</span>[<span>$k</span>], CURLOPT_TIMEOUT, 30<span>);
  curl_multi_add_handle (</span><span>$handle</span>, <span>$curl</span>[<span>$k</span><span>]);
}

</span><span>$active</span> = <span>null</span><span>;

</span><span>do</span><span> {
    </span><span>$mrc</span> = curl_multi_exec(<span>$handle</span>, <span>$active</span><span>);
} </span><span>while</span> (<span>$mrc</span> ==<span> CURLM_CALL_MULTI_PERFORM);

</span><span>while</span> (<span>$active</span> && <span>$mrc</span> ==<span> CURLM_OK) {
    // 这句在php5.3以后的版本很关键,因为没有这句,可能curl_multi_select可能会永远返回-1,这样就永远死在循环里了
    </span><span>while</span> (curl_multi_exec(<span>$handle</span>, <span>$active</span>) ===<span> CURLM_CALL_MULTI_PERFORM);

    </span><span>if</span> (curl_multi_select(<span>$handle</span>) != -1<span>) {
        </span><span>do</span><span> {
            </span><span>$mrc</span> = curl_multi_exec(<span>$handle</span>, <span>$active</span><span>);
        } </span><span>while</span> (<span>$mrc</span> ==<span> CURLM_CALL_MULTI_PERFORM);
    }
}

</span><span>foreach</span> (<span>$curl</span> <span>as</span> <span>$k</span> => <span>$v</span><span>) {
    </span><span>if</span> (curl_error(<span>$curl</span>[<span>$k</span>]) == ""<span>) {
        </span><span>$data</span>[<span>$k</span>] = curl_multi_getcontent(<span>$curl</span>[<span>$k</span><span>]);
    }
    curl_multi_remove_handle(</span><span>$handle</span>, <span>$curl</span>[<span>$k</span><span>]);
    curl_close(</span><span>$curl</span>[<span>$k</span><span>]);
}

</span><span>foreach</span>(<span>$data</span> <span>as</span> <span>$k</span>=><span>$v</span><span>) {
    </span><span>$file</span> = <span>time</span>().'_'.<span>rand</span>(1000, 9999).'.jpg'<span>;
    </span><span>file_put_contents</span>('./pic3/'.<span>$file</span>, <span>$v</span><span>); 
}

curl_multi_close(</span><span>$handle</span><span>);

</span><span>$runtime</span>-><span>stop();  
</span><span>echo</span> "页面执行时间: ".<span>$runtime</span>->spent()." 毫秒"; 

  好了,多線程的採集真是非常酸爽,然後通過一系列的測試和對比,5次測試,curl多線程有4次是快於file_get_content的,而且時間還是file_get_content的3~5倍,總結起來,以後採集都盡量使用這種辦法,提高效率不在話下。

以上就介紹了使用file_get_content系列函數和使用curl系列函數來擷取圖片的效能對比,包括了方面的內容,希望對PHP教學有興趣的朋友有所幫助。

陳述:
本文內容由網友自願投稿,版權歸原作者所有。本站不承擔相應的法律責任。如發現涉嫌抄襲或侵權的內容,請聯絡admin@php.cn
上一篇:PHP 定義下一篇:PHP 定義