For more complex http network operations


QueryListThe built-in network operation is very simple, QueryList focuses on DOM selection; for more complex network operations, you can choose to use Request extension , it can be easily implemented: carrying cookies, forging origins, forging browsers and other functions, but if you feel that it still cannot meet your needs, here are a few solutions for reference:

1. Use curl to encapsulate an http request yourself
    ##Example:

    function getHtml($url)
{
            $ch = curl_init();
            curl_setopt($ch, CURLOPT_URL, $url);
            curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
            curl_setopt($ch, CURLOPT_AUTOREFERER, true);
            curl_setopt($ch, CURLOPT_REFERER, $url);
            curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
            $result = curl_exec($ch);
            curl_close($ch);
            return $result;
}
$rules = array(
//采集规则
);
//获取页面源码
$html = getHtml('http://xxx.com');
//采集
$data = QueryList::Query($html,$rules)->data;

    QueryList can be used seamlessly with any third-party http package. The following takes the guzzlehttp/guzzle package as an example. Guzzle is a PHP HTTP client. , used to send requests easily and integrated into our WEB services.

    GuzzleChinese manual: http://guzzle-cn.readthedocs.io/zh_CN/latest/

    Installation
    //安装QueryList
composer require jaeger/querylist
//安装Guzzle
composer require guzzlehttp/guzzle

    Use 
    <?php
require 'vendor/autoload.php';
//实例化一个Http客户端
$client = new GuzzleHttp\Client(['base_uri' => 'https://phphub.org']);
$jar = new \GuzzleHttp\Cookie\CookieJar();
//发送一个Http请求
$response = $client->request('GET', '/categories/6', [
   'headers' => [
       'User-Agent' => 'testing/1.0',
       'Accept'     => 'application/json',
       'X-Foo'      => ['Bar', 'Baz']
   ],
   'form_params' => [
       'foo' => 'bar',
       'baz' => ['hi', 'there!']
   ],
   // 'cookies' => $jar,
   'timeout' => 3.14,
   // 'proxy' => 'tcp://localhost:8125',
   // 'cert' => ['/path/server.pem', 'password'],
]);
$body = $response->getBody();
//获取到页面源码
$html = (string)$body;
//采集规则
$rules = array(
   //文章标题
   'title' => ['.media-heading a','text'],
   //文章链接
   'link' => ['.media-heading a','href'],
   //文章作者名
   'author' => ['.img-thumbnail','alt']
);
//列表选择器
$rang = '.topic-list>li';
//采集
$data = \QL\QueryList::Query($html,$rules,$rang)->data;
//查看采集结果
print_r($data);

    Result:

    Array
(
    [0] => Array
        (
            [title] => 好友动态的实现原理
            [link] => https://phphub.org/topics/2750
            [author] => luo975974740
        )
    [1] => Array
        (
            [title] => 打造完美的 Ubuntu16.04 开发环境【持续更新】
            [link] => https://phphub.org/topics/2723
            [author] => liuwantao
        )
    //省略........
     [19] => Array
        (
            [title] => [Laravel 5.3 新功能] 10. 全文搜索方案 Laravel Scout 介绍
            [link] => https://phphub.org/topics/2673
            [author] => monkey
        )
)

    ←