How to use PHP Goutte class library for web crawling and data extraction?

How to use PHP Goutte class library for web crawling and data extraction?

Overview:
In the daily development process, we often need to obtain various data from the Internet, such as movie rankings, weather forecasts, etc. Web crawling is one of the common methods to obtain this data. In PHP development, we can use the Goutte class library to implement web crawling and data extraction functions. This article will introduce how to use the PHP Goutte class library for web crawling and data extraction, and attach code examples.

What is Goutte?
Goutte is a PHP class library based on Symfony, specially used for web crawling and data extraction. It's built on top of Symfony's CSS selector component, providing a simple yet powerful way to manipulate web pages. Through Goutte, we can easily perform web crawling, form submission, data extraction and other operations.

Install the Goutte class library:
First, we need to install the Goutte class library through Composer. Open the terminal, enter your project directory, and execute the following command:

composer require fabpot/goutte

After the installation is complete, we can introduce the Goutte class library into the code and start using it.

Web crawling and data extraction examples:
Suppose we want to obtain information about currently popular movies from a movie ranking website, such as movie names, ratings, etc. First, find the URL of your target page. Take Douban movie rankings as an example, the URL is: https://movie.douban.com/chart.

Next, we use Goutte to crawl web pages and extract data. The following is a sample code:

// 引入Goutte类库
require 'vendor/autoload.php';

use GoutteClient;

// 创建一个Goutte客户端实例
$client = new Client();

// 发送GET请求，获取目标网页内容
$crawler = $client->request('GET', 'https://movie.douban.com/chart');

// 使用CSS选择器获取电影列表
$movies = $crawler->filter('.indent table tr')->each(function ($node) {
    // 提取电影名称
    $title = $node->filter('.pl2 a')->text();
    
    // 提取电影评分
    $rating = $node->filter('.star .rating_nums')->text();
    
    // 返回电影信息
    return [
        'title' => $title,
        'rating' => $rating,
    ];
});

// 输出结果
foreach ($movies as $movie) {
    echo $movie['title'] . ' - ' . $movie['rating'] . "
";
}

In the above code, we first create a Client instance of Goutte, and then use the request method to send a GET request to the target web page to obtain the web page content. Next, use a CSS selector to extract the movie list, using the CSS selector '.indent table tr' to represent all eligible elements in the target web page. Finally, we perform some data extraction operations on each movie node, extract the movie name and rating, save them to the result array, and finally print out the results.

Through the above code, we can quickly implement the functions of web crawling and data extraction. Of course, Goutte has more powerful functions, such as form submission, simulated user operations, etc. Readers can explore further as needed.

Summary:
This article introduces how to use the PHP Goutte class library for web crawling and data extraction, and demonstrates the basic usage through code examples. Web crawling and data extraction are very useful in many scenarios, such as data analysis, information collection, etc. Through the Goutte class library, we can easily implement these functions and greatly improve development efficiency. I hope this article will be helpful to readers, and welcome exchanges and discussions.

The above is the detailed content of How to use PHP Goutte class library for web crawling and data extraction?. For more information, please follow other related articles on the PHP Chinese website!