Use Crawler component to analyze HTML in laravel

This article mainly introduces the use of Symfony's Crawler component to analyze HTML in laravel. Friends in need can refer to it

The full name of Crawler is DomCrawler, which is a component of the Symfony framework. What is outrageous is that DomCrawler does not have Chinese documentation, and Symfony has not translated this part, so development using DomCrawler can only be explored bit by bit. Now I will summarize the experience in the use process.

The first thing is to install

composer require symfony/dom-crawler
composer require symfony/css-selector

css-seelctor is a css selector. Some functions will be used when selecting nodes with css

The example used in the manual is

use Symfony\Component\DomCrawler\Crawler;
$html = <<<‘HTML‘
Hello World!
Hello Crawler!
HTML;
$crawler = new Crawler($html);
foreach ($crawler as $domElement)
{
var_dump($domElement->nodeName);
}

The printed result is

string ‘html‘ (length=4)

because of this paragraph The nodeName of the html code is html. My English is not good. When I started using it, I thought the program was wrong. . .

In the actual use process, if new Crawler ($html) will have garbled characters, it should be related to the page encoding, so you can use the following method, first initialize the crawler, and then add node

$crawler = new Crawler();
$crawler->addHtmlContent($html);

The second parameter of addHtmlContent is charset, and the default is utf-8.

For other examples, please refer to the official documentation, http://symfony.com/doc/current/components/dom_crawler.html

Record the usages that you have tried little by little at work

filterXPath(string $xpath) method, according to the manual, the parameter of this method is $xpath, and p, p and other blocks are often used.

echo $crawler->filterXPath(‘//body/p‘)->text();
echo $crawler->filterXPath(‘//body/p‘)->last()->text();

The output is the text of the first and next p tag block

var_dump($crawler->filterXPath(‘//body‘)->html());

Output the html within the body

foreach ($crawler->filterXPath(‘//body/p‘) as $i => $node) {
$c = new Crawler($node);
echo $c->filter(‘p‘)->text();
}

filterXPath obtains an array of DOMElement blocks. Each DOMElement block can use a new crawler object to continue parsing

$nodeValues =
$crawler->filterXPath(‘//body/p‘)->each(function (Crawler $node, $i) {
return $node->text();
});

crawler provides each loop and uses closure functions to simplify the code. However, please note that this way of writing $nodeValues ​​results in an array, which requires further processing.

Other usage

##

echo $crawler->filterXPath(‘//body/p‘)->attr(‘class‘);

You can get the value "message" of the class attribute corresponding to the first p tag ”

$crawler->filterXPath(‘//p[@class="样式"]‘)->filter(‘a‘)->attr(‘href‘);
$crawler->filterXPath(‘//p[@class="样式"]‘)->filter(‘a>img‘)->extract(array(‘alt‘, ‘href‘))

The above are some methods of obtaining tag attributes

Filter and filterXPath are different. The manual says css selector. I don’t quite understand. I It is understood that the elements contained in XPath nodes such as p need to be tried in actual development.

Generally speaking, I feel that DomCrawler is easier to use than simple html dom. Maybe it is because I use it more easily.

The above are just the basic functions of Crawler. For more usage, please refer to the functions in the Crawler part of the symfony manual

http://api.symfony.com/3.2/Symfony/Component/DomCrawler/Crawler .html

The main problem with Crawler is that there are too few examples. There are no usage examples in the function manual, so you can only explore it in actual use. . . .

symfony's documentation about DomCrawler, which contains a few examples

http://symfony.com/doc/current/components/dom_crawler.html

The above is the detailed content of Use Crawler component to analyze HTML in laravel. For more information, please follow other related articles on the PHP Chinese website!