来源:http://www.ido321.com/1158.html 抓取某一个网页中的内容,需要对DOM树进行解析,找到指定节点后,再抓取我们需要的内容,过程有点繁琐。LZ总结了几种常用的、易于实现的网页抓取方式,如果熟悉JQuery选择器,这几种框架会相当简单。 一、Ganon 项目
来源:http://www.ido321.com/1158.html
抓取某一个网页中的内容,需要对DOM树进行解析,找到指定节点后,再抓取我们需要的内容,过程有点繁琐。LZ总结了几种常用的、易于实现的网页抓取方式,如果熟悉JQuery选择器,这几种框架会相当简单。
一、Ganon
项目地址: http://code.google.com/p/ganon/
文档: http://code.google.com/p/ganon/w/list
测试:抓取我的网站首页所有class属性值是focus的div元素,并且输出class值
<span><?php <span>include</span> <span>'ganon.php'</span>; $html = file_get_dom(<span>'http://www.ido321.com/'</span>); <span>foreach</span>($html(<span>'div[class="focus"]'</span>) <span>as</span> $element) { <span>echo</span> $element-><span>class</span>, <span>"<br>\n"</span>; } ?>
结果:
二、phpQuery
项目地址:http://code.google.com/p/phpquery/
文档:https://code.google.com/p/phpquery/wiki/Manual
<span><?php <span>include</span> <span>'phpQuery/phpQuery.php'</span>; phpQuery::newDocumentFile(<span>'http://www.ido321.com/'</span>); $artlist = pq(<span>"article"</span>); <span>foreach</span>($artlist <span>as</span> $title){ <span>echo</span> pq($title)->find(<span>'h2'</span>)->html().<span>"<br>"</span>; } ?>
结果:

三、Simple-Html-Dom
项目地址:
http://simplehtmldom.sourceforge.net/
文档:
http://simplehtmldom.sourceforge.net/manual.htm
测试:抓取我网站首页的所有链接
<span><?php <span>include</span> <span>'simple_html_dom.php'</span>; <span>//使用url和file都可以创建DOM</span> $html = file_get_html(<span>'http://www.ido321.com/'</span>); <span>//找到所有图片</span> <span>// foreach($html->find('img') as $element)</span> <span>// echo $element->src . '<br>';</span> <span>//找到所有链接</span> <span>foreach</span>($html->find(<span>'a'</span>) <span>as</span> $element) <span>echo</span> $element->href . <span>'<br>'</span>; ?>
结果:(截图是一部分)

四、Snoopy
项目地址:http://code.google.com/p/phpquery/
文档:http://code.google.com/p/phpquery/wiki/Manual
测试:抓取我的网站首页
<span><?php <span>include</span>(<span>"Snoopy.class.php"</span>); $url = <span>"http://www.ido321.com"</span>; $snoopy = <span>new</span> Snoopy; $snoopy->fetch($url); <span>//获取所有内容</span> <span>echo</span> $snoopy->results; <span>//显示结果</span> <span>// echo $snoopy->fetchtext ;//获取文本内容(去掉html代码)</span> <span>// echo $snoopy->fetchlinks($url) ;//获取链接</span> <span>// $snoopy->fetchform ;//获取表单 </span> ?>
结果:

五、手动编写爬虫
如果编写能力ok,可以手写一个网页爬虫,实现网页抓取。网上有千篇一律的介绍此方法的文章,LZ就不赘述了。有兴趣了解的,可以百度 php 网页抓取。
ps:资源分享
常见的开源爬虫项目请戳:http://blog.chinaunix.net/uid-22414998-id-3774291.html
国民岳父的“屁民理论”

Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

MantisBT
Mantis is an easy-to-deploy web-based defect tracking tool designed to aid in product defect tracking. It requires PHP, MySQL and a web server. Check out our demo and hosting services.

mPDF
mPDF is a PHP library that can generate PDF files from UTF-8 encoded HTML. The original author, Ian Back, wrote mPDF to output PDF files "on the fly" from his website and handle different languages. It is slower than original scripts like HTML2FPDF and produces larger files when using Unicode fonts, but supports CSS styles etc. and has a lot of enhancements. Supports almost all languages, including RTL (Arabic and Hebrew) and CJK (Chinese, Japanese and Korean). Supports nested block-level elements (such as P, DIV),

Dreamweaver CS6
Visual web development tools

DVWA
Damn Vulnerable Web App (DVWA) is a PHP/MySQL web application that is very vulnerable. Its main goals are to be an aid for security professionals to test their skills and tools in a legal environment, to help web developers better understand the process of securing web applications, and to help teachers/students teach/learn in a classroom environment Web application security. The goal of DVWA is to practice some of the most common web vulnerabilities through a simple and straightforward interface, with varying degrees of difficulty. Please note that this software

ZendStudio 13.5.1 Mac
Powerful PHP integrated development environment