search
Homephp教程php手册PHP simple_html_dom.php+正则 采集文章代码

PHP simple_html_dom.php+正则 采集文章代码

Jun 13, 2016 pm 12:19 PM
htmlphpsimplecodeIncludecopyarticleregularcollection

复制代码 代码如下:


//包含PHP Simple html Dom 类库文件
include_once('./simplehtmldom/simple_html_dom.php');

//采集html
function getwebcontent($url){
$ch = curl_init();
$timeout = 10;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1);
$contents = trim(curl_exec($ch));
curl_close($ch);
return $contents;
}


//获得标题和url
$string =
getwebcontent('http://www.babytree.com/learn/zhunbeihuaiyun/jijibeiyun/2');
//正则匹配

  • 获取标题和地址
    preg_match_all ("/
  • (.*)/",
    $string, $out, PREG_SET_ORDER);

    foreach($out as $key => $value){
    $article['title'][] = $out[$key][2];
    $article['link'][] = "http://www.babytree.com/learn/article/".$out[$key][1];
    }

    //根据url获取文章内容
    foreach($article['link'] as $key=>$value){
    $html = file_get_html($value);
    $div = $html->find('div[id=pagenum_0]');
    $article[content][] = $div[0]->innertext;
    }
    //标题转码---真正用的时候不用这步--因为咱本来就要用utf8的
    //不转码还真不能保存成文件
    foreach($article[title] as $key=>$value){
    $article[title][$key] = iconv('utf-8', 'gbk', $value);//转码
    }
    //存入文件
    $num = count($article['title']);
    for($i=0; $ifile_put_contents("{$article[title][$i]}.txt", $article['content'][$i]);
    }

    /*本来想12点之前发的。。但小看一下都3点半了。。。 就算昨天的吧
    本来获取文章内容时用正则是最好的,速度也是最快的,
    奈何正则是好,但正则表达式是真难!于是乎小查了一下,
    网上也有很多人也在用PHP Simple Dom 虽然效率慢了点,但效果还是不错的
    从包含类库文件到写入txt文件 大概是7/8就秒 还有带于进一步优化,特别是那获取文章内容时的正则,那个太恶心了
    大家可以小研究一下*/
    ?>
  • Statement
    The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

    Hot AI Tools

    Undresser.AI Undress

    Undresser.AI Undress

    AI-powered app for creating realistic nude photos

    AI Clothes Remover

    AI Clothes Remover

    Online AI tool for removing clothes from photos.

    Undress AI Tool

    Undress AI Tool

    Undress images for free

    Clothoff.io

    Clothoff.io

    AI clothes remover

    Video Face Swap

    Video Face Swap

    Swap faces in any video effortlessly with our completely free AI face swap tool!

    Hot Tools

    Dreamweaver CS6

    Dreamweaver CS6

    Visual web development tools

    EditPlus Chinese cracked version

    EditPlus Chinese cracked version

    Small size, syntax highlighting, does not support code prompt function

    DVWA

    DVWA

    Damn Vulnerable Web App (DVWA) is a PHP/MySQL web application that is very vulnerable. Its main goals are to be an aid for security professionals to test their skills and tools in a legal environment, to help web developers better understand the process of securing web applications, and to help teachers/students teach/learn in a classroom environment Web application security. The goal of DVWA is to practice some of the most common web vulnerabilities through a simple and straightforward interface, with varying degrees of difficulty. Please note that this software

    MantisBT

    MantisBT

    Mantis is an easy-to-deploy web-based defect tracking tool designed to aid in product defect tracking. It requires PHP, MySQL and a web server. Check out our demo and hosting services.

    Safe Exam Browser

    Safe Exam Browser

    Safe Exam Browser is a secure browser environment for taking online exams securely. This software turns any computer into a secure workstation. It controls access to any utility and prevents students from using unauthorized resources.