I have been using PHP’s Snoopy class for two days and found it very useful. To get all the links in the requested web page, just use fetchlinks directly. To get all the text information, use fetchtext (it still uses regular expressions for processing internally), and there are many other functions, such as simulating form submission, etc.
How to use:
Download the Snoopy class first, download address: http://sourceforge.net/projects/snoopy/
First instantiate an object, and then call the corresponding method to obtain the crawled web page information
include 'snoopy/Snoopy.class.php';
$snoopy = new Snoopy();
$sourceURL = "http://www.jb51.net";
$snoopy->fetchlinks($sourceURL);
$a = $snoopy->results;
It does not provide a method to obtain the addresses of all images in a web page. I have a need to obtain the image addresses of all articles in a page. Then I wrote one myself, mainly because the matching of regular expressions is important.
//Regular expression to match images
$reTag = "/

Because the needs are quite special, we only need to capture the images that begin with http:// (the images from external sites may prevent hotlinking, so we want to capture them locally first)
1. Crawl the specified web page and filter out all expected article addresses;
2. Loop to grab the article address in the first step, and then use the regular expression to match the image to get all the image addresses that match the rules on the page;
3. Save the image according to the image suffix and ID (only gif, jpg here) --- If this image file exists, delete it first and then save it.
Include 'snoopy/Snoopy.class.php';
$snoopy = new Snoopy();
$sourceURL = "http://xxxxx";
$snoopy->fetchlinks($sourceURL);
$a = $snoopy->results;
$re = "/d+.html$/";
//Filter the request to obtain the specified file address
foreach ($a as $tmp) {
If (preg_match($re, $tmp)) {
getImgURL($tmp);
}
}
Function getImgURL($siteName) {
$snoopy = new Snoopy();
$snoopy->fetch($siteName);
$fileContent = $snoopy->results;
//Regular expression to match images
$reTag = "/

If (preg_match($reTag, $fileContent)) {
$ret = preg_match_all($reTag, $fileContent, $matchResult);
for ($i = 0, $len = count($matchResult[1]); $i saveImgURL($matchResult[1][$i], $matchResult[2][$i]);
}
}
}
Function saveImgURL($name, $suffix) {
$url = $name.".".$suffix;
echo "Requested image address: ".$url."
";
$imgSavePath = "E:/xxx/style/images/";
$imgId = preg_replace("/^.+/(d+)$/", "1", $name);
If ($suffix == "gif") {
$imgSavePath .= "emotion";
} else {
$imgSavePath .= "topic";
}
$imgSavePath .= ("/".$imgId.".".$suffix);
If (is_file($imgSavePath)) {
unlink($imgSavePath);
echo "
The file ".$imgSavePath." already exists and will be deleted
";}
$imgFile = file_get_contents($url);
$flag = file_put_contents($imgSavePath, $imgFile);
if ($flag) {
echo "
File".$imgSavePath."Save successfully
";}
}
?>
When using PHP to crawl web pages: content, pictures, and links, I think the most important thing is regularity (obtaining the desired data based on the crawled content and specified rules). The ideas are actually relatively simple. Use There are not many methods, just a few (and to capture content, you can just directly call the methods in the class written by others)
But what I thought before is that PHP does not seem to implement the following method. For example, if there are N lines in a file (N is very large), the content of the lines that conform to the rules needs to be replaced. For example, the 3rd line is aaa and needs to be converted. into bbbbb. Common practices when you need to modify files:
1. Read the entire file at once (or read line by line), then use a temporary file to save the final conversion result, and then replace the original file
2. Read line by line, use fseek to control the position of the file pointer, and then fwrite to write
Option 1 is not advisable to read in one go when the file is large (reading line by line, then writing to a temporary file and then replacing the original file is not efficient), option 2 is when the length of the replaced string is less than or equal to There is no problem when the target value is exceeded, but there will be problems if it exceeds it. It will "cross the boundary" and disrupt the data in the next row (it cannot be replaced with new content like the concept of "selection" in JavaScript).
The following is the code for testing using option 2:
$mode = "r+";
$filename = "d:/file.txt";
$fp = fopen($filename, $mode);
if ($fp) {
$i = 1;
while (!feof($fp)) {
$str = fgets($fp);
echo $str;
If ($i == 1) {
$len = strlen($str);
fseek($fp, -$len, SEEK_CUR);//Move the pointer forward
fwrite($fp, "123");
}
i++;
}
fclose($fp);
}
?>
Read a line first. At this time, the file pointer actually points to the beginning of the next line. Use fseek to move the file pointer back to the beginning of the previous line, and then use fwrite to perform the replacement operation. Because it is a replacement operation, if you do not specify In the case of length, it will affect the data of the next row, and what I want is to only operate on this row, such as deleting this row or replacing the entire row with only one 1. The above example does not meet the requirements. Maybe I haven't found the right method yet...

php把负数转为正整数的方法:1、使用abs()函数将负数转为正数,使用intval()函数对正数取整,转为正整数,语法“intval(abs($number))”;2、利用“~”位运算符将负数取反加一,语法“~$number + 1”。

实现方法:1、使用“sleep(延迟秒数)”语句,可延迟执行函数若干秒;2、使用“time_nanosleep(延迟秒数,延迟纳秒数)”语句,可延迟执行函数若干秒和纳秒;3、使用“time_sleep_until(time()+7)”语句。

php字符串有下标。在PHP中,下标不仅可以应用于数组和对象,还可应用于字符串,利用字符串的下标和中括号“[]”可以访问指定索引位置的字符,并对该字符进行读写,语法“字符串名[下标值]”;字符串的下标值(索引值)只能是整数类型,起始值为0。

php除以100保留两位小数的方法:1、利用“/”运算符进行除法运算,语法“数值 / 100”;2、使用“number_format(除法结果, 2)”或“sprintf("%.2f",除法结果)”语句进行四舍五入的处理值,并保留两位小数。

判断方法:1、使用“strtotime("年-月-日")”语句将给定的年月日转换为时间戳格式;2、用“date("z",时间戳)+1”语句计算指定时间戳是一年的第几天。date()返回的天数是从0开始计算的,因此真实天数需要在此基础上加1。

在php中,可以使用substr()函数来读取字符串后几个字符,只需要将该函数的第二个参数设置为负值,第三个参数省略即可;语法为“substr(字符串,-n)”,表示读取从字符串结尾处向前数第n个字符开始,直到字符串结尾的全部字符。

方法:1、用“str_replace(" ","其他字符",$str)”语句,可将nbsp符替换为其他字符;2、用“preg_replace("/(\s|\ \;||\xc2\xa0)/","其他字符",$str)”语句。

查找方法:1、用strpos(),语法“strpos("字符串值","查找子串")+1”;2、用stripos(),语法“strpos("字符串值","查找子串")+1”。因为字符串是从0开始计数的,因此两个函数获取的位置需要进行加1处理。


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

AI Hentai Generator
Generate AI Hentai for free.

Hot Article

Hot Tools

EditPlus Chinese cracked version
Small size, syntax highlighting, does not support code prompt function

VSCode Windows 64-bit Download
A free and powerful IDE editor launched by Microsoft

Dreamweaver Mac version
Visual web development tools

MinGW - Minimalist GNU for Windows
This project is in the process of being migrated to osdn.net/projects/mingw, you can continue to follow us there. MinGW: A native Windows port of the GNU Compiler Collection (GCC), freely distributable import libraries and header files for building native Windows applications; includes extensions to the MSVC runtime to support C99 functionality. All MinGW software can run on 64-bit Windows platforms.

PhpStorm Mac version
The latest (2018.2.1) professional PHP integrated development tool