PHP collection class Snoopy.class.php

Snoopy is a php collection class, used to simulate the browser to obtain web content and send forms.

Here are some Snoopy features:

Easy to crawl web content

Easy to grab page text (remove html tags)

Easy to grab links within web pages

Support proxy crawling

Supports basic username and password authentication

Supports setting user-agent, referer, cookies and header content

Support browser steering and control steering depth

Can convert the links on the page into high-quality links

Easy to submit data and get rebates

Can track HTML frames

Supports passing cookies when redirecting

Snoopy class, methods:

fetch($URI)

Method used to crawl the content of web pages. The $URI parameter is the URL address of the crawled web page. The fetched results are stored in $this->results. If you are scraping a frame, Snoopy will track each frame and store it in an array, then store it in $this->results.

fetchtext($URI)

This method is similar to fetch(). The only difference is that this method will remove HTML tags and other irrelevant data, and only return the text content in the web page.

fetchform($URI)

This method is similar to fetch(). The only difference is that this method will remove HTML tags and other irrelevant data, and only return the form content (form) in the web page.

fetchlinks($URI)

This method is similar to fetch(). The only difference is that this method will remove HTML tags and other irrelevant data, and only return links in the web page. By default, relative links will be automatically completed and converted into full URLs.

submit($URI,$formvars)

This method sends a confirmation form to the link address specified by $URL. $formvars is an array that stores form parameters.

submittext($URI,$formvars)

This method is similar to submit(). The only difference is that this method will remove HTML tags and other irrelevant data, and only return the text content of the web page after login.

submitlinks($URI)

This method is similar to submit(). The only difference is that this method will remove HTML tags and other irrelevant data, and only return the link in the web page. By default, relative links will be automatically completed and converted into full URLs.

Class attribute: (Default value is in brackets)

$host The connected host

$port The port to connect to

$proxy_host The proxy host used, if any

$proxy_port The proxy host port used, if any

$agent User agent disguise (Snoopy v0.1)

$referer route information, if available

$cookies, if any

$rawheaders Other header information, if any

$maxredirs Maximum number of redirects, 0=not allowed (5)

$offsiteok whether or not to allow redirects off-site. (true)

$expandlinks Whether to complete all links to complete addresses (true)

$user authentication username, if available

$pass authentication username, if available

$accept http accept type (image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*)

$error Where to report the error, if any

$response_code The response code returned from the server

$headers Header information returned from the server

$maxlength The longest returned data length

$read_timeout read operation timeout (requires PHP 4 Beta 4+) set to 0 for no timeout

$timed_out If a read operation times out, this property returns true (requires PHP 4 Beta 4+)

$maxframes Maximum number of frames allowed to be tracked

$status The status of the captured http

$temp_dir The temporary file directory (/tmp) that the web server can write to

$curl_path cURL binary directory, if there is no cURL binary, set it to false

Snoopy official website: http://sourceforge.net/projects/snoopy/

I hope this article will be helpful to the majority of php developers, thank you for reading this article. For more php technical issues welcome to join the group to discuss: 256271784, verification code: cxy, if you don’t write the verification, it will not be passed~