Home >Backend Development >PHP Tutorial >Summary of Six Methods of Calling Remote URLs in PHP_PHP Tutorial

Summary of Six Methods of Calling Remote URLs in PHP_PHP Tutorial

WBOY
WBOYOriginal
2016-07-21 15:43:38902browse

Sample code 1: Use file_get_contents to get the content in get mode

Copy the code The code is as follows:

$url='http://www.baidu.com/';
$html=file_get_contents($url);
//print_r($http_response_header);
ec($html);
printhr();
printarr($http_response_header);
printhr();
?>


Example code 2: Open the url with fopen and get Method to obtain content
Copy code The code is as follows:

$fp=fopen($url,' r');
printarr(stream_get_meta_data($fp));
printhr();
while(!feof($fp)){
$result.=fgets($fp,1024) ;
}
echo "url body: $result";
printhr();
fclose($fp);
?>

Sample code 3: Use the file_get_contents function to get the url in post mode
Copy the code The code is as follows:

$data=array('foo'=>'bar');
$data=http_build_query($data);

$opts=array(
'http'=>array(
'method'=>'POST',
'header'=>"Content-type: application/x-www-form-urlencodedrn".
"Content-Length: ".strlen( $data)."rn",
'content'=>$data
),
);
$context=stream_context_create($opts);
$html=file_get_contents(' http://localhost/e/admin/test.html',false,$context);
echo$html;
?>

Sample code 4: Use fsockopen function Open the url and get the complete data in get mode, including header and body
Copy the code The code is as follows:

functionget_url($url,$cookie=false){
$url=parse_url($url);
$query=$url[path]."?".$url[query];
ec("Query:".$query);
$fp=fsockopen($url[host],$url[port]?$url[port]:80,$errno,$errstr,30);
if(!$fp){
returnfalse;
}else{
$request="GET$queryHTTP/1.1rn";
$request.="Host:$url[host] rn";
$request.="Connection: Closern";
if($cookie)$request.="Cookie: $cookien";
$request.="rn";
fwrite ($fp,$request);
while(!@feof($fp)){
$result.=@fgets($fp,1024);
}
fclose($fp) ;
return$result;
}
}
//Get the html part of the url, remove the header
functionGetUrlHTML($url,$cookie=false){
$rowdata=get_url ($url,$cookie);
if($rowdata)
{
$body=stristr($rowdata,"rnrn");
$body=substr($body,4,strlen ($body));
return$body;
}
returnfalse;
}
?>

Sample code 5: Open url with fsockopen function , obtain complete data in POST mode, including header and body
Copy code The code is as follows:

>functionHTTP_Post($URL,$data,$cookie,$referrer=""){
// parsing the given URL
$URL_Info=parse_url($URL);

// Building referrer
if($referrer=="")// if not given use this script. as referrer
$referrer="111";

// making string from $data
foreach ($dataas$key=>$value)
$values[]="$key=".urlencode($value);
$data_string=implode("&",$values);

// Find out which port is needed - if not given use standard (=80)
if(!isset($URL_Info["port"]))
$URL_Info["port"]=80 ;

// building POST-request:
$request.="POST ".$URL_Info["path"]." HTTP/1.1n";
$request.="Host: ".$URL_Info["host"]."n";
$request.="Referer:$referern";
$request.="Content-type: application/x-www-form-urlencodedn" ;
$request.="Content-length: ".strlen($data_string)."n";
$request.="Connection: closen";
$request.="Cookie: $cookien ";
$request.="n";
$request.=$data_string."n";

$fp=fsockopen($URL_Info["host"],$URL_Info[" port"]);
fputs($fp,$request);
while(!feof($fp)){
$result.=fgets($fp,1024);
}
fclose($fp);
return$result;
}
printhr();
?>

Sample code 6: Using the curl library, use Before curling the library, you may need to check php.ini to see if the curl extension has been turned on
Copy the code The code is as follows:

$ch = curl_init();
$timeout = 5;
curl_setopt ($ch, CURLOPT_URL, 'http://www.baidu.com/');
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$file_contents = curl_exec($ch);
curl_close($ch);
echo $ file_contents;
?>

About curl library:
curl official website http://curl.haxx.se/
curl is a file transfer tool using URL syntax, supporting FTP, FTPS, HTTP HTTPS SCP SFTP TFTP TELNET DICT FILE and LDAP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploads, Kerberos, HTTP-based uploads, proxies, cookies, user + password proof, file transfer recovery, http proxy channels and a lot of other useful tricks
Copy code The code is as follows:

functionprintarr(array$arr)
{
echo"";
foreach($arras$key=>$value)
{
echo"$key=$value < ;br>";
}
}
?>

====================== ================================
The code for PHP to capture remote website data
There may still be Many programming enthusiasts will encounter the same question, that is, how to crawl the HTML code of other people's websites like a search engine, and then collect and organize the code into useful data for themselves! Let me introduce some simple examples today.

Ⅰ. Example of grabbing the title of a remote web page:
The following is the code snippet:
Copy the code The code is as follows:

/*
+--------------------------------- ----------------------------
+ Grab the code for the web page title, directly copy this code snippet and save it as a .php file Just execute.
+--------------------------------------------- ------------------
*/

error_reporting(7);
$file = fopen ("http://www. jb51.net/", "r");
if (!$file) {
echo "Unable to open remote file.n";
exit;
}
while (!feof ($file)) {
$line = fgets ($file, 1024);
if (eregi ("(.*)< ;/title>", $line, $out)) { <br>$title = $out[1]; <br>echo "".$title.""; <br>break; <br>} <br>} <br>fclose($file); <br><br>//End <br>?> <br> </div> <br>Ⅱ. Example of grabbing the HTML code of a remote web page: <br><br>The following is the code snippet: <br><div class="codetitle"> <span style="CURSOR: pointer" onclick="doCopy('code34679')"><u>Copy code </u></span> The code is as follows: </div> <div class="codebody" id="code34679"> <br><? php <BR>/* <BR>+---------------- <BR>+DNSing Sprider <BR>+---------------- <BR>*/ <br><br>$fp = fsockopen("www.dnsing.com", 80, $errno, $errstr, 30); <BR>if (!$fp) { <BR>echo "$errstr ($errno) <br/>n"; <br>} else { <br>$out = "GET / HTTP/1.1rn"; <br>$out .= "Host:www.dnsing.comrn"; <br> $out .= "Connection: Close rnrn"; <br>fputs($fp, $out); <br>while (!feof($fp)) { <br>echo fgets($fp, 128); <br>} <br>fclose($fp); <br>} <br>//End <br>?> <br> </div>Just copy the above two code snippets and run them back to see the effect. The above example is just a prototype of grabbing web page data. To make it more suitable for your own use, the situation will be different. So, all program enthusiasts, please take care of yourself. Let’s research it. <br><br>================================== <br><br>It makes a little sense The functions are: get_content_by_socket(), get_url(), get_content_url(), get_content_object. Several functions may be able to give you some ideas. <br><?php <br><br>//Get all content urls and save them to files <BR>function get_index($save_file, $prefix="index_"){ <BR>$count = 68; <BR> $i = 1; <BR>if (file_exists($save_file)) @unlink($save_file); <BR>$fp = fopen($save_file, "a+") or die("Open ". $save_file ." failed "); <BR>while($i<$count){ <BR>$url = $prefix . $i .".htm"; <BR>echo "Get ". $url ."..."; <BR>$url_str = get_content_url(get_url($url)); <BR>echo " OKn"; <BR>fwrite($fp, $url_str); <BR>++$i; <BR>} <BR>fclose ($fp); <BR>} <br><br>//Get the target multimedia object<BR>function get_object($url_file, $save_file, $split="|--:**:--|"){ <BR>if (!file_exists($url_file)) die($url_file ." not exist"); <BR>$file_arr = file($url_file); <BR>if (!is_array($file_arr) || empty( $file_arr)) die($url_file ." not content"); <BR>$url_arr = array_unique($file_arr); <BR>if (file_exists($save_file)) @unlink($save_file); <BR>$fp = fopen($save_file, "a+") or die("Open save file ". $save_file ." failed"); <BR>foreach($url_arr as $url){ <BR>if (empty($url)) continue; <BR>echo "Get ". $url ."..."; <BR>$html_str = get_url($url); <BR>echo $html_str; <BR>echo $url; <BR>exit; <BR>$obj_str = get_content_object($html_str); <BR>echo " OKn"; <BR>fwrite($fp, $obj_str); <BR>} <BR>fclose($fp); <BR>} <br><br>//Traverse the directory to get the file content<BR>function get_dir($save_file, $dir){ <BR>$dp = opendir($dir); <BR>if (file_exists($save_file)) @unlink ($save_file); <BR>$fp = fopen($save_file, "a+") or die("Open save file ". $save_file ." failed"); <BR>while(($file = readdir($dp )) != false){ <BR>if ($file!="." && $file!=".."){ <BR>echo "Read file ". $file ."..."; <BR>$file_content = file_get_contents($dir . $file); <BR>$obj_str = get_content_object($file_content); <BR>echo " OKn"; <BR>fwrite($fp, $obj_str); <BR>} <BR>} <BR>fclose($fp); <BR>} <br><br><BR>//Get the content of the specified url<BR>function get_url($url){ <BR>$reg = '/^ http://[^/].+$/'; <BR>if (!preg_match($reg, $url)) die($url ." invalid"); <BR>$fp = fopen($url, "r") or die("Open url: ". $url ." failed."); <BR>while($fc = fread($fp, 8192)){ <BR>$content .= $fc; <BR>} <BR>fclose($fp); <BR>if (empty($content)){ <BR>die("Get url: ". $url ." content failed."); <BR>} <BR>return $content; <BR>} <br><br>//Use socket to get the specified web page<BR>function get_content_by_socket($url, $host){ <BR>$fp = fsockopen($host, 80) or die("Open ". $url ." failed"); <BR>$header = "GET /".$url ." HTTP/1.1rn"; <BR>$header .= "Accept: */*rn" ; <BR>$header .= "Accept-Language: zh-cnrn"; <BR>$header .= "Accept-Encoding: gzip, deflatern"; <BR>$header .= "User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Maxthon; InfoPath.1; .NET CLR 2.0.50727)rn"; <BR>$header .= "Host: ". $host ."rn"; <BR> $header .= "Connection: Keep-Alivern"; <BR>//$header .= "Cookie: cnzz02=2; rtime=1; ltime=1148456424859; cnzz_eid=56601755-rnrn"; <BR>$header .= "Connection: Closernrn"; <br><br>fwrite($fp, $header); <BR>while (!feof($fp)) { <BR>$contents .= fgets($fp, 8192); <BR>} <BR>fclose($fp); <BR>return $contents; <BR>} <br><br><BR>//Get the url in the specified content <BR>function get_content_url($host_url, $ file_contents){ <br><br>//$reg = '/^(#|javascript.*?|ftp://.+|http://.+|.*?href.*?|play.* ?|index.*?|.*?asp)+$/i'; <BR>//$reg = '/^(down.*?.html|d+_d+.htm.*?)$/i' ; <BR>$rex = "/([hH][rR][eE][Ff])s*=s*['"]*([^>'"s]+)["'>] *s*/i"; <br>$reg = '/^(down.*?.html)$/i'; <br>preg_match_all ($rex, $file_contents, $r); <br>$result = ""; //array(); <br>foreach($r as $c){ <br>if (is_array($c)){ <br>foreach($c as $d){ <br>if ( preg_match($reg, $d)){ $result .= $host_url . $d."n"; } <br>} <br>} <br>} <br>return $result; <br>} <br><br>//Get the multimedia files in the specified content<br>function get_content_object($str, $split="|--:**:--|"){ <br>$regx = "/hrefs*= s*['"]*([^>'"s]+)["'>]*s*(<b>.*?</b>)/i"; <br>preg_match_all( $regx, $str, $result); <br><br>if (count($result) == 3){ <br>$result[2] = str_replace("<b>Multimedia: ", "" , $result[2]); <br>$result[2] = str_replace("</b>", "", $result[2]); <br>$result = $result[1][0 ] . $split .$result[2][0] . "n"; <br>} <br>return $result; <br>} <br><br>?> <br><br>== ================================================== == <br><br>When the same domain name corresponds to multiple IPs, PHP's function to obtain the content of the remote webpage<br><br>fgc simply reads it and encapsulates all operations<br>fopen also performs some Encapsulated, but you need to read all the data in a loop. <br>fsockopen This is a straight-line socket operation.<br>If you just read an html page, fgc is better. <br>If the company accesses the Internet through a firewall, the general file_get_content function will not work. Of course, it is also possible to directly write http requests to the proxy through some socket operations, but it is more troublesome. <br>If you can confirm that the file is small, you can choose any of the above two methods fopen,join('',file($file));. For example, if you only operate files smaller than 1k, it is best to use file_get_contents. <br><br>If you are sure that the file is large, or the size of the file cannot be determined, it is best to use file streams. There is no obvious difference between fopening a 1K file and fopening a 1G file. The longer the content, the longer it takes to read, rather than letting the script die. <br><br>-------------------------------------------------- -------- <br>http://www.phpcake.cn/archives/tag/fsockopen <br>PHP has many ways to obtain remote web content, such as using its own functions such as file_get_contents and fopen. <br><br><?php <br><br>echo file_get_contents("http://img.jb51.net/abc.php"); <BR>?> <br> However, in the DNS round In load balancing such as query, the same domain name may correspond to multiple servers and multiple IPs. Assume that img.jb51.net is resolved to three IPs: 72.249.146.213, 72.249.146.214, and 72.249.146.215 by DNS. Every time a user accesses img.jb51.net, the system will access one of the servers based on the corresponding load balancing algorithm. <br> When I was working on a video project last week, I encountered such a requirement: I needed to access a PHP interface program (assumed to be abc.php) on each server in order to query the transmission status of this server. <br><br> At this time, you cannot directly use file_get_contents to access http://img.jb51.net/abc.php, because it may keep accessing a certain server repeatedly. <br><br> And use the method of visiting http://72.249.146.213/abc.php, http://72.249.146.214/abc.php, http://72.249.146.215/abc.php in sequence, here It is also not possible when the Web Server on three servers is equipped with multiple virtual hosts. <br><br> It is not possible to set local hosts, because hosts cannot set multiple IPs corresponding to the same domain name. <br><br> Then it can only be achieved through PHP and HTTP protocols: when accessing abc.php, add the img.jb51.net domain name to the header. So, I wrote the following PHP function: <br><div class="codetitle"> <span style="CURSOR: pointer" onclick="doCopy('code34015')"><u>Copy code </u></span> The code is as follows: </div> <div class="codebody" id="code34015"> <br><?php <br><br>/************************ <BR>* Function usage: When the same domain name corresponds to multiple IPs, obtain the remote web page content of the specified server <BR>* Creation time :2008-12-09 <BR>* Created by: Zhang Yan (img.jb51.net) <BR>* Parameter description: <BR>* $ip The IP address of the server<BR>* $host The host name of the server<BR>* $url URL address of the server (excluding domain name) <BR>* Return value: <BR>* Obtained remote web page content<BR>* false Failed to access remote web page<BR>******* *****************/ <BR>function HttpVisit($ip, $host, $url) <BR>{ <BR>$errstr = ''; <BR>$errno = ''; <BR> $fp = fsockopen ($ip, 80, $errno, $errstr, 90); <BR>if (!$fp) <BR>{ <BR>return false; <BR>} <BR>else <BR>{ <BR>$out = "GET {$url} HTTP/1.1rn"; <BR>$out .= "Host:{$host}rn"; <BR>$out .= "Connection: closernrn"; <BR>fputs ($fp, $out); <br><br>while($line = fread($fp, 4096)){ <BR>$response .= $line; <BR>} <BR>fclose( $ fp ); <br><br>//Remove Header information<BR>$pos = strpos($response, "rnrn"); <BR>$response = substr($response, $pos + 4); <br><br>return $response; <BR>} <BR>} <br><br>//Calling method: <BR>$server_info1 = HttpVisit("72.249.146.213", "img.jb51.net", " /abc.php"); <BR>$server_info2 = HttpVisit("72.249.146.214", "img.jb51.net", "/abc.php"); <BR>$server_info3 = HttpVisit("72.249.146.215" , "img.jb51.net", "/abc.php"); <BR>?> <br> </div> <p align="left"></p> <div style="display:none;"> <span id="url" itemprop="url">http://www.bkjia.com/PHPjc/320696.html</span><span id="indexUrl" itemprop="indexUrl">www.bkjia.com</span><span id="isOriginal" itemprop="isOriginal">true</span><span id="isBasedOnUrl" itemprop="isBasedOnUrl">http: //www.bkjia.com/PHPjc/320696.html</span><span id="genre" itemprop="genre">TechArticle</span><span id="description" itemprop="description">Sample code 1: Use file_get_contents to get the content in get mode. Copy the code as follows: ?php $url='http: //www.baidu.com/'; $html=file_get_contents($url); //print_r($http_response_he...</span> </div> <div class="art_confoot"></div></div><div class="nphpQianMsg"><div class="clear"></div></div><div class="nphpQianSheng"><span>Statement:</span><div>The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn</div></div></div><div class="nphpSytBox"><span>Previous article:<a class="dBlack" title="Implement jQuery extension function in php_PHP tutorial" href="https://m.php.cn/faq/310315.html">Implement jQuery extension function in php_PHP tutorial</a></span><span>Next article:<a class="dBlack" title="Implement jQuery extension function in php_PHP tutorial" href="https://m.php.cn/faq/310317.html">Implement jQuery extension function in php_PHP tutorial</a></span></div><div class="nphpSytBox2"><div class="nphpZbktTitle"><h2>Related articles</h2><em><a href="https://m.php.cn/article.html" class="bBlack"><i>See more</i><b></b></a></em><div class="clear"></div></div><ins class="adsbygoogle" style="display:block" data-ad-format="fluid" data-ad-layout-key="-6t+ed+2i-1n-4w" data-ad-client="ca-pub-5902227090019525" data-ad-slot="8966999616"></ins><script> (adsbygoogle = window.adsbygoogle || []).push({}); </script><ul class="nphpXgwzList"><li><b></b><a href="https://m.php.cn/faq/1.html" title="How to use cURL to implement Get and Post requests in PHP" class="aBlack">How to use cURL to implement Get and Post requests in PHP</a><div class="clear"></div></li><li><b></b><a href="https://m.php.cn/faq/1.html" title="How to use cURL to implement Get and Post requests in PHP" class="aBlack">How to use cURL to implement Get and Post requests in PHP</a><div class="clear"></div></li><li><b></b><a href="https://m.php.cn/faq/1.html" title="How to use cURL to implement Get and Post requests in PHP" class="aBlack">How to use cURL to implement Get and Post requests in PHP</a><div class="clear"></div></li><li><b></b><a href="https://m.php.cn/faq/1.html" title="How to use cURL to implement Get and Post requests in PHP" class="aBlack">How to use cURL to implement Get and Post requests in PHP</a><div class="clear"></div></li><li><b></b><a href="https://m.php.cn/faq/2.html" title="All expression symbols in regular expressions (summary)" class="aBlack">All expression symbols in regular expressions (summary)</a><div class="clear"></div></li></ul></div></div><ins class="adsbygoogle" style="display:block" data-ad-format="autorelaxed" data-ad-client="ca-pub-5902227090019525" data-ad-slot="5027754603"></ins><script> (adsbygoogle = window.adsbygoogle || []).push({}); </script><footer><div class="footer"><div class="footertop"><img src="/static/imghwm/logo.png" alt=""><p>Public welfare online PHP training,Help PHP learners grow quickly!</p></div><div class="footermid"><a href="https://m.php.cn/about/us.html">About us</a><a href="https://m.php.cn/about/disclaimer.html">Disclaimer</a><a href="https://m.php.cn/update/article_0_1.html">Sitemap</a></div><div class="footerbottom"><p> © php.cn All rights reserved </p></div></div></footer><script>isLogin = 0;</script><script type="text/javascript" src="/static/layui/layui.js"></script><script type="text/javascript" src="/static/js/global.js?4.9.47"></script></div><script src="https://vdse.bdstatic.com//search-video.v1.min.js"></script><link rel='stylesheet' id='_main-css' href='/static/css/viewer.min.css' type='text/css' media='all'/><script type='text/javascript' src='/static/js/viewer.min.js?1'></script><script type='text/javascript' src='/static/js/jquery-viewer.min.js'></script><script>jQuery.fn.wait = function (func, times, interval) { var _times = times || -1, //100次 _interval = interval || 20, //20毫秒每次 _self = this, _selector = this.selector, //选择器 _iIntervalID; //定时器id if( this.length ){ //如果已经获取到了,就直接执行函数 func && func.call(this); } else { _iIntervalID = setInterval(function() { if(!_times) { //是0就退出 clearInterval(_iIntervalID); } _times <= 0 || _times--; //如果是正数就 -- _self = $(_selector); //再次选择 if( _self.length ) { //判断是否取到 func && func.call(_self); clearInterval(_iIntervalID); } }, _interval); } return this; } $("table.syntaxhighlighter").wait(function() { $('table.syntaxhighlighter').append("<p class='cnblogs_code_footer'><span class='cnblogs_code_footer_icon'></span></p>"); }); $(document).on("click", ".cnblogs_code_footer",function(){ $(this).parents('table.syntaxhighlighter').css('display','inline-table');$(this).hide(); }); $('.nphpQianCont').viewer({navbar:true,title:false,toolbar:false,movable:false,viewed:function(){$('img').click(function(){$('.viewer-close').trigger('click');});}}); </script></body><!-- Matomo --><script> var _paq = window._paq = window._paq || []; /* tracker methods like "setCustomDimension" should be called before "trackPageView" */ _paq.push(['trackPageView']); _paq.push(['enableLinkTracking']); (function() { var u="https://tongji.php.cn/"; _paq.push(['setTrackerUrl', u+'matomo.php']); _paq.push(['setSiteId', '9']); var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s); })(); </script><!-- End Matomo Code --></html>