求助php无法抓取网页，问了几个人都没解决-PHP Tutorial-php.cn

Home

Backend Development

PHP Tutorial

求助php无法抓取网页，问了几个人都没解决

WBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWB

Jun 23, 2016 pm 02:17 PM

本帖最后由 dz215136304 于 2013-06-11 11:35:47 编辑

url必须为以下代码中的url，经测试,在抓取时,如果q后面的参数带空格,他会自动把"&"转换成"&",从而造成数据无法抓取，在网页中直接输入网址是可以得到内容的，求解决方法

$url="http://110.75.65.8/search_turn_page_iphone.htm?sort=&q=liz claiborne&page=1&showMode=list";echo Post($url);function Post($url, $post = null)//请求的网页{$context = array();	if (is_array($post))	{		ksort($post);				$context['http'] = array		(				'timeout'=>60,		'method' => 'POST',		'header'=>">Accept-language: en/r/n",		'content' => http_build_query($post, '', '&'),		);	}return file_get_contents($url, false, stream_context_create($context));}

错误提示：

Warning: file_get_contents(http://110.75.65.8/search_turn_page_iphone.htm?sort=&q=liz claiborne&page=1&showMode=list) [<a href='function.file-get-contents'>function.file-get-contents</a>]: failed to open stream: HTTP request failed! HTTP/1.1 505 HTTP Version Not Supported in F:\wwwroot\getTaobao\test.php on line 25

回复讨论(解决方案)

你可以先看看html 字符实体

file_get_contents ? 将整个文件读入一个字符串

说明

string file_get_contents ( string $filename [, bool $use_include_path [, resource $context [, int $offset [, int $maxlen ]]]] )

和 file() 一样，只除了 file_get_contents() 把文件读入一个字符串。将在参数 offset 所指定的位置开始读取长度为 maxlen 的内容。如果失败，file_get_contents()将返回 FALSE。

file_get_contents()函数是用来将文件的内容读入到一个字符串中的首选方法。如果操作系统支持还会使用内存映射技术来增强性能。

Note: 如果要打开有特殊字符的 URL （比如说有空格），就需要使用 urlencode() 进行 URL 编码。

另外
'header'=>" >Accept-language: en /r/n"
红字部分是什么？
>是多余的，/r/n应为\r\n
header不正确的话，服务器端返回错误(505)就是正常的了

$url="http://110.75.65.8/search_turn_page_iphone.htm?sort=&q=lizclaiborne&page=1&showMode=list";echo Post(urlencode($url));function Post($url, $post = null)//请求的网页{$context = array();	if (is_array($post))	{		ksort($post);				$context['http'] = array		(				'timeout'=>60,		'method' => 'POST',		'header'=>"Accept-language: en\r\n",		'content' => http_build_query($post, '', '&'),		);	}return file_get_contents($url, false, stream_context_create($context));}

实际的错误是：HTTP/1.1 505 HTTP Version Not Supported

file_get_contents(str_replace(' ', '%20', $url));

现在可以了，刚才可能是他的服务器出现了问题

$url="http://110.75.65.8/search_turn_page_iphone.htm?sort=&q=lizclaiborne&page=1&showMode=list";echo file_get_contents($url);

{"result":"true","totalPage":"100","catmap":"","ppath":"","category":"","auctionTagFlag1":"","auctionTagFlag2":"","auctionTagFlag3":"","listItem":[
         {"name":"团购价美国真品liz claiborne丽资克莱本女款中款钱包 liz钱包" ,"img":"http://q.i02.wimg.taobao.com/bao/uploaded/i1/T18ZyyXfXgXXXc8SLa_122312.jpg_90x90.jpg","img2":"http://q.i04.wimg.taobao.com/bao/uploaded/i1/T18ZyyXfXgXXXc8SLa_122312.jpg","iswebp":"","url":"http://a.m.taobao.com/i2431550873.htm?rn=bwHGEi1-ZClPeKBbGc1lfJhm45-D1gLR8O-pug7&sid=8b9c27255c655b1e","previewUrl":"http://a.m.taobao.com/ajax/pre_view.do?itemId=2431550873&sid=8b9c27255c655b1e","favoriteUrl":"http://fav.m.taobao.com/favorite/to_collection.htm?itemNumId=2431550873&sid=8b9c27255c655b1e",
    "icon":["0" ],
    "price":"39.00","originalPrice":"39.00","freight":"10","area":"天津","act":"月售1","itemNumId":"2431550873","nick":"金缕衣_2007",
..........

嗯，粘错了数据
$url="http://110.75.65.8/search_turn_page_iphone.htm?sort=&q=liz claiborne&page=1&showMode=list";
这个不行 HTTP/1.1 505 HTTP Version Not Supported

这样都行
$url="http://110.75.65.8/search_turn_page_iphone.htm?sort=&q=liz +claiborne&page=1&showMode=list";
$url="http://110.75.65.8/search_turn_page_iphone.htm?sort=&q=liz %20claiborne&page=1&showMode=list";

他的服务器不知做了什么设置，不接受未经 url 编码的数据

服务器可以接受“未经 url 编码的数据”吗？
怎么我理解的是服务器只能接受经过url编码的数据呢，
如果我们直接把带空格的地址在浏览器打开，
那浏览器已经自动把url编码了，
所以打开正常，
但是php并不是浏览器，
所以它不会自动做这些事情，
需要手动进行编码，
难道不是这样的吗？

正确的写法是:
$url="http://110.75.65.8/search_turn_page_iphone.htm?sort=&q=". urlencode('liz claiborne') . "&page=1&showMode=list";

这个问题我以前遇到过，把“&”单独拿出来就可以比方说http://www.123.com?id=123&num=123;
写成$url='http://www.123.com?id=123'.“&”.'num=123';这样编译器会把它当字符串算不给转换。

转码即可 urlencode()

Statement

The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

PHP in Action: Real-World Examples and ApplicationsApr 14, 2025 am 12:19 AM

PHP is widely used in e-commerce, content management systems and API development. 1) E-commerce: used for shopping cart function and payment processing. 2) Content management system: used for dynamic content generation and user management. 3) API development: used for RESTful API development and API security. Through performance optimization and best practices, the efficiency and maintainability of PHP applications are improved.

PHP: Creating Interactive Web Content with EaseApr 14, 2025 am 12:15 AM

PHP makes it easy to create interactive web content. 1) Dynamically generate content by embedding HTML and display it in real time based on user input or database data. 2) Process form submission and generate dynamic output to ensure that htmlspecialchars is used to prevent XSS. 3) Use MySQL to create a user registration system, and use password_hash and preprocessing statements to enhance security. Mastering these techniques will improve the efficiency of web development.

PHP and Python: Comparing Two Popular Programming LanguagesApr 14, 2025 am 12:13 AM

PHP and Python each have their own advantages, and choose according to project requirements. 1.PHP is suitable for web development, especially for rapid development and maintenance of websites. 2. Python is suitable for data science, machine learning and artificial intelligence, with concise syntax and suitable for beginners.

The Enduring Relevance of PHP: Is It Still Alive?Apr 14, 2025 am 12:12 AM

PHP is still dynamic and still occupies an important position in the field of modern programming. 1) PHP's simplicity and powerful community support make it widely used in web development; 2) Its flexibility and stability make it outstanding in handling web forms, database operations and file processing; 3) PHP is constantly evolving and optimizing, suitable for beginners and experienced developers.

PHP's Current Status: A Look at Web Development TrendsApr 13, 2025 am 12:20 AM

PHP remains important in modern web development, especially in content management and e-commerce platforms. 1) PHP has a rich ecosystem and strong framework support, such as Laravel and Symfony. 2) Performance optimization can be achieved through OPcache and Nginx. 3) PHP8.0 introduces JIT compiler to improve performance. 4) Cloud-native applications are deployed through Docker and Kubernetes to improve flexibility and scalability.

PHP vs. Other Languages: A ComparisonApr 13, 2025 am 12:19 AM

PHP is suitable for web development, especially in rapid development and processing dynamic content, but is not good at data science and enterprise-level applications. Compared with Python, PHP has more advantages in web development, but is not as good as Python in the field of data science; compared with Java, PHP performs worse in enterprise-level applications, but is more flexible in web development; compared with JavaScript, PHP is more concise in back-end development, but is not as good as JavaScript in front-end development.

PHP vs. Python: Core Features and FunctionalityApr 13, 2025 am 12:16 AM

PHP and Python each have their own advantages and are suitable for different scenarios. 1.PHP is suitable for web development and provides built-in web servers and rich function libraries. 2. Python is suitable for data science and machine learning, with concise syntax and a powerful standard library. When choosing, it should be decided based on project requirements.

PHP: A Key Language for Web DevelopmentApr 13, 2025 am 12:08 AM

PHP is a scripting language widely used on the server side, especially suitable for web development. 1.PHP can embed HTML, process HTTP requests and responses, and supports a variety of databases. 2.PHP is used to generate dynamic web content, process form data, access databases, etc., with strong community support and open source resources. 3. PHP is an interpreted language, and the execution process includes lexical analysis, grammatical analysis, compilation and execution. 4.PHP can be combined with MySQL for advanced applications such as user registration systems. 5. When debugging PHP, you can use functions such as error_reporting() and var_dump(). 6. Optimize PHP code to use caching mechanisms, optimize database queries and use built-in functions. 7

See all articles