Home >Backend Development >PHP Tutorial >CURL抓取失败请教

CURL抓取失败请教

WBOY
WBOYOriginal
2016-06-23 14:38:471448browse

问个问题哈~
用curl抓取不到这个页面的数据~请教一下方法
http://mybus.xiamentd.com/LineDetailQuery?lineId=118&direction=1
thx!


回复讨论(解决方案)

根据返回的头调整你的程序

$url = 'http://mybus.xiamentd.com/LineDetailQuery?lineId=118&direction=1';print_r(get_headers($url));
Array(    [0] => HTTP/1.1 302 Moved Temporarily    [1] => Server: nginx    [2] => Date: Tue, 03 Dec 2013 12:00:53 GMT    [3] => Content-Length: 0    [4] => Connection: close    [5] => Set-Cookie: JSESSIONID=8852C73A784F0B95C9DE7FB33812EDA0; Path=/    [6] => P3P: CP=CAO PSA OUR    [7] => Set-Cookie: HttpOnly=true    [8] => Set-Cookie: CALLBACKURL=http%3A%2F%2Fmybus.xiamentd.com%2FLineDetailQuery%3FlineId%3D118%26direction%3D1%26random%3D1386072053450; Path=/    [9] => Location: http://xiamentd.com/getphoneinfo2.do?code=aHR0cDovL215YnVzLnhpYW1lbnRkLmNvbS9MaW5lRGV0YWlsUXVlcnk=    [10] => HTTP/1.1 302 Moved Temporarily    [11] => Server: nginx    [12] => Date: Tue, 03 Dec 2013 12:00:53 GMT    [13] => Content-Type: text/html;charset=utf-8    [14] => Content-Length: 0    [15] => Connection: close    [16] => Set-Cookie: Wcity_xm=4F706AE207628F6A3DE165FA08BE9E20; Path=/; HttpOnly    [17] => Location: http://mybus.xiamentd.com/LineDetailQuery?return=2&tel=null&model=null    [18] => HTTP/1.1 302 Moved Temporarily    [19] => Server: nginx    [20] => Date: Tue, 03 Dec 2013 12:00:53 GMT    [21] => Content-Length: 0    [22] => Connection: close    [23] => Set-Cookie: JSESSIONID=09C8413DA570C390ED8F9F1346C1D78E; Path=/    [24] => P3P: CP=CAO PSA OUR    [25] => Set-Cookie: HttpOnly=true    [26] => Set-Cookie: CALLBACKURL=http%3A%2F%2Fmybus.xiamentd.com%2FLineDetailQuery%3Freturn%3D2%26tel%3Dnull%26model%3Dnull%26random%3D1386072053965; Path=/    [27] => Location: http://xiamentd.com/getphoneinfo2.do?code=aHR0cDovL215YnVzLnhpYW1lbnRkLmNvbS9MaW5lRGV0YWlsUXVlcnk=    [28] => HTTP/1.1 302 Moved Temporarily    [29] => Server: nginx    [30] => Date: Tue, 03 Dec 2013 12:00:54 GMT    [31] => Content-Type: text/html;charset=utf-8    [32] => Content-Length: 0    [33] => Connection: close    [34] => Set-Cookie: Wcity_xm=E1C22FC11A9DC09587B66D224257D91C; Path=/; HttpOnly    [35] => Location: http://mybus.xiamentd.com/LineDetailQuery?return=2&tel=null&model=null    [36] => HTTP/1.1 302 Moved Temporarily    [37] => Server: nginx    [38] => Date: Tue, 03 Dec 2013 12:00:54 GMT    [39] => Content-Length: 0    [40] => Connection: close    [41] => Set-Cookie: JSESSIONID=B3034C324C287382DED4EF07D81CA04F; Path=/    [42] => P3P: CP=CAO PSA OUR    [43] => Set-Cookie: HttpOnly=true    [44] => Set-Cookie: CALLBACKURL=http%3A%2F%2Fmybus.xiamentd.com%2FLineDetailQuery%3Freturn%3D2%26tel%3Dnull%26model%3Dnull%26random%3D1386072054419; Path=/    [45] => Location: http://xiamentd.com/getphoneinfo2.do?code=aHR0cDovL215YnVzLnhpYW1lbnRkLmNvbS9MaW5lRGV0YWlsUXVlcnk=    [46] => HTTP/1.1 302 Moved Temporarily    [47] => Server: nginx    [48] => Date: Tue, 03 Dec 2013 12:00:54 GMT    [49] => Content-Type: text/html;charset=utf-8    [50] => Content-Length: 0    [51] => Connection: close    [52] => Set-Cookie: Wcity_xm=C38E38FF759C0707F5D45057227AA363; Path=/; HttpOnly    [53] => Location: http://mybus.xiamentd.com/LineDetailQuery?return=2&tel=null&model=null    [54] => HTTP/1.1 302 Moved Temporarily    [55] => Server: nginx    [56] => Date: Tue, 03 Dec 2013 12:00:54 GMT    [57] => Content-Length: 0    [58] => Connection: close    [59] => Set-Cookie: JSESSIONID=CEA724315DE5284BA898B7BDED29BAB8; Path=/    [60] => P3P: CP=CAO PSA OUR    [61] => Set-Cookie: HttpOnly=true    [62] => Set-Cookie: CALLBACKURL=http%3A%2F%2Fmybus.xiamentd.com%2FLineDetailQuery%3Freturn%3D2%26tel%3Dnull%26model%3Dnull%26random%3D1386072054876; Path=/    [63] => Location: http://xiamentd.com/getphoneinfo2.do?code=aHR0cDovL215YnVzLnhpYW1lbnRkLmNvbS9MaW5lRGV0YWlsUXVlcnk=    [64] => HTTP/1.1 302 Moved Temporarily    [65] => Server: nginx    [66] => Date: Tue, 03 Dec 2013 12:00:55 GMT    [67] => Content-Type: text/html;charset=utf-8    [68] => Content-Length: 0    [69] => Connection: close    [70] => Set-Cookie: Wcity_xm=8A85502B900F458662118B57E5CEA9A5; Path=/; HttpOnly    [71] => Location: http://mybus.xiamentd.com/LineDetailQuery?return=2&tel=null&model=null    [72] => HTTP/1.1 302 Moved Temporarily    [73] => Server: nginx    [74] => Date: Tue, 03 Dec 2013 12:00:55 GMT    [75] => Content-Length: 0    [76] => Connection: close    [77] => Set-Cookie: JSESSIONID=5BE8764088155917F3F2EB96D785244E; Path=/    [78] => P3P: CP=CAO PSA OUR    [79] => Set-Cookie: HttpOnly=true    [80] => Set-Cookie: CALLBACKURL=http%3A%2F%2Fmybus.xiamentd.com%2FLineDetailQuery%3Freturn%3D2%26tel%3Dnull%26model%3Dnull%26random%3D1386072055352; Path=/    [81] => Location: http://xiamentd.com/getphoneinfo2.do?code=aHR0cDovL215YnVzLnhpYW1lbnRkLmNvbS9MaW5lRGV0YWlsUXVlcnk=    [82] => HTTP/1.1 302 Moved Temporarily    [83] => Server: nginx    [84] => Date: Tue, 03 Dec 2013 12:00:55 GMT    [85] => Content-Type: text/html;charset=utf-8    [86] => Content-Length: 0    [87] => Connection: close    [88] => Set-Cookie: Wcity_xm=D152A1B50778E8C4B541694ABFC67B4F; Path=/; HttpOnly    [89] => Location: http://mybus.xiamentd.com/LineDetailQuery?return=2&tel=null&model=null    [90] => HTTP/1.1 302 Moved Temporarily    [91] => Server: nginx    [92] => Date: Tue, 03 Dec 2013 12:00:55 GMT    [93] => Content-Length: 0    [94] => Connection: close    [95] => Set-Cookie: JSESSIONID=7FB9C88B0499882CB2E6020EF71CD936; Path=/    [96] => P3P: CP=CAO PSA OUR    [97] => Set-Cookie: HttpOnly=true    [98] => Set-Cookie: CALLBACKURL=http%3A%2F%2Fmybus.xiamentd.com%2FLineDetailQuery%3Freturn%3D2%26tel%3Dnull%26model%3Dnull%26random%3D1386072055777; Path=/    [99] => Location: http://xiamentd.com/getphoneinfo2.do?code=aHR0cDovL215YnVzLnhpYW1lbnRkLmNvbS9MaW5lRGV0YWlsUXVlcnk=    [100] => HTTP/1.1 302 Moved Temporarily    [101] => Server: nginx    [102] => Date: Tue, 03 Dec 2013 12:00:56 GMT    [103] => Content-Type: text/html;charset=utf-8    [104] => Content-Length: 0    [105] => Connection: close    [106] => Set-Cookie: Wcity_xm=14DE39508CA07769CDA8E7ABC0A0E74C; Path=/; HttpOnly    [107] => Location: http://mybus.xiamentd.com/LineDetailQuery?return=2&tel=null&model=null    [108] => HTTP/1.1 302 Moved Temporarily    [109] => Server: nginx    [110] => Date: Tue, 03 Dec 2013 12:00:56 GMT    [111] => Content-Length: 0    [112] => Connection: close    [113] => Set-Cookie: JSESSIONID=A32F3833ADFDF773B3F097A76EFC682A; Path=/    [114] => P3P: CP=CAO PSA OUR    [115] => Set-Cookie: HttpOnly=true    [116] => Set-Cookie: CALLBACKURL=http%3A%2F%2Fmybus.xiamentd.com%2FLineDetailQuery%3Freturn%3D2%26tel%3Dnull%26model%3Dnull%26random%3D1386072056242; Path=/    [117] => Location: http://xiamentd.com/getphoneinfo2.do?code=aHR0cDovL215YnVzLnhpYW1lbnRkLmNvbS9MaW5lRGV0YWlsUXVlcnk=    [118] => HTTP/1.1 302 Moved Temporarily    [119] => Server: nginx    [120] => Date: Tue, 03 Dec 2013 12:00:56 GMT    [121] => Content-Type: text/html;charset=utf-8    [122] => Content-Length: 0    [123] => Connection: close    [124] => Set-Cookie: Wcity_xm=4016ABA220428B15BFE4AD64599830A9; Path=/; HttpOnly    [125] => Location: http://mybus.xiamentd.com/LineDetailQuery?return=2&tel=null&model=null    [126] => HTTP/1.1 302 Moved Temporarily    [127] => Server: nginx    [128] => Date: Tue, 03 Dec 2013 12:00:56 GMT    [129] => Content-Length: 0    [130] => Connection: close    [131] => Set-Cookie: JSESSIONID=65B7BF224ECE837BF5E1DDC9C170E3FE; Path=/    [132] => P3P: CP=CAO PSA OUR    [133] => Set-Cookie: HttpOnly=true    [134] => Set-Cookie: CALLBACKURL=http%3A%2F%2Fmybus.xiamentd.com%2FLineDetailQuery%3Freturn%3D2%26tel%3Dnull%26model%3Dnull%26random%3D1386072056735; Path=/    [135] => Location: http://xiamentd.com/getphoneinfo2.do?code=aHR0cDovL215YnVzLnhpYW1lbnRkLmNvbS9MaW5lRGV0YWlsUXVlcnk=    [136] => HTTP/1.1 302 Moved Temporarily    [137] => Server: nginx    [138] => Date: Tue, 03 Dec 2013 12:00:57 GMT    [139] => Content-Type: text/html;charset=utf-8    [140] => Content-Length: 0    [141] => Connection: close    [142] => Set-Cookie: Wcity_xm=32F478E1B2F062FD1E6737996280A263; Path=/; HttpOnly    [143] => Location: http://mybus.xiamentd.com/LineDetailQuery?return=2&tel=null&model=null    [144] => HTTP/1.1 302 Moved Temporarily    [145] => Server: nginx    [146] => Date: Tue, 03 Dec 2013 12:00:57 GMT    [147] => Content-Length: 0    [148] => Connection: close    [149] => Set-Cookie: JSESSIONID=E2DC3FBFC8AF3DB7EA9208D13E3B9AB1; Path=/    [150] => P3P: CP=CAO PSA OUR    [151] => Set-Cookie: HttpOnly=true    [152] => Set-Cookie: CALLBACKURL=http%3A%2F%2Fmybus.xiamentd.com%2FLineDetailQuery%3Freturn%3D2%26tel%3Dnull%26model%3Dnull%26random%3D1386072057198; Path=/    [153] => Location: http://xiamentd.com/getphoneinfo2.do?code=aHR0cDovL215YnVzLnhpYW1lbnRkLmNvbS9MaW5lRGV0YWlsUXVlcnk=    [154] => HTTP/1.1 302 Moved Temporarily    [155] => Server: nginx    [156] => Date: Tue, 03 Dec 2013 12:00:57 GMT    [157] => Content-Type: text/html;charset=utf-8    [158] => Content-Length: 0    [159] => Connection: close    [160] => Set-Cookie: Wcity_xm=96AB72F98012ABF5D1A396EF2259D203; Path=/; HttpOnly    [161] => Location: http://mybus.xiamentd.com/LineDetailQuery?return=2&tel=null&model=null    [162] => HTTP/1.1 302 Moved Temporarily    [163] => Server: nginx.....

根据返回的头调整你的程序

$url = 'http://mybus.xiamentd.com/LineDetailQuery?lineId=118&direction=1';print_r(get_headers($url));
Array(    [0] => HTTP/1.1 302 Moved Temporarily    [1] => Server: nginx    [2] => Date: Tue, 03 Dec 2013 12:00:53 GMT    [3] => Content-Length: 0    [4] => Connection: close    [5] => Set-Cookie: JSESSIONID=8852C73A784F0B95C9DE7FB33812EDA0; Path=/.....


页面好像不停的302 跳转的页面抓取也没数据

所以你要调整你的程序
如有 302 则需要设置 CURLOPT_FOLLOWLOCATION
如有 cookie 需要设置 CURLOPT_COOKIEJAR 和 CURLOPT_COOKIEFILE

活人还能被尿憋死了?

所以你要调整你的程序
如有 302 则需要设置 CURLOPT_FOLLOWLOCATION
如有 cookie 需要设置 CURLOPT_COOKIEJAR 和 CURLOPT_COOKIEFILE

活人还能被尿憋死了?
这两个昨晚试了 但是不行 所以来指教下

$url = 'http://mybus.xiamentd.com/LineDetailQuery?lineId=118&direction=1';$cookie = tempnam('./','cookie');$ch = curl_init($url);curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);curl_setopt($ch,CURLOPT_COOKIEJAR,$cookie);curl_setopt($ch, CURLOPT_FOLLOWLOCATION,1);curl_exec($ch);curl_close($ch);$ch = curl_init($url);curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);curl_setopt($ch,CURLOPT_COOKIEFILE,$cookie);curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);$content = curl_exec($ch);echo $content; //空

Statement:
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn