首页 >后端开发 >php教程 >CURL抓取失败请教

CURL抓取失败请教

WBOY
WBOY原创
2016-06-23 14:38:471456浏览

问个问题哈~
用curl抓取不到这个页面的数据~请教一下方法
http://mybus.xiamentd.com/LineDetailQuery?lineId=118&direction=1
thx!


回复讨论(解决方案)

根据返回的头调整你的程序

$url = 'http://mybus.xiamentd.com/LineDetailQuery?lineId=118&direction=1';print_r(get_headers($url));
Array(    [0] => HTTP/1.1 302 Moved Temporarily    [1] => Server: nginx    [2] => Date: Tue, 03 Dec 2013 12:00:53 GMT    [3] => Content-Length: 0    [4] => Connection: close    [5] => Set-Cookie: JSESSIONID=8852C73A784F0B95C9DE7FB33812EDA0; Path=/    [6] => P3P: CP=CAO PSA OUR    [7] => Set-Cookie: HttpOnly=true    [8] => Set-Cookie: CALLBACKURL=http%3A%2F%2Fmybus.xiamentd.com%2FLineDetailQuery%3FlineId%3D118%26direction%3D1%26random%3D1386072053450; Path=/    [9] => Location: http://xiamentd.com/getphoneinfo2.do?code=aHR0cDovL215YnVzLnhpYW1lbnRkLmNvbS9MaW5lRGV0YWlsUXVlcnk=    [10] => HTTP/1.1 302 Moved Temporarily    [11] => Server: nginx    [12] => Date: Tue, 03 Dec 2013 12:00:53 GMT    [13] => Content-Type: text/html;charset=utf-8    [14] => Content-Length: 0    [15] => Connection: close    [16] => Set-Cookie: Wcity_xm=4F706AE207628F6A3DE165FA08BE9E20; Path=/; HttpOnly    [17] => Location: http://mybus.xiamentd.com/LineDetailQuery?return=2&tel=null&model=null    [18] => HTTP/1.1 302 Moved Temporarily    [19] => Server: nginx    [20] => Date: Tue, 03 Dec 2013 12:00:53 GMT    [21] => Content-Length: 0    [22] => Connection: close    [23] => Set-Cookie: JSESSIONID=09C8413DA570C390ED8F9F1346C1D78E; Path=/    [24] => P3P: CP=CAO PSA OUR    [25] => Set-Cookie: HttpOnly=true    [26] => Set-Cookie: CALLBACKURL=http%3A%2F%2Fmybus.xiamentd.com%2FLineDetailQuery%3Freturn%3D2%26tel%3Dnull%26model%3Dnull%26random%3D1386072053965; Path=/    [27] => Location: http://xiamentd.com/getphoneinfo2.do?code=aHR0cDovL215YnVzLnhpYW1lbnRkLmNvbS9MaW5lRGV0YWlsUXVlcnk=    [28] => HTTP/1.1 302 Moved Temporarily    [29] => Server: nginx    [30] => Date: Tue, 03 Dec 2013 12:00:54 GMT    [31] => Content-Type: text/html;charset=utf-8    [32] => Content-Length: 0    [33] => Connection: close    [34] => Set-Cookie: Wcity_xm=E1C22FC11A9DC09587B66D224257D91C; Path=/; HttpOnly    [35] => Location: http://mybus.xiamentd.com/LineDetailQuery?return=2&tel=null&model=null    [36] => HTTP/1.1 302 Moved Temporarily    [37] => Server: nginx    [38] => Date: Tue, 03 Dec 2013 12:00:54 GMT    [39] => Content-Length: 0    [40] => Connection: close    [41] => Set-Cookie: JSESSIONID=B3034C324C287382DED4EF07D81CA04F; Path=/    [42] => P3P: CP=CAO PSA OUR    [43] => Set-Cookie: HttpOnly=true    [44] => Set-Cookie: CALLBACKURL=http%3A%2F%2Fmybus.xiamentd.com%2FLineDetailQuery%3Freturn%3D2%26tel%3Dnull%26model%3Dnull%26random%3D1386072054419; Path=/    [45] => Location: http://xiamentd.com/getphoneinfo2.do?code=aHR0cDovL215YnVzLnhpYW1lbnRkLmNvbS9MaW5lRGV0YWlsUXVlcnk=    [46] => HTTP/1.1 302 Moved Temporarily    [47] => Server: nginx    [48] => Date: Tue, 03 Dec 2013 12:00:54 GMT    [49] => Content-Type: text/html;charset=utf-8    [50] => Content-Length: 0    [51] => Connection: close    [52] => Set-Cookie: Wcity_xm=C38E38FF759C0707F5D45057227AA363; Path=/; HttpOnly    [53] => Location: http://mybus.xiamentd.com/LineDetailQuery?return=2&tel=null&model=null    [54] => HTTP/1.1 302 Moved Temporarily    [55] => Server: nginx    [56] => Date: Tue, 03 Dec 2013 12:00:54 GMT    [57] => Content-Length: 0    [58] => Connection: close    [59] => Set-Cookie: JSESSIONID=CEA724315DE5284BA898B7BDED29BAB8; Path=/    [60] => P3P: CP=CAO PSA OUR    [61] => Set-Cookie: HttpOnly=true    [62] => Set-Cookie: CALLBACKURL=http%3A%2F%2Fmybus.xiamentd.com%2FLineDetailQuery%3Freturn%3D2%26tel%3Dnull%26model%3Dnull%26random%3D1386072054876; Path=/    [63] => Location: http://xiamentd.com/getphoneinfo2.do?code=aHR0cDovL215YnVzLnhpYW1lbnRkLmNvbS9MaW5lRGV0YWlsUXVlcnk=    [64] => HTTP/1.1 302 Moved Temporarily    [65] => Server: nginx    [66] => Date: Tue, 03 Dec 2013 12:00:55 GMT    [67] => Content-Type: text/html;charset=utf-8    [68] => Content-Length: 0    [69] => Connection: close    [70] => Set-Cookie: Wcity_xm=8A85502B900F458662118B57E5CEA9A5; Path=/; HttpOnly    [71] => Location: http://mybus.xiamentd.com/LineDetailQuery?return=2&tel=null&model=null    [72] => HTTP/1.1 302 Moved Temporarily    [73] => Server: nginx    [74] => Date: Tue, 03 Dec 2013 12:00:55 GMT    [75] => Content-Length: 0    [76] => Connection: close    [77] => Set-Cookie: JSESSIONID=5BE8764088155917F3F2EB96D785244E; Path=/    [78] => P3P: CP=CAO PSA OUR    [79] => Set-Cookie: HttpOnly=true    [80] => Set-Cookie: CALLBACKURL=http%3A%2F%2Fmybus.xiamentd.com%2FLineDetailQuery%3Freturn%3D2%26tel%3Dnull%26model%3Dnull%26random%3D1386072055352; Path=/    [81] => Location: http://xiamentd.com/getphoneinfo2.do?code=aHR0cDovL215YnVzLnhpYW1lbnRkLmNvbS9MaW5lRGV0YWlsUXVlcnk=    [82] => HTTP/1.1 302 Moved Temporarily    [83] => Server: nginx    [84] => Date: Tue, 03 Dec 2013 12:00:55 GMT    [85] => Content-Type: text/html;charset=utf-8    [86] => Content-Length: 0    [87] => Connection: close    [88] => Set-Cookie: Wcity_xm=D152A1B50778E8C4B541694ABFC67B4F; Path=/; HttpOnly    [89] => Location: http://mybus.xiamentd.com/LineDetailQuery?return=2&tel=null&model=null    [90] => HTTP/1.1 302 Moved Temporarily    [91] => Server: nginx    [92] => Date: Tue, 03 Dec 2013 12:00:55 GMT    [93] => Content-Length: 0    [94] => Connection: close    [95] => Set-Cookie: JSESSIONID=7FB9C88B0499882CB2E6020EF71CD936; Path=/    [96] => P3P: CP=CAO PSA OUR    [97] => Set-Cookie: HttpOnly=true    [98] => Set-Cookie: CALLBACKURL=http%3A%2F%2Fmybus.xiamentd.com%2FLineDetailQuery%3Freturn%3D2%26tel%3Dnull%26model%3Dnull%26random%3D1386072055777; Path=/    [99] => Location: http://xiamentd.com/getphoneinfo2.do?code=aHR0cDovL215YnVzLnhpYW1lbnRkLmNvbS9MaW5lRGV0YWlsUXVlcnk=    [100] => HTTP/1.1 302 Moved Temporarily    [101] => Server: nginx    [102] => Date: Tue, 03 Dec 2013 12:00:56 GMT    [103] => Content-Type: text/html;charset=utf-8    [104] => Content-Length: 0    [105] => Connection: close    [106] => Set-Cookie: Wcity_xm=14DE39508CA07769CDA8E7ABC0A0E74C; Path=/; HttpOnly    [107] => Location: http://mybus.xiamentd.com/LineDetailQuery?return=2&tel=null&model=null    [108] => HTTP/1.1 302 Moved Temporarily    [109] => Server: nginx    [110] => Date: Tue, 03 Dec 2013 12:00:56 GMT    [111] => Content-Length: 0    [112] => Connection: close    [113] => Set-Cookie: JSESSIONID=A32F3833ADFDF773B3F097A76EFC682A; Path=/    [114] => P3P: CP=CAO PSA OUR    [115] => Set-Cookie: HttpOnly=true    [116] => Set-Cookie: CALLBACKURL=http%3A%2F%2Fmybus.xiamentd.com%2FLineDetailQuery%3Freturn%3D2%26tel%3Dnull%26model%3Dnull%26random%3D1386072056242; Path=/    [117] => Location: http://xiamentd.com/getphoneinfo2.do?code=aHR0cDovL215YnVzLnhpYW1lbnRkLmNvbS9MaW5lRGV0YWlsUXVlcnk=    [118] => HTTP/1.1 302 Moved Temporarily    [119] => Server: nginx    [120] => Date: Tue, 03 Dec 2013 12:00:56 GMT    [121] => Content-Type: text/html;charset=utf-8    [122] => Content-Length: 0    [123] => Connection: close    [124] => Set-Cookie: Wcity_xm=4016ABA220428B15BFE4AD64599830A9; Path=/; HttpOnly    [125] => Location: http://mybus.xiamentd.com/LineDetailQuery?return=2&tel=null&model=null    [126] => HTTP/1.1 302 Moved Temporarily    [127] => Server: nginx    [128] => Date: Tue, 03 Dec 2013 12:00:56 GMT    [129] => Content-Length: 0    [130] => Connection: close    [131] => Set-Cookie: JSESSIONID=65B7BF224ECE837BF5E1DDC9C170E3FE; Path=/    [132] => P3P: CP=CAO PSA OUR    [133] => Set-Cookie: HttpOnly=true    [134] => Set-Cookie: CALLBACKURL=http%3A%2F%2Fmybus.xiamentd.com%2FLineDetailQuery%3Freturn%3D2%26tel%3Dnull%26model%3Dnull%26random%3D1386072056735; Path=/    [135] => Location: http://xiamentd.com/getphoneinfo2.do?code=aHR0cDovL215YnVzLnhpYW1lbnRkLmNvbS9MaW5lRGV0YWlsUXVlcnk=    [136] => HTTP/1.1 302 Moved Temporarily    [137] => Server: nginx    [138] => Date: Tue, 03 Dec 2013 12:00:57 GMT    [139] => Content-Type: text/html;charset=utf-8    [140] => Content-Length: 0    [141] => Connection: close    [142] => Set-Cookie: Wcity_xm=32F478E1B2F062FD1E6737996280A263; Path=/; HttpOnly    [143] => Location: http://mybus.xiamentd.com/LineDetailQuery?return=2&tel=null&model=null    [144] => HTTP/1.1 302 Moved Temporarily    [145] => Server: nginx    [146] => Date: Tue, 03 Dec 2013 12:00:57 GMT    [147] => Content-Length: 0    [148] => Connection: close    [149] => Set-Cookie: JSESSIONID=E2DC3FBFC8AF3DB7EA9208D13E3B9AB1; Path=/    [150] => P3P: CP=CAO PSA OUR    [151] => Set-Cookie: HttpOnly=true    [152] => Set-Cookie: CALLBACKURL=http%3A%2F%2Fmybus.xiamentd.com%2FLineDetailQuery%3Freturn%3D2%26tel%3Dnull%26model%3Dnull%26random%3D1386072057198; Path=/    [153] => Location: http://xiamentd.com/getphoneinfo2.do?code=aHR0cDovL215YnVzLnhpYW1lbnRkLmNvbS9MaW5lRGV0YWlsUXVlcnk=    [154] => HTTP/1.1 302 Moved Temporarily    [155] => Server: nginx    [156] => Date: Tue, 03 Dec 2013 12:00:57 GMT    [157] => Content-Type: text/html;charset=utf-8    [158] => Content-Length: 0    [159] => Connection: close    [160] => Set-Cookie: Wcity_xm=96AB72F98012ABF5D1A396EF2259D203; Path=/; HttpOnly    [161] => Location: http://mybus.xiamentd.com/LineDetailQuery?return=2&tel=null&model=null    [162] => HTTP/1.1 302 Moved Temporarily    [163] => Server: nginx.....

根据返回的头调整你的程序

$url = 'http://mybus.xiamentd.com/LineDetailQuery?lineId=118&direction=1';print_r(get_headers($url));
Array(    [0] => HTTP/1.1 302 Moved Temporarily    [1] => Server: nginx    [2] => Date: Tue, 03 Dec 2013 12:00:53 GMT    [3] => Content-Length: 0    [4] => Connection: close    [5] => Set-Cookie: JSESSIONID=8852C73A784F0B95C9DE7FB33812EDA0; Path=/.....


页面好像不停的302 跳转的页面抓取也没数据

所以你要调整你的程序
如有 302 则需要设置 CURLOPT_FOLLOWLOCATION
如有 cookie 需要设置 CURLOPT_COOKIEJAR 和 CURLOPT_COOKIEFILE

活人还能被尿憋死了?

所以你要调整你的程序
如有 302 则需要设置 CURLOPT_FOLLOWLOCATION
如有 cookie 需要设置 CURLOPT_COOKIEJAR 和 CURLOPT_COOKIEFILE

活人还能被尿憋死了?
这两个昨晚试了 但是不行 所以来指教下

$url = 'http://mybus.xiamentd.com/LineDetailQuery?lineId=118&direction=1';$cookie = tempnam('./','cookie');$ch = curl_init($url);curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);curl_setopt($ch,CURLOPT_COOKIEJAR,$cookie);curl_setopt($ch, CURLOPT_FOLLOWLOCATION,1);curl_exec($ch);curl_close($ch);$ch = curl_init($url);curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);curl_setopt($ch,CURLOPT_COOKIEFILE,$cookie);curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);$content = curl_exec($ch);echo $content; //空

声明:
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系admin@php.cn