搜索

首页  >  问答  >  正文

python - Scrapy模拟登陆遇到404问题

用python模拟登陆一个网站,一直遇到404问题,求指导!

代码

-- 编码:utf-8 --

导入 scrapy
from scrapy.http import Request, FormRequest
from scrapy.selector import Selector

类 StackSpiderSpider(scrapy.Spider):

雷雷

调试信息
2017-04-18 11:19:23 [scrapy.utils.log] INFO: Scrapy 1.3.3 已启动(bot: text5)
2017-04-18 11:19:23 [scrapy.utils.log ] 信息:覆盖设置:{'NEWSPIDER_MO
DULE':'text5.spiders','SPIDER_MODULES':['text5.spiders'],'BOT_NAME':'text5'
}
2017-04-18 11:19: 23 [scrapy.middleware]信息:启用扩展:
['scrapy.extensions.logstats.LogStats',
'scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.corestats.CoreStats']
2017-04- 18 11:19:24 [scrapy.middleware] 信息:启用下载器中间件:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware' ,
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'scrapy.downloadermiddlewares.retry.RetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy .downloader中间件.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2017-04-18 11:19:24 [scrapy.middleware]信息:启用蜘蛛中间件:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spi中间件.depth.DepthMiddleware']
2017-04-18 11:19:24 [scrapy.middleware] INFO: 启用项目管道:
[]
2017-04-18 11:19:24 [scrapy.core.engine] INFO : 蜘蛛打开了
2017-04-18 11:19:24 [scrapy.extensions.logstats] INFO: 抓取了 0 个页面 (at 0 pag
es/min),抓取了 0 个项目 (at 0 items/min)
2017-04 -18 11:19:24 [scrapy.extensions.telnet] DEBUG:Telnet 控制台监听 127.0.0.1:6023
2017-04-18 11:19:24 [scrapy.core.engine] DEBUG:已爬网(200 ) overflow.com/users/login>; (参考:无)
1145f3f2e28e56c298bc28a1a735254b

2017-04-18 11:19:25 [scrapy.core.engine] 调试:已爬网(404)overflow.com/search?q=&ssrc=&openid_username=&oauth_server=&oauth_version=&fkey =
1145f3f2e28e56c298bc28a1a735254b&密码=wanglihong1993&email=1067863906%40qq.c
om&openid_identifier=> (参考:https://stackoverflow.com/use...
2017-04-18 11:19:25 [scrapy.spidermiddlewares.httperror] 信息:忽略响应
<404 https://stackoverflow.com/sea ...
auth_version=&fkey=1145f3f2e28e56c298bc28a1a735254b&password=wanglihong1993&emai
l=1067863906%40qq.com&openid_identifier=>:HTTP状态代码未处理或不允许
2017-04-18 11 :19:25 [scrapy.core。引擎] INFO: 关闭蜘蛛(已完成)
2017-04-18 11:19:25 [scrapy.statscollectors] INFO: 转储 Scrapy stats:
{'downloader/request_bytes': 881,
'downloader/request_count': 2,
'下载器/request_method_count/GET': 2,
'下载器/response_bytes': 12631,
'下载器/response_count': 2,
'下载器/response_status_count/200': 1,
'下载器/response_status_count/404': 1 ,
'finish_reason': '完成',
'finish_time': datetime.datetime(2017, 4, 18, 3, 19, 25, 143000),
'log_count/DEBUG': 3,
'log_count/INFO': 8,
'request_depth_max': 1,
'response_received_count': 2,
'调度器/出队': 2,
'调度器/出队/内存': 2,
'调度器/入队': 2,
'调度器/入队' /内存': 2,
'start_time': datetime.datetime(2017, 4, 18, 3, 19, 24, 146000)}
2017-04-18 11:19:25 [scrapy.core.engine] 信息:蜘蛛关闭(完成)

黄舟黄舟2753 天前1052

全部回复(1)我来回复

  • PHPz

    PHPz2017-05-18 11:03:22

    老弟,你的密码泄漏了

    回复
    0
  • 取消回复