search

Home  >  Q&A  >  body text

windows - Python爬虫爬资源时由于连接方在一段时间后没有正确答复或连接的主机没有反应,连接尝试失败。怎么破?

迷茫迷茫2801 days ago4424

reply all(2)I'll reply

  • 高洛峰

    高洛峰2017-04-18 10:16:55

    You can use the scrapy framework to implement the crawler. You can try 3 times if the crawl fails. If it still fails, you can customize it and write it into the log, and you can deal with it later by yourself

    reply
    0
  • PHPz

    PHPz2017-04-18 10:16:55

    requests.get has a timeout parameter.

     a = requests.get("http://www.baidu.com",timeout = 500)

    reply
    0
  • Cancelreply