html - "DNS lookup failed" appears when crawling, but there is no problem when opening the web page. What is happening?

Question

RT, when using python to crawl, everything is normal at the beginning, but suddenly the web page freezes, and after waiting for a while, it returns to normal. However, when crawling again at this time, it will report "DNS lookup failed". I have tried many websites. , but the web page opens normally.

Please tell me the reasons and solutions. Kneel down and thank you! !

Answer 1

DNS lookup failed ： address "http" not found

http is a protocol, not part of the address, so DNS resolution fails. Let’s start from this aspect.

For example, if you want to parse Baidu’s website address:
Correct address: www.baidu.com
Incorrect address: http://www.baidu.com