python爬虫防封IP的方法应该放到代码的哪个位置

Question

想要爬豆瓣电影，但是很容易403，就想代理IP或者修改请求头，但是看了很多资料，不知道他们那些代码应该放到整个程序的哪个位置，我用的是美丽汤和request，应该增添什么代码，增添到什么位置

Answer 1

BeautifulSoup’s Chinese name is originally called Beautiful Soup. . .

Complete the complaint, the server usually detects the requested IP address through IP packets, so simply modifying the content of the HTTP request generally does not work. The best way is to use the proxy function of Requests. Access can remove IP restrictions.

Answer 2

Please take a look
Python crawler word association video and code
https://zhuanlan.zhihu.com/p/...

Learn Python crawler to capture proxy IP and verification from Brother Huang.
https://zhuanlan.zhihu.com/p/...
Learn Python crawler to capture proxy IP from Huang Ge
https://zhuanlan.zhihu.com/p/...