Home  >  Article  >  Backend Development  >  javascript - Python writes a web crawler to crawl the information visible after logging in

javascript - Python writes a web crawler to crawl the information visible after logging in

WBOY
WBOYOriginal
2016-12-01 00:25:481482browse

  1. The source code on a web page is as shown in the picture: javascript - Python writes a web crawler to crawl the information visible after logging in

2. Use phantomJS selenium to obtain and use BeautifulSoup to parse the code as shown in the figure:
javascript - Python writes a web crawler to crawl the information visible after logging in

3. Tried to use session to obtain cookies:
The cookies obtained on the login page are all different:
javascript - Python writes a web crawler to crawl the information visible after logging in

Please tell me how you can crawl down the information you see in the source code?
Thank you! ! !

Reply content:

  1. The source code on a webpage is as shown in the picture: javascript - Python writes a web crawler to crawl the information visible after logging in

2. Use phantomJS selenium to obtain and use BeautifulSoup to parse the code as shown in the figure:
javascript - Python writes a web crawler to crawl the information visible after logging in

3. I tried to use session to obtain cookies:
The cookies obtained on the login page are all different:
javascript - Python writes a web crawler to crawl the information visible after logging in

Please tell me how you can crawl down the information you see in the source code?
Thank you! ! !

It has been solved~
My login window cannot be found. .
I’m a beginner at crawling, so my questions are of little value, so please forgive me!

Statement:
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn