网页爬虫 - python爬虫案例

Question

请问大家能不能给我一点提点，我想要知道有没有 简单 一点的 爬虫 案例可以让我摸索学习

---

目前已知的爬虫工具有:

1. Beautiful Soup

2. Scrapy

3. cola

4. pyspider

5. PyQuery

6. grab

Answer 1

If you really have no idea, here is some information for your reference, but the key point is that you still have to read it...

First of all, what is a web crawler:

1. You can simply take a look at wiki-web crawler

2. Come to this website again: The University Mathematics School has some simple teachings (and videos), which I believe are very suitable for beginners. You can start from this article: What is a web crawler

Then comes Introduction tutorial:

1. In fact, there is a very simple method. Just pick a crawler tool to read the document. If you have no direction, you can choose to read the Beautiful Soup Chinese document. It is a Chinese version and it is not too complicated. Take some time. You can read the whole thing.

2. Just now, the University Mathematics Hall has a series of introductory teaching and practical teaching. I think it should be worth referring to. Here are the first few articles of the introductory course:

   1. Start writing a web crawler (Crawler) using Python

   2. How to install Jupyter (Ipython Notebook)

   3. Introduction to Jupyter operation (1)

   4. How to use GET to crawl web content?

   5. How to use POST to capture web content?

   6. How to use Python package: BeautifulSoup4 to analyze web content?

   7. How to use Python requests and BeautifulSoup4 to complete Taobao crawler?

The next step is to understand those tools and crawler framework:

1. This place is very complete: Python crawler tool list with Github code download link

2. This blog also has a lot of teachings

For discussions about crawler tools and frameworks, please refer to this article Zhihu: When writing crawlers in Python, which method and framework is better?

---

I think there are a lot of resources on the Internet, you can give them a try. Everything is difficult at the beginning, so come on!