2017-05-12 23:39:52 [scrapy.extensions.logstats] INFO:
Crawled 3000 pages (at 61 pages/min), scraped 454 items (at 10 items/min)
Why do I visit more pages than actually enter items when I use scrapy to crawl?
It’s just that after crawling for a long time, there are not many pieces of data in the items.
Is there any way to solve it?
ringa_lee2017-05-18 10:49:40
Redirected, as you can see, redirection requires two pages to be requested. Also check to see if there are any page parsing errors.
