Home  >  Q&A  >  body text

If scrapy wants to crawl the content of the entire website, does it need to get the URLs of all pages into the queue before starting to extract the content?

scrapy If you want to crawl the entire site, do you need to get the URLs of all pages into the queue before starting to extract the content?

LaicalLaical1634 days ago988

reply all(0)I'll reply

No reply
  • Cancelreply