I have many spiders, so I can’t manually create a debugger every time I debug (scrapy crawl spider_name)
I would like to right-click the spider file to run/debug directly.
欧阳克2017-06-28 09:27:49
Reference article: Summary of common problems with Scrapy crawlers
In the spider file, add the cmdline
calling method
import scrapy.cmdline
#Your Spider Class...
def main():
scrapy.cmdline.execute(['scrapy', 'crawl', 'your_spider_name'])
if __name__ == '__main__':
main()
过去多啦不再A梦2017-06-28 09:27:49
Create a new py file and use this to debug
from scrapy.cmdline import execute
import sys
import os
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
execute(["scrapy","crawl","YOUR_SPIDER"])