python - scrapy cannot crawl data and cannot create txt files

Question

(www.sf.gg) is China's leading developer technology community. We hope to provide a pure, high-quality technical exchange platform for programming enthusiasts, learn, communicate and grow with developers, and create an era for developers!

phpcn_u1582 · Answer

It’s a problem with xpath parsing path, thank you everyone

天蓬老师 · Answer

There should be a problem with the writing mode, try ab.

大家讲道理 · Answer

Is it running on Linux? Is there any permission issue?

代言 · Answer

Looking at the log output, it seems that scrapy exited early when it encountered robot.txt. Try changing ROBOTSTXT_OBEY to ROBOTSTXT_OBEY = False in settings. If it still doesn’t work, stop and debug it

python - scrapy cannot crawl data and cannot create txt files

reply all(4)I'll reply