PHP中文网2017-04-17 17:50:02
I did a 0 o'clock ticket grab for the Taiwan Railway. It was also done with PHP at first. Later I found that there were a lot of tickets to grab at 0 o'clock every day. I made it concurrent and executed at the same time. I used nodejs to call php and solved the problem. Concurrency issues, although crude, can be used. . . It used to take 5 minutes to complete 20 tickets, but now it only takes more than 30 seconds to complete 20 tickets. . .
大家讲道理2017-04-17 17:50:02
I have written a lot of crawlers using PHP, and I am very good at processing data, but PHP itself is not suitable for things like crawlers. Maybe Python is worth a try, I haven’t tried it
天蓬老师2017-04-17 17:50:02
Why do so many people recommend node.js? Could it be that there were no crawlers when there was no node.js a few years ago?
高洛峰2017-04-17 17:50:02
The simplest one is javascript. It is completely scripted to write a crawler. It is recommended to use the Archer Cloud crawler framework.
PHPz2017-04-17 17:50:02
You can also write crawlers using javascript and write and execute them in the cloud. Source code address:
https://github.com/ShenJianSh...
天蓬老师2017-04-17 17:50:02
I have seen a crawler written in Go language. According to the author, the efficiency is super high
