Home  >  Q&A  >  body text

Web crawler - python crawler, the data to be crawled is not in the source code of the web page, how to deal with it?

I want to capture the scores of live broadcast events: live broadcast links, but I can only capture the date, home and away teams, etc. using Scrapy. My question is: are the scores loaded via Script? Is it in bf4.js in this script? How to accurately obtain the score as shown below in this Request Method: GET situation?
The source code of the web page viewed through the developer tools is as follows:
<li label="Football, China League One, Wuhan Zall" id="saishi93287" data-time="2017-06- 24 15:00">15:00 <b>China League One 15th Round</b> Yunnan Lijiang<img src="//duihui.qiumibao.com/zuqiu/yunnanlijiang.png" > < span> - </span> <img src="//duihui.qiumibao.com/zuqiu/wuhanzhuoer.png" > Wuhan Zall<a href="/zhibo/zuqiu/2017/0624yunnanlijiangvswuhanzhuoer.htm" target="_blank">Wuhan Culture and Sports Guangzhou Competition PPTV</a> <a href="http://wenzi.zhibo8.cc/zhibo/zuqiu/2017/0624yunnanlijiangvswuhanzhuoer.htm" target="_blank">Text </a> <a href="//www.zhibo8.cc/shouji.htm" target="_blank">Watch live broadcast on mobile phone</a> <a href="http://www. 188bifen.com/" target="_blank">score</a> <a href="http://ogzq2.wanjiashe.com/game.php?sid=51&zhibo" target="_blank">UEFA Champions League New football uniform</a> </li>

That is, the score is not displayed in <span> - </span>. How can we capture the rendered score webpage?

世界只因有你世界只因有你2694 days ago1870

reply all(2)I'll reply

  • 巴扎黑

    巴扎黑2017-06-28 09:27:38

    数据在https://bifen4pc.qiumibao.com/json/list.htm?74520
    
    

    reply
    0
  • PHP中文网

    PHP中文网2017-06-28 09:27:38

    Then use Selenium to get it,

    Link description

    reply
    0
  • Cancelreply