How to obtain real-time application and viewer data on the 58.com work page?

How to use crawlers to obtain real-time data of 58.com recruitment page?

When crawling 58.com's recruitment information, you often encounter a problem: the number of applicants and viewers displayed on the page are updated in real time, but the source code of the web page is displayed as 0. However, the Elements panel in the browser developer tool (F12) displays the correct data. This is because 58.com uses AJAX technology to load data asynchronously. This article will guide you on how to obtain this dynamic data.

Problem analysis

Taking a medical position in Beijing as an example, the page displays the number of applicants and viewers in real time, but the data in the source code is always 0, and the Elements panel data of F12 is consistent with the page. Extracting data directly from source code doesn't work.

Solution

The page data is loaded asynchronously through AJAX requests, and we need to find and grab the JSON data for these requests.

After analysis, 58.com uses specific API interfaces to provide these data. For example:

 <code>https://statisticszp.58.com/position/totalcount/?infoId=27988...</code>

This interface returns JSON data, for example:

 {
  "deliveryCount": 1141,
  "commentCount": 0,
  "infoCount": 4,
  "resumeReadPercent": 0,
  "referUrl": "",
  "nextUrl": "null"
}

Where "deliveryCount" represents the number of applicants, and "infoCount" represents the number of visitors.

Operation steps

1. Positioning AJAX requests: Use the browser developer tool (F12) to find AJAX requests related to page dynamic data in the Network panel.
2. Send API requests: Use crawler tools (such as Python's requests library) to send GET requests to the above API interface.
3. Analyze JSON data: Extract the "deliveryCount" and "infoCount" fields from the returned JSON data, which is the number of applicants and viewers in real time.

Through the above steps, you can successfully obtain the real-time application and viewer data of the 58.com recruitment page. Please note that infoId parameter needs to be replaced according to the target recruitment page. To avoid excessive pressure on the server, please control the crawling frequency.

The above is the detailed content of How to obtain real-time application and viewer data on the 58.com work page?. For more information, please follow other related articles on the PHP Chinese website!