Node.js crawls Douban data example

This time I will bring you Node.jsan example of crawling Douban data, what are the notes of Node.js crawling Douban data, the following is a practical case, let's come together take a look.

I always thought that my vue was good, and I always thought that webpack was good. When I was browsing node in MOOC today, I realized that I was still far behind. As we all know, vue-cli is based on webpack, and webpack is based on node. If you don’t know node, how can you understand webpack? So I asked myself a question to crawl Douban data, which is still in its infancy. Today I will briefly talk about the data crawled from Douban and display it in your own way on another page. I will follow up later.

1. Problems that need to be solved

1. Building services

2. How to process the crawled data

3. How to automatically open the default browser

2. Build a service

It’s good to build a service There are several ways. I used http at the beginning, but one drawback of http is that it cannot parse URLs of https protocol, so I used express. To parse URLs of https protocol, I used the request package. Douban’s URL is https,

Today I crawled the URL https://movie.douban.com/chart; as shown below, there are three parts I want to get, picture, movie name , movie link.

3. How to process the crawled data

We use How to deal with the data crawled by request? The cheerio package allows us to process crawled html data like Jq.

①. First parse the data and get the html data of the crawled web page;

②. Then use the cheerio package to operate the crawled data and get the data you want.

③. Get the data, create html, and output it to the page. As shown in the picture below, I used string splicing. The method is a bit stupid and I haven’t found a better method yet.

4. How to automatically open the default browser

I don’t know if you have looked at the configuration of webpack in vue-cli. Automatically Open the browser and see the opn package used by vue-cli.

This package is very convenient to use. Just import the package and call opn (url) directly;

5. Display

I believe you have mastered the method after reading the case in this article. Please come for more exciting information. Pay attention to other related articles on php Chinese website!

Recommended reading:

Use H5 to add and prohibit scaling

How to operate json and echarts charts with WebGL

html5How to achieve the animation effect of picture rotation

The above is the detailed content of Node.js crawls Douban data example. For more information, please follow other related articles on the PHP Chinese website!