Making HTTP Requests in Node.js

In previous articles, I’ve focused on creating a Node.js server to handle HTTP requests. This article looks at the problem in reverse, by showing you how to make HTTP requests from your Node.js applications. You may be asking yourself why you would want to do that. Two applications come to mind right away – web scraping and proxying. Scrapers are pieces of software which download web pages and programatically extract information from them. Proxy servers act as intermediaries, forwarding client requests to other servers and returning the responses.

Key Takeaways

• The request module in Node.js is a simple and flexible tool for making HTTP requests, including GET, POST, PUT, and DELETE. It is ideal for interacting with RESTful APIs and can be used for tasks such as web scraping and proxying.
• The request module can be parameterized to modify HTTP requests, allowing for configuration of parameters such as URI, method, timeout, and redirect handling. It also supports the submission of form data using POST requests and can handle cookies for tracking user interactions.
• While the request module is powerful and versatile, it is recommended to refer to its documentation for a comprehensive understanding of its capabilities. The module can also be combined with other modules like the file system module for more complex tasks such as streaming HTTP requests to and from files.

The Request Module

The simplest way to create HTTP requests in Node.js is by using the request module. Written by Mikeal Rogers, request allows you to make all types of HTTP requests, including GET, POST, PUT, and DELETE. Its flexibility makes the request module ideal for interacting with RESTful APIs. You can install request using the following npm command.

npm install request

Once you have installed request, create a new Node.js file containing the code shown below. The first line of the example is used to import request into the program. Making an HTTP request is as simple as calling the request() function. In this example, we are making a request to download the JSPro homepage. The second argument to request() is, of course, a callback function.

<span>var request = require("request");
</span><span>request("http://www.sitepoint.com", function(error<span>, response, body</span>) {
</span>  <span>console.log(body);
</span><span>});</span>

The request() callback function takes three arguments – error, response, and body. As the name implies, error contains information regarding any problems that occurred during the HTTP request. The response argument is an object containing a variety of data returned from the call, including the status code, headers, and much more. The third callback argument, body, is the body of the response. In the previous example, body contained the source code for the JSPro homepage.

The request module can also be combined with the file system module to stream HTTP requests to and from files. The following example again fetches JSPro. However, instead of displaying the source code in the console, it is written to a file named “jspro.htm”.

npm install request

Parameterizing Requests

The previous examples have created simple GET requests for URLs. However, we can modify the HTTP request by passing a configuration object as the first argument to request(). The following example uses the more powerful syntax to again download JSPro. Notice the numerous configuration parameters. The uri and method parameters tell request to send a GET request to JSPro.

<span>var request = require("request");
</span><span>request("http://www.sitepoint.com", function(error<span>, response, body</span>) {
</span>  <span>console.log(body);
</span><span>});</span>

The timeout parameter tells request to wait 10,000 milliseconds (ten seconds) for a response before giving up. The final two parameters deal with following redirects. Many pages perform redirects behind the scenes without the user’s knowledge. The followRedirect and maxRedirects parameters instruct request to follow up to ten 3xx HTTP redirects. Also, note that request() supports many other parameters which are not shown here.

Submitting Form Data

As previously mentioned, request supports more than just GET requests. One very common operation is the submission of form data using a POST request. The following example uses the form parameter of request() to send a user’s name as part of a POST. The PHP page then responds by greeting the user by name.

<span>var request = require("request");
</span><span>var fs = require("fs");
</span><span>request("http://www.sitepoint.com").pipe(fs.createWriteStream("jspro.htm"));</span>

Dealing with Cookies

Many sites track various user interactions using cookies. Cookies are small pieces of data that are passed back and forth between the client and server with every HTTP transaction. To facilitate the use of cookies, request provides the jar() and cookie() methods. Individual cookies are created using the cookie() method. On line 3 of the following example, a cookie is created which specifies the user’s name. The cookie is then added to the cookie jar created on line 2. When the HTTP request is made, the jar parameter is used to send the cookie jar to the server.

<span>var request = require("request");
</span><span>request({
</span>  <span>uri: "http://www.sitepoint.com",
</span>  <span>method: "GET",
</span>  <span>timeout: 10000,
</span>  <span>followRedirect: true,
</span>  <span>maxRedirects: 10
</span><span>}, function(error<span>, response, body</span>) {
</span>  <span>console.log(body);
</span><span>});</span>

When the response is received, the user will be greeted as John, the name stored in the cookie. The cookie jar is also updated with any changes made by the server. If you print out the cookie jar before and after the request, you’ll see that the server changed the value of the cookie from "John" to "Stranger". The same cookie jar can be passed to consecutive HTTP requests in order to mimic a real user’s interactions with a site.

Conclusion

This article has introduced the simple, yet powerful, request module. As this article is not comprehensive, I recommend at least browsing over the documentation to see what else request can do. Stay tuned for an upcoming article which will cover web scraping using request in more depth.

And if you enjoyed reading this post, you’ll love Learnable; the place to learn fresh skills and techniques from the masters. Members get instant access to all of SitePoint’s ebooks and interactive online courses, like Jump Start JavaScript.
Comments on this article are closed. Have a question about JavaScript? Why not ask it on our forums?

FAQs on Making HTTP Requests in Node.js

How can I make HTTP requests in Node.js?

You can make HTTP requests in Node.js using the built-in http or https modules for low-level control, or you can use third-party libraries like Axios, node-fetch, or the request module for simplified and higher-level HTTP requests.

What are the advantages of using a third-party library like Axios or node-fetch for HTTP requests in Node.js?

Third-party libraries offer more user-friendly APIs, promise-based or async/await support, and added features like request/response handling and automatic JSON parsing, making them popular choices for developers.

What is the difference between GET and POST requests in Node.js?

GET requests are used to retrieve data from a server, while POST requests are used to send data to the server for processing or storage, often used in forms or API requests.

Can I make GET and POST requests with the built-in http module in Node.js?

Yes, you can make GET and POST requests using the http or https modules, but it requires more low-level code for request configuration and response handling.

How do I make a simple GET request using the http module in Node.js?

You can use the http.get() method to make a GET request, passing the URL and a callback function to handle the response data.

What is the difference between synchronous and asynchronous HTTP requests in Node.js?

Synchronous requests block the event loop until the request is complete, whereas asynchronous requests allow your application to continue processing other tasks while waiting for the response.

How can I handle errors when making HTTP requests in Node.js?

You can handle errors by using try/catch blocks for synchronous requests, and by adding error handling in the callback or using promises for asynchronous requests.

Can I make authenticated requests using HTTP libraries in Node.js?

Yes, you can include authentication tokens or credentials in the request headers when using third-party libraries like Axios or node-fetch. Many APIs require authentication for access.

Is it possible to upload files in HTTP requests using Node.js?

Yes, you can upload files by sending a POST request with a multipart/form-data content type. Libraries like Axios and the request module provide options for handling file uploads.

How can I make concurrent or parallel HTTP requests in Node.js?

To make multiple HTTP requests concurrently, you can use asynchronous features like Promises or async/await, or you can use libraries that support parallel request handling.

The above is the detailed content of Making HTTP Requests in Node.js. For more information, please follow other related articles on the PHP Chinese website!