A Beginner's Guide to HTTP Python Requests

All content on the network is accessible via request. If your Python application requires web information, you need to make a web request. This article will dive into Python requests. We will learn about the structure of web requests and how to make Python requests. Ultimately, you will be able to use the Python requests library, which will make the whole process much easier.

Key points

• HTTP (Hypertext Transfer Protocol) is a client-server protocol used to exchange data on the Web. It uses TCP as the transport protocol for reliable transport. HTTP requests are initiated by the client and processed by the server, and the server returns the corresponding response. HTTP is stateless, which means there is no association between two consecutive requests.
The
• The Python requests library simplifies the process of making HTTP requests in Python. It abstracts the complexity of making requests and provides an easy-to-use interface. This library allows sending Python HTTP requests from basic to complexity. It can be installed using pip and is used to issue GET requests, process status codes, read response body, and interact with the API.
• HTTP header provides additional information in HTTP communication. They can be customized in the Python requests library to provide additional information about the sender or message. For example, the User-Agent header provides information about the client making the request, while the Accept-Language header conveys a language that the client can understand.

Introduction to HTTP Request

To exchange data on the web, we first need a communication protocol. The protocol we use when browsing the web is Hypertext Transfer Protocol, or HTTP. HTTP uses TCP as the transport protocol because it requires reliable transport, and only TCP can guarantee this.

Suppose there is a resource we need - such as an HTML page on a web server located somewhere in the world. We want to access this resource, or in other words, we want to view the page in our web browser. The first thing we have to do is make HTTP requests. HTTP is a client-server protocol, which means that requests are initiated by the client.

After receiving the request, the server will process it and return the corresponding response.

The server's reply method may vary. It may send the resource we requested, or reply to the status code if an unexpected situation occurs.

In each communication protocol, the information needs to be located in a specific field. This is because both the client and the server should know how to interpret the request or response. In the next section, we will learn how HTTP requests and HTTP responses are built. We will also discuss the role of the most important fields.

HTTP Request

One of the most important design features of HTTP is that it is human-readable. This means that when we look at HTTP requests, we can read everything easily even with a lot of complexity at the bottom. Another feature of HTTP is that it is stateless. This means there is no association between two consecutive requests. The HTTP protocol does not remember any previous requests. This means that each request must contain everything the server needs to perform the request.

A valid HTTP request must contain the following elements:

• HTTP method—for example, GET or POST
• Version of HTTP protocol
• Path of the resource to be obtained

We can then add some optional headers that specify additional information about the sender or message. Examples of common HTTP request headers include User-Agent or the preferred natural language of the client. Both optional headers provide information about the client making the request.

This is an HTTP message example where we can clearly understand all the specified fields:

<code>~~~http
GET / HTTP/1.1
Host: www.google.com
Accept-Language: en-GB,en;q=0.5
~~~</code>

The first line specifies the request type and the version of the HTTP protocol. We then specify the host and language that the client making the request accepts. Usually, messages are much longer, but this can prompt their appearance.

HTTP Response

Now that we understand what the HTTP request looks like, we can continue to view the HTTP response.

HTTP responses usually contain the following elements:

• Version of HTTP protocol
• Status code with descriptive short message
• HTTP header list
• The message body containing the request resource

Now that we have introduced the basic elements you need, it is worth summarizing before taking the next step. It should now be clear that whenever a client wants to communicate with an HTTP server, it must create and send an HTTP request. Then, when the server receives it, it creates and sends an HTTP response.

We are finally ready to introduce the Python requests library.

Python requests library

The Python requests library allows you to send Python HTTP requests—from basic requests to complex requests. The Python requests library abstracts the complexity of making complex Python requests and provides an easy-to-use interface. In the next section, we will learn how to create a simple Python request and interpret the response. We will also learn about some of the features provided by the Python requests library.

Install Python requests

First, we need to install the Python requests library. Let's install it using pip:

<code>$ pip install requests</code>

After installing the Python requests library correctly, we can start using it.

Send our first GET request using Python requests

First, we need to create a Python file. In this example, we name it web.py. In this source file, insert the following code:

<code>import requests

URL = "https://www.google.com"
resp = requests.get(URL)

print(resp)</code>

This program issues a GET request to Google. If we run this program, we may get the following output:

<code>$ python web.py
<response></response></code>

So, what does this mean?

We have discussed status codes before. This output tells us that our request has been successfully received, understood, and processed. There are other codes, we can list some of the most common codes:

• 301 Moved Permanently. This is a redirect message. The URL of the resource we are looking for has been moved. The new URL comes with a response.
• 401 Unauthorized. This indicates a client error response. In this case, the server tells us that we must authenticate before we can continue to issue the request.
• 404 Not found. This also indicates the client's error response. In particular, this means that the server cannot find the resource we are looking for.

What if we want to check the status conditionally and provide different operations according to the status code? We can do this easily:

<code>~~~http
GET / HTTP/1.1
Host: www.google.com
Accept-Language: en-GB,en;q=0.5
~~~</code>

If we run the script now, we will get different results. Try it and see what we get. ?

If we also need the descriptive short message that comes with each status code, we can use resp.reason. For the 200 status code, we will simply get OK.

Check the response of Python request

At this point, we know how to make basic Python requests. After the request, we need to respond, right?

In the previous section, we saw how to get the status code for the response. Now we want to read the body of the response, i.e. the actual resource we requested. To do this, we need to use resp.content. Suppose we are looking for the Google homepage.

When we run the script, we get the following:

<code>$ pip install requests</code>

I added [...] because the resource we obtained (a text/html document) was too long to print. How long is it? We can use len(resp.content) to obtain this information. In the example above, it is 13931 bytes – there must be too many printing here!

(The following content is limited by space, only the summary is retained. Please refer to the original text for details)

Using API

One of the reasons why the Python requests library is so popular is that it makes interacting with the API very easy. In this case, we will use a simple API to predict a person's age, given their name. This API is called Agify.

Custom header

The HTTP header provides additional information to both parties to HTTP communication. In the following example, we will see how to change the header of an HTTP GET request. In particular, we will change the User-Agent and Accept-Language headers. User-Agent tells the server some information about the application, operating system, and vendor requesting the agent. The Accept-Language header conveys a language that the client can understand.

Conclusion

In this article, we discuss the HTTP protocol and give a brief theoretical introduction. We then looked at the Python requests library. We learned how to write basic Python HTTP requests and how to customize them according to our needs.

FAQs about HTTP requests in Python

(The following content is only reserved due to space limitations. Please refer to the original text for details)

• What is the requests library in Python?
• How to install the requests library?
• How to make a simple GET request using requests?
• How to handle query parameters in GET requests?
• How to make a POST request using requests?

The above is the detailed content of A Beginner's Guide to HTTP Python Requests. For more information, please follow other related articles on the PHP Chinese website!