How to retrieve redirected URLs using the Python Requests library?

Retrieving Redirected URL using Python Requests Library

In the realm of web scraping and automation, it often becomes necessary to track the redirects that occur when accessing a web page. The Python Requests library provides comprehensive functionality for managing HTTP requests, including the ability to navigate redirects. This article aims to clarify how to retrieve the redirected URL using the Requests library, addressing a query raised in the community.

Problem:

Within a script, the allow_redirects=True flag is set to automatically follow redirects. However, the user wishes to determine the final URL to which the request is redirected. The question arises: how can one obtain this information programmatically?

Solution:

The solution lies in utilizing the response.history attribute provided by Requests. This attribute stores a list of responses that encompass the entire journey of redirects leading to the final destination. To access the final URL, the response.url property can be employed.

Here's a code snippet that demonstrates the process:

response = requests.get(someurl)
if response.history:
    print("Request was redirected")
    for resp in response.history:
        print(resp.status_code, resp.url)
    print("Final destination:")
    print(response.status_code, response.url)
else:
    print("Request was not redirected")

Demo:

Consider the following example where a request is made to a website configured to perform multiple redirects:

import requests
response = requests.get('http://httpbin.org/redirect/3')
for resp in response.history:
    print(resp.status_code, resp.url)
print(response.status_code, response.url)

Output:

302 http://httpbin.org/redirect/3
302 http://httpbin.org/redirect/2
302 http://httpbin.org/redirect/1
200 http://httpbin.org/get

As evident from the output, the code successfully logs the status codes and URLs of each redirect, as well as the final destination.

The above is the detailed content of How to retrieve redirected URLs using the Python Requests library?. For more information, please follow other related articles on the PHP Chinese website!