Home  >  Article  >  Backend Development  >  How to Download Files from the Web in Python 3?

How to Download Files from the Web in Python 3?

Mary-Kate Olsen
Mary-Kate OlsenOriginal
2024-11-04 03:53:01796browse

How to Download Files from the Web in Python 3?

Downloading Files from the Web in Python 3

Introduction

When creating programs that interact with the internet, often times it is necessary to download files from a web server. In Python 3, there are multiple ways to accomplish this task.

Python 3 Solution

The initially provided code encounters an error because the function expects a byte type for the URL argument, but the extracted URL from the JAD file is a string. To download a file when the URL is stored as a string, convert it to a byte type using the UTF-8 encoding:

<code class="python">import urllib.request

def downloadFile(URL=None):
    h = urllib.request.urlopen(URL.encode('utf-8'))
    return h.read()

downloadFile(URL_from_file)</code>

Alternative Solutions:

There are several alternative methods to download files from the web:

  • urllib.request.urlopen: Obtain the contents of a web page by reading the response of urlopen:

    <code class="python">response = urllib.request.urlopen(URL)
    data = response.read() # a `bytes` object
    text = data.decode('utf-8') # a `str`</code>
  • urllib.request.urlretrieve: Download and save a file locally:

    <code class="python">urllib.request.urlretrieve(URL, file_name)</code>
  • urllib.request.urlopen shutil.copyfileobj: Offer a highly recommended and the most correct approach to file downloading:

    <code class="python">with urllib.request.urlopen(URL) as response, open(file_name, 'wb') as out_file:
      shutil.copyfileobj(response, out_file)</code>
  • urllib.request.urlopen write to bytes object: Simpler option, but recommended only for small files:

    <code class="python">with urllib.request.urlopen(URL) as response, open(file_name, 'wb') as out_file:
      data = response.read() # a `bytes` object
      out_file.write(data)</code>

Compressed Data Handling

Finally, the extraction of compressed data on-the-fly is also possible:

<code class="python">url = 'http://example.com/something.gz'
with urllib.request.urlopen(url) as response:
    with gzip.GzipFile(fileobj=response) as uncompressed:
        file_header = uncompressed.read(64) # a `bytes` object</code>

The above is the detailed content of How to Download Files from the Web in Python 3?. For more information, please follow other related articles on the PHP Chinese website!

Statement:
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn