Why Does `re.findall()` Throw a `TypeError: Can\'t Use a String Pattern on a Bytes-Like Object` When Extracting URLs?

TypeError: Can't Use a String Pattern on a Bytes-Like Object in re.findall()

While attempting to automatically fetch URLs from a web page, you may encounter the following error:

TypeError: can't use a string pattern on a bytes-like object in re.findall()

In your code, you use re.findall() to find matches for a regular expression regex. However, when you try to apply the regular expression to the HTML content you've fetched, you get the error.

Underlying Cause:

The issue stems from the fact that the HTML content you're working with is in byte form, whereas the regular expression you're using is in string form. The regular expression cannot be applied directly to a byte-like object.

Lösung:

To resolve this issue, you need to convert the HTML content to a string:

html = response.read().decode('utf-8')

This will decode the byte-like HTML content into a string, allowing the regular expression to be applied successfully.

Once you've made the conversion, you can proceed with using the regular expression to find the title of the web page. The corrected code should look like this:

import urllib.request
import re

url = "http://www.google.com"
regex = r'<title>(,+?)</title>'
pattern = re.compile(regex)

with urllib.request.urlopen(url) as response:
   html = response.read().decode('utf-8')

title = re.findall(pattern, html)
print(title)

The above is the detailed content of Why Does `re.findall()` Throw a `TypeError: Can\'t Use a String Pattern on a Bytes-Like Object` When Extracting URLs?. For more information, please follow other related articles on the PHP Chinese website!