Home > Article > Backend Development > How to get the value of an element in a crawler in python
There are many ways to get the value of an element in the crawler. Here are some commonly used methods:
import re html = "<a href='https://www.example.com'>Example</a>" links = re.findall(r"<a.*?href=['\"](.*?)['\"].*?>(.*?)</a>", html) for link in links: url = link[0] text = link[1] print("URL:", url) print("Text:", text)
from bs4 import BeautifulSoup html = "<h1>This is a title</h1>" soup = BeautifulSoup(html, 'html.parser') titles = soup.find_all('h1') for title in titles: print("Title:", title.text)
from lxml import etree html = "<p>This is a paragraph.</p>" tree = etree.HTML(html) paragraphs = tree.xpath('//p') for paragraph in paragraphs: print("Text:", paragraph.text)
These are common methods. Which method to use depends on the characteristics of the website you crawl and the data structure.
The above is the detailed content of How to get the value of an element in a crawler in python. For more information, please follow other related articles on the PHP Chinese website!