Rumah > Artikel > pembangunan bahagian belakang > Bagaimana untuk menggunakan Python untuk xpath, JsonPath, dan bs4?

Bagaimana untuk menggunakan Python untuk xpath, JsonPath, dan bs4?

WBOYke hadapan: 2023-05-09 21:04:061557semak imbas

1.xpath

1.1 penggunaan xpath

Google memasang pemalam xpath terlebih dahulu, tekan ctrl + shift + x dan kotak hitam kecil akan muncul
Pasang perpustakaan lxmlpip install lxml ‐i https://pypi.douban.com/simple
Import lxml.etreefrom lxml import etree
etree .parse() untuk menghuraikan fail setempat html_tree = etree.parse('XX.html')
etree.HTML() fail respons pelayanhtml_tree = etree.HTML(response.read().decode('utf‐8')
.html_tree.xpath (laluan xpath)

1.2 Sintaks asas xpath

1 Pertanyaan Laluan

Cari semua nod keturunan, tanpa mengira hubungan hierarki
Cari nod anak langsung

2 pertanyaan predikat

//div[@id] 
//div[@id="maincontent"]

3 🎜>4. Pertanyaan kabur

//@class

5. Pertanyaan kandungan

//div[contains(@id, "he")] 
//div[starts‐with(@id, "he")]

6. 🎜>

//div/h2/text()

//div[@id="head" and @class="s_down"] 
//title | //price

1.4

Merangkak nilai butang carian Baidu

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8"/>
    <title>Title</title>
</head>
<body>
    <ul>
        <li id="l1" class="class1">北京</li>
        <li id="l2" class="class2">上海</li>
        <li id="d1">广州</li>
        <li>深圳</li>
    </ul>
</body>
</html>

1.5 Merangkak gambar bahan webmaster

from lxml import etree

# xpath解析
# 本地文件：                                          etree.parse
# 服务器相应的数据    response.read().decode(&#39;utf-8&#39;)  etree.HTML()


tree = etree.parse(&#39;xpath.html&#39;)

# 查找url下边的li
li_list = tree.xpath(&#39;//body/ul/li&#39;)
print(len(li_list))  # 4

# 获取标签中的内容
li_list = tree.xpath(&#39;//body/ul/li/text()&#39;)
print(li_list)  # [&#39;北京&#39;, &#39;上海&#39;, &#39;广州&#39;, &#39;深圳&#39;]

# 获取带id属性的li
li_list = tree.xpath(&#39;//ul/li[@id]&#39;)
print(len(li_list))  # 3

# 获取id为l1的标签内容
li_list = tree.xpath(&#39;//ul/li[@id="l1"]/text()&#39;)
print(li_list)  # [&#39;北京&#39;]

# 获取id为l1的class属性值
c1 = tree.xpath(&#39;//ul/li[@id="l1"]/@class&#39;)
print(c1)  # [&#39;class1&#39;]

# 获取id中包含l的标签
li_list = tree.xpath(&#39;//ul/li[contains(@id, "l")]/text()&#39;)
print(li_list)  # [&#39;北京&#39;, &#39;上海&#39;]
# 获取id以d开头的标签
li_list = tree.xpath(&#39;//ul/li[starts-with(@id,"d")]/text()&#39;)
print(li_list)  # [&#39;广州&#39;]
# 获取id为l2并且class为class2的标签
li_list = tree.xpath(&#39;//ul/li[@id="l2" and @class="class2"]/text()&#39;)
print(li_list)  # [&#39;上海&#39;]
# 获取id为l2或id为d1的标签
li_list = tree.xpath(&#39;//ul/li[@id="l2"]/text() | //ul/li[@id="d1"]/text()&#39;)
print(li_list)  # [&#39;上海&#39;, &#39;广州&#39;]

2. JsonPath

Pemasangan 2.1 pip

import urllib.request
from lxml import etree
url = &#39;http://www.baidu.com&#39;
headers = {
    &#39;User-Agent&#39;: &#39;Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36&#39;
}
request = urllib.request.Request(url=url, headers=headers)
response = urllib.request.urlopen(request)
content = response.read().decode(&#39;utf-8&#39;)
tree = etree.HTML(content)
value = tree.xpath(&#39;//input[@id="su"]/@value&#39;)
print(value)

Bagaimana untuk menggunakan Python untuk xpath, JsonPath, dan bs4?

2.2 Penggunaan jsonpath