如何使用 BeautifulSoup 抓取特定的Google天氣文字？

Question

如何使用 BeautifulSoup 在 Python 中找到課程文字「美國紐約市」？

嘗試複製影片進行練習，但不再有效。

嘗試在官方文件中找到一些內容，但沒有成功。或者我的 get_html_content 函數無法正常工作，Google 只是阻止我，從而返回一個空的 list / None ？

這是我目前的程式碼：

from django.shortcuts import render
import requests

def get_html_content(city):
    USER_AGENT = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36"
    LANGUAGE = "en-US,en;q=0.5"
    session = requests.Session()
    session.headers['User-Agent'] = USER_AGENT
    session.headers['Accept-Language'] = LANGUAGE
    session.headers['Content-Language'] = LANGUAGE
    city.replace(" ", "+")
    html_content = session.get(f"https://www.google.com/search?q=weather+in+{city}").text
    return html_content

def home(request):
    result = None
    if 'city' in request.GET: 
        city = request.GET.get('city')
        html_content = get_html_content(city)
        from bs4 import BeautifulSoup
        soup = BeautifulSoup(html_content, 'html.parser')
        soup.find_all('div', attrs={'class': 'wob_loc q8U8x'})
        **OR**
        soup.find_all('div', attrs={'id': 'wob_loc'})

--> 都會傳回空列表（= .find 方法回傳 None）

Answer 1

Google 頁面的佈局可能同時發生了變化，因此要獲取有關天氣的數據，您必須更改程式碼。例如：

import requests
from bs4 import BeautifulSoup


params = {'q':'weather in New York City, New York, USA', 'hl': 'en'}
headers = {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:108.0) Gecko/20100101 Firefox/108.0'}
cookies = {'CONSENT':"YES+cb.20220419-08-p0.cs+FX+111"}

url = 'https://www.google.com/search'


soup = BeautifulSoup(requests.get(url, params=params, headers=headers, cookies=cookies).content, 'html.parser')

for t in soup.select('#wob_dp [aria-label]'):
    how = t.find_next('img')['alt']
    temp = t.find_next('span').get_text(strip=True)
    print('{:<5} {:<20} {}'.format(t.text, how, temp))

列印:

Mon   Sunny                8
Tue   Cloudy               7
Wed   Partly cloudy        11
Thu   Rain                 7
Fri   Mostly cloudy        8
Sat   Partly cloudy        6
Sun   Scattered showers    8
Mon   Showers              8