python - Scrapy uses xpath to report errors in Chinese

Question

Problem Description

links = sel.xpath('//i[contains(@title,"置顶")]/following-sibling::a/@href').extract()

Error: ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters

Answer 1

See the article: Solve the problem of Chinese error reporting when xpath is used in Scrapy

Solution

Method 1: Convert the entire xpath statement to Unicode

links = sel.xpath(u'//i[contains(@title,"置顶")]/following-sibling::a/@href').extract()

Method 2: Use the title variable that has been converted to Unicode in the xpath statement

title = u"置顶"
links = sel.xpath('//i[contains(@title,"%s")]/following-sibling::a/@href' %(title)).extract()

Method 3: Directly use the variable syntax in xpath ($ symbol plus variable name)$title, and pass the parameter title

links = sel.xpath('//i[contains(@title,$title)]/following-sibling::a/@href', title="置顶").extract()

Answer 2

Try adding a u before the whole string