Home  >  Q&A  >  body text

A usage in python is unclear

In the process of writing a crawler in Python (crawling Wikipedia entries), the url["href"] appeared during the output of the iterator. I thought it should be a usage in the iterator, but I couldn't find it. Please help. This usage means, thank you

#coding:utf-8

import urllib
import urllib2
import re
from bs4 import BeautifulSoup

resp = urllib2.urlopen("https://en.wikipedia.org/wiki/Main_Page").read()

soup = BeautifulSoup(resp,"html.parser")

listurl = soup.findAll('a',href=re.compile("^/wiki/"))
for url in listurl:
        print url.get_text(),"------>","https://en.wikipedia.org"+url["href"]

The url["href"] in the last line has a truncation effect on the crawled data. Before adding it, the output is:
print url
Output: Disclaimers
After adding it, the output is: :
print url["href"]
Output:/wiki/Wikipedia:General_disclaimer
Solution, thank you

阿神阿神2711 days ago748

reply all(1)I'll reply

  • 某草草

    某草草2017-05-18 11:01:46

    As long as the class implements the __getitem__ method, you can use square brackets to get the value.

                                                       
    In [16]: class A():                                
        ...:     def __getitem__(self,a):              
        ...:         return a                          
        ...:                                           
                                                       
    In [17]: a = A()                                   
                                                       
    In [18]: a['a'], a[1]                              
    Out[18]: ('a', 1)                                  

    reply
    0
  • Cancelreply