Recently, I watched Tanzhou Education’s online live courses, and I was quite impressed by the practicality of the teacher’s lectures. Just study it as your own notes. We all know that when learning a program, you first copy it and then create it. Here, I completely follow the teacher’s explanation and copy it for study.
1. Python captures pictures of Douban girls.
Tools: python3.6.0; bs4.6.0; You can upgrade online: pip install update buautifulsoup4
1.pip list, you can view the local installation.
1. Crawl the Douban girl picture and know its address, url = ''.
2. Check the source code of the web page, F12, network, find the web page information captured on the left, and find the User-agent. The main purpose is to imitate browser login and prevent anti-crawlers.
Find the element element. What we want is the img tag, the image information in it, and the src link download address.
Write all codes according to the teacher
1 import urllib 2 import urllib.request 3 from bs4 import BeautifulSoup 4 url = '' 5 x=0 6 #获取源码 7 #自定义函数 8 #User-Agent模拟浏览器进行访问,反爬虫 9 def crawl(url):10 headers={'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3088.3 Safari/537.36'}11 req=urllib.request.Request(url,headers=headers)#创建对象12 page=urllib.request.urlopen(req,timeout=20)#设置超时13 contents=page.read()#获取源码14 #print (contents.decode())15 soup = BeautifulSoup(contents,'html.parser')#html.parser主要是解析网页的一种形式。16 my_girl=soup.find_all('img')#找到所有img标签17 # 5.获取图片18 for girl in my_girl:#遍历19 link=girl.get('src')#获取src20 print(link)21 global x#全局变量22 # 6.下载 urlretrieve23 urllib.request.urlretrieve(link,'image\%s.jpg'%x)#下载,urlretrieve(需要下载的,路径)24 x+=125 print('正在下载第%s张'%x)26 #7.多页27 for page in range(1,10):#range本身自动生成整数序列,爬取多页图片。28 #page+=129 url='{}'.format(page)#30 #url = 'http://www.dbmeinv.com/?pager_offset=%d' % page31 crawl(url)32 33 print('图片下载完毕')
The final running result, the picture is saved in the image folder .
2. Grab Dianping and import it into excel. Source code of Zhao Ben's manifesto teacher.
import requestsfrom bs4 import BeautifulSoupimport xlwtdef get_content(url,headers=None,proxy=None): html=requests.get(url,headers=headers).contentreturn htmldef get_url(html): soup = BeautifulSoup(html,'html.parser') shop_url_list=soup.find_all('div',class_='tit')#class在Python是关键字,# 列表推导式return [i.find('a')['href'] for i in shop_url_list]#商品的详细信息,名字,评论,人均def get_detail_content(html): soup=BeautifulSoup(html,'html.parser') price=soup.find('span',id='avgPriceTitle').text evaluation=soup.find('span',id='comment_score').find_all('span',class_='item')#find_all是有多个,这里三个#for i in evaluation: # print(i.text)the_star=soup.find('div',class_='brief-info').find('span')['title'] title=soup.find('div',class_='breadcrumb').find('span').text comments=soup.find('span',id='reviewCount').text address=soup.find('span',itemprop='street-address').textprint(u'店名:'+title)for i in evaluation:print(i.text)print(price)print(u'评论数量:'+comments)print(u'地址:'+address.strip())print(u'评价星级:'+the_star)print('================')return (title,evaluation[0].text,evaluation[1].text,evaluation[2].text,price,comments,address,the_star)if __name__=='__main__': items=[] start_url=''base_url=''headers={'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3088.3 Safari/537.36','Cookie':'_hc.v=461407bd-5a08-f3fa-742e-681a434748bf.1496365678; __utma=1.1522471392.1496365678.1496365678.1496365678.1; __utmc=1; __utmz=1.1496365678.1.1.utmcsr=baidu|utmccn=(organic)|utmcmd=organic; PHOENIX_ID=0a0102b7-15c6659b548-25fc89; s_ViewType=10; JSESSIONID=E815A43E028078AFA73AF08D9C9E4A15; aburl=1; cy=344; cye=changsha; __mta=147134984.1496365814252.1496383356849.1496383925586.4'} start_html=get_content(start_url)#一页# url_list=get_url(start_html)#多页url_list = [base_url + url for url in get_url(start_html)]for i in url_list: detail_html=get_content(i,headers=headers) item=get_detail_content(detail_html) items.append(item)#写excel,txt差别,Excel:xlwgnewTable='DZDP.xls'wb=xlwt.Workbook(encoding='utf-8') ws=wb.add_sheet('test1') headData=['商户名字','口味评分','环境评分','服务评分','人均价格','评论数量','地址','商户评价']for colnum in range(0,8): ws.write(0,colnum,headData[colnum],xlwt.easyxf('font:bold on')) index=1lens=len(items)for j in range(0,lens):for i in range(0,8): ws.write(index,i,items[j][i]) index +=1wb.save(newTable)
The above is the detailed content of python crawler pictures, operate excel. For more information, please follow other related articles on the PHP Chinese website!

Python and C each have their own advantages, and the choice should be based on project requirements. 1) Python is suitable for rapid development and data processing due to its concise syntax and dynamic typing. 2)C is suitable for high performance and system programming due to its static typing and manual memory management.

Choosing Python or C depends on project requirements: 1) If you need rapid development, data processing and prototype design, choose Python; 2) If you need high performance, low latency and close hardware control, choose C.

By investing 2 hours of Python learning every day, you can effectively improve your programming skills. 1. Learn new knowledge: read documents or watch tutorials. 2. Practice: Write code and complete exercises. 3. Review: Consolidate the content you have learned. 4. Project practice: Apply what you have learned in actual projects. Such a structured learning plan can help you systematically master Python and achieve career goals.

Methods to learn Python efficiently within two hours include: 1. Review the basic knowledge and ensure that you are familiar with Python installation and basic syntax; 2. Understand the core concepts of Python, such as variables, lists, functions, etc.; 3. Master basic and advanced usage by using examples; 4. Learn common errors and debugging techniques; 5. Apply performance optimization and best practices, such as using list comprehensions and following the PEP8 style guide.

Python is suitable for beginners and data science, and C is suitable for system programming and game development. 1. Python is simple and easy to use, suitable for data science and web development. 2.C provides high performance and control, suitable for game development and system programming. The choice should be based on project needs and personal interests.

Python is more suitable for data science and rapid development, while C is more suitable for high performance and system programming. 1. Python syntax is concise and easy to learn, suitable for data processing and scientific computing. 2.C has complex syntax but excellent performance and is often used in game development and system programming.

It is feasible to invest two hours a day to learn Python. 1. Learn new knowledge: Learn new concepts in one hour, such as lists and dictionaries. 2. Practice and exercises: Use one hour to perform programming exercises, such as writing small programs. Through reasonable planning and perseverance, you can master the core concepts of Python in a short time.

Python is easier to learn and use, while C is more powerful but complex. 1. Python syntax is concise and suitable for beginners. Dynamic typing and automatic memory management make it easy to use, but may cause runtime errors. 2.C provides low-level control and advanced features, suitable for high-performance applications, but has a high learning threshold and requires manual memory and type safety management.


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

MinGW - Minimalist GNU for Windows
This project is in the process of being migrated to osdn.net/projects/mingw, you can continue to follow us there. MinGW: A native Windows port of the GNU Compiler Collection (GCC), freely distributable import libraries and header files for building native Windows applications; includes extensions to the MSVC runtime to support C99 functionality. All MinGW software can run on 64-bit Windows platforms.

SublimeText3 English version
Recommended: Win version, supports code prompts!

SublimeText3 Chinese version
Chinese version, very easy to use

VSCode Windows 64-bit Download
A free and powerful IDE editor launched by Microsoft

DVWA
Damn Vulnerable Web App (DVWA) is a PHP/MySQL web application that is very vulnerable. Its main goals are to be an aid for security professionals to test their skills and tools in a legal environment, to help web developers better understand the process of securing web applications, and to help teachers/students teach/learn in a classroom environment Web application security. The goal of DVWA is to practice some of the most common web vulnerabilities through a simple and straightforward interface, with varying degrees of difficulty. Please note that this software