最近学习Python,于是就用Python写了一个抓取Discuz!用户名的脚本,代码很少但是很搓。思路很简单,就是正则匹配title然后提取用户名写入文本文档。程序以百度站长社区为例(一共有40多万用户),挂在VPS上就没管了,虽然用了延时但是后来发现一共只抓取了50000多个用户名就被封了。。。
代码如下:
# -*- coding: utf-8 -*-
# Author: 天一
# Blog: http://www.90blog.org
# Version: 1.0
# 功能: Python抓取百度站长平台用户名脚本
import urllib
import urllib2
import re
import time
def BiduSpider():
pattern = re.compile(r'
uid=1
thedatas = []
while uid theUrl = "http://bbs.zhanzhang.baidu.com/home.php?mod=space&uid="+str(uid)
uid +=1
theResponse = urllib2.urlopen(theUrl)
thePage = theResponse.read()
#正则匹配用户名
theFindall = re.findall(pattern,thePage)
#等待0.5秒,以防频繁访问被禁止
time.sleep(0.5)
if theFindall :
#中文编码防止乱码输出
thedatas = theFindall[0].decode('utf-8').encode('gbk')
#写入txt文本文档
f = open('theUid.txt','a')
f.writelines(thedatas+'\n')
f.close()
if __name__ == '__main__':
BiduSpider()
最终成果如下:

ForhandlinglargedatasetsinPython,useNumPyarraysforbetterperformance.1)NumPyarraysarememory-efficientandfasterfornumericaloperations.2)Avoidunnecessarytypeconversions.3)Leveragevectorizationforreducedtimecomplexity.4)Managememoryusagewithefficientdata

Inpython,ListSusedynamicMemoryAllocationWithOver-Asalose,而alenumpyArraySallaySallocateFixedMemory.1)listssallocatemoremoremoremorythanneededinentientary上,respizeTized.2)numpyarsallaysallaysallocateAllocateAllocateAlcocateExactMemoryForements,OfferingPrediCtableSageButlessemageButlesseflextlessibility。

Inpython,YouCansspecthedatatAtatatPeyFelemereModeRernSpant.1)Usenpynernrump.1)Usenpynyp.dloatp.dloatp.ploatm64,formor professisconsiscontrolatatypes。

NumPyisessentialfornumericalcomputinginPythonduetoitsspeed,memoryefficiency,andcomprehensivemathematicalfunctions.1)It'sfastbecauseitperformsoperationsinC.2)NumPyarraysaremorememory-efficientthanPythonlists.3)Itoffersawiderangeofmathematicaloperation

Contiguousmemoryallocationiscrucialforarraysbecauseitallowsforefficientandfastelementaccess.1)Itenablesconstanttimeaccess,O(1),duetodirectaddresscalculation.2)Itimprovescacheefficiencybyallowingmultipleelementfetchespercacheline.3)Itsimplifiesmemorym

SlicingaPythonlistisdoneusingthesyntaxlist[start:stop:step].Here'showitworks:1)Startistheindexofthefirstelementtoinclude.2)Stopistheindexofthefirstelementtoexclude.3)Stepistheincrementbetweenelements.It'susefulforextractingportionsoflistsandcanuseneg

numpyallowsforvariousoperationsonArrays:1)basicarithmeticlikeaddition,減法,乘法和division; 2)evationAperationssuchasmatrixmultiplication; 3)element-wiseOperations wiseOperationswithOutexpliitloops; 4)

Arresinpython,尤其是Throughnumpyandpandas,weessentialFordataAnalysis,offeringSpeedAndeffied.1)NumpyArseNable efflaysenable efficefliceHandlingAtaSetSetSetSetSetSetSetSetSetSetSetsetSetSetSetSetsopplexoperationslikemovingaverages.2)


熱AI工具

Undresser.AI Undress
人工智慧驅動的應用程序,用於創建逼真的裸體照片

AI Clothes Remover
用於從照片中去除衣服的線上人工智慧工具。

Undress AI Tool
免費脫衣圖片

Clothoff.io
AI脫衣器

Video Face Swap
使用我們完全免費的人工智慧換臉工具,輕鬆在任何影片中換臉!

熱門文章

熱工具

SAP NetWeaver Server Adapter for Eclipse
將Eclipse與SAP NetWeaver應用伺服器整合。

Atom編輯器mac版下載
最受歡迎的的開源編輯器

Dreamweaver Mac版
視覺化網頁開發工具

記事本++7.3.1
好用且免費的程式碼編輯器

ZendStudio 13.5.1 Mac
強大的PHP整合開發環境