Home > Article > Backend Development > Usage of asyncore asynchronous module in Python and implementation of httpclient
asyncore is an asynchronous socket package. In particular, the dispatcher class contains many socket operation methods for asynchronous calls, which is very sharp. Let's explain the usage of the asyncore asynchronous module in Python and the implementation of httpclient.
Basics
This module is an asynchronous implementation of socket. Let us first familiarize ourselves with some classes and methods in the module:
1.asyncore.loop
Enter a polling loop until the pass count or the open channel has been closed.
2.asyncore.dispatcher
The dispatcher class is a wrapper object of the underlying socket class. To make it more useful, it has some event handling methods that are called asynchronously in a loop. Otherwise it is a standard non-blocking socket object.
The low-level events tell the asynchronous loop that certain high-level events have occurred at specific events or specific connection states. For example, we ask a socket to connect to another host.
(1) handle_connect() first read or write event.
(2) handle_close() No data is available for the read event.
(3) handle_accept reads the event and monitors a socket.
(4)handle_read
Called when the asynchronous loop detects that the channel calls read().
(5) handle_write
is called when the asynchronous loop detects that a writable socket can be written. This approach often achieves buffering performance. For example,
def handle_write(self): sent = self.send(self.buffer) self.buffer = self.buffer[sent:]
(6) handle_expt
When there is an (OOB) data socket connection. This almost never happens, since OOB is poorly supported and rarely used.
(7)handle_connect
Called when the socket creates a connection.
(8)handle_close
Called when the socket connection is closed.
(9) handle_error
Called when an exception is raised and there is no other processing.
(10)handle_accept
Called when the local listening channel establishes a connection with the remote end (passive connection).
(11)readable
Called every time in the asynchronous loop to determine whether to add a channel socket to the read event list, the default is True.
(12)writable
Called every time in the asynchronous loop to determine whether to add a channel socket to the write event list, the default is True.
(13) create_socket
is the same as when creating a standard socket.
(14)connect
is the same as the port setting of standard socket. It accepts a tuple whose first parameter is the host address and the second parameter is the port number.
(15) send
Send data to the remote socket.
(16)recv
Read up to buffer_size data from the remote socket. An empty string means the channel is closed from the other end.
(17)listen
Listen to the socket connection.
(18)bind
Bind the socket to the address.
(19)accept
To accept a connection, it must be bound to a socket and listening address.
(20)close
Close the socket.
3.asyncore.dispatcher_with_send
The dispatcher subclass adds a simple buffered output function for simple clients, and more complex ones use asynchat.async_chat.
4.asyncore.file_dispatcher
file_dispatcher takes a file descriptor or file object map and an optional argument, wrapper, using survey() or loop() function. If a file object or any fileno() method is provided, that method will be called and passed to the file_wrapper constructor. Availability: UNIX.
5.asyncore.file_wrapper
file_wrapper takes an integer file descriptor and calls os.dup() to copy the handler, so that the original handler may be closed independently of file_wrapper . This class implements enough methods to simulate a socket using the file_dispatcher class. Availability: UNIX.
asyncore Example
1. Implementation of an http client.
import socket import asyncore class Client(asyncore.dispatcher): def __init__(self, host, path): asyncore.dispatcher.__init__(self) self.create_socket(socket.AF_INET, socket.SOCK_STREAM) self.connect((host, 80)) self.buffer = 'GET %s HTTP/1.0\r\n\r\n' % path def handle_connect(self): pass def handle_close(self): self.close() def handle_read(self): print self.recv(8192) def writable(self): return (len(self.buffer) > 0) def handle_write(self): sent = self.send(self.buffer) self.buffer = self.buffer[sent:] client = Client('www.python.org', '/') asyncore.loop()
Server accepts connections and assigns tasks
import socket import asyncore class EchoHandler(asyncore.dispatcher_with_send): def handle_read(self): data = self.recv(8192) if data: self.send(data) class EchoServer(asyncore.dispatcher): def __init__(self, host, port): asyncore.dispatcher.__init__(self) self.create_socket(socket.AF_INET, socket.SOCK_STREAM) self.set_reuse_add() self.bind((host, port)) self.listen(5) def handle_accept(self): pair = self.accept() if pair is not None: sock, addr = pair print 'Incoming connection from %s' % repr(addr) handler = EchoHandler(sock) server = EchoServer('localhost', 8080) asyncore.loop()
2. Use asyncore’s port mapping (port forwarding)
import socket,asyncore class forwarder(asyncore.dispatcher): def __init__(self, ip, port, remoteip,remoteport,backlog=5): asyncore.dispatcher.__init__(self) self.remoteip=remoteip self.remoteport=remoteport self.create_socket(socket.AF_INET,socket.SOCK_STREAM) self.set_reuse_addr() self.bind((ip,port)) self.listen(backlog) def handle_accept(self): conn, addr = self.accept() # print '--- Connect --- ' sender(receiver(conn),self.remoteip,self.remoteport) class receiver(asyncore.dispatcher): def __init__(self,conn): asyncore.dispatcher.__init__(self,conn) self.from_remote_buffer='' self.to_remote_buffer='' self.sender=None def handle_connect(self): pass def handle_read(self): read = self.recv(4096) # print '%04i -->'%len(read) self.from_remote_buffer += read def writable(self): return (len(self.to_remote_buffer) > 0) def handle_write(self): sent = self.send(self.to_remote_buffer) # print '%04i <--'%sent self.to_remote_buffer = self.to_remote_buffer[sent:] def handle_close(self): self.close() if self.sender: self.sender.close() class sender(asyncore.dispatcher): def __init__(self, receiver, remoteaddr,remoteport): asyncore.dispatcher.__init__(self) self.receiver=receiver receiver.sender=self self.create_socket(socket.AF_INET, socket.SOCK_STREAM) self.connect((remoteaddr, remoteport)) def handle_connect(self): pass def handle_read(self): read = self.recv(4096) # print '<-- %04i'%len(read) self.receiver.to_remote_buffer += read def writable(self): return (len(self.receiver.from_remote_buffer) > 0) def handle_write(self): sent = self.send(self.receiver.from_remote_buffer) # print '--> %04i'%sent self.receiver.from_remote_buffer = self.receiver.from_remote_buffer[sent:] def handle_close(self): self.close() self.receiver.close() if __name__=='__main__': import optparse parser = optparse.OptionParser() parser.add_option( '-l','--local-ip', dest='local_ip',default='127.0.0.1', help='Local IP address to bind to') parser.add_option( '-p','--local-port', type='int',dest='local_port',default=80, help='Local port to bind to') parser.add_option( '-r','--remote-ip',dest='remote_ip', help='Local IP address to bind to') parser.add_option( '-P','--remote-port', type='int',dest='remote_port',default=80, help='Remote port to bind to') options, args = parser.parse_args() forwarder(options.local_ip,options.local_port,options.remote_ip,options.remote_port) asyncore.loop()
More articles related to the usage of asyncore asynchronous module in Python and the implementation of httpclient Please pay attention to PHP Chinese website!