Introduction to epoll about python network programming learning IO multiplexing

What is epoll

What is epoll? In network programming in linux, select has been used for event triggering for a long time. In the new Linux kernel, there is a mechanism to replace it, which is epoll. Of course, this is not unique to the 2.6 kernel, it was introduced in the 2.5.44 kernel (epoll(4) is a new API introduced in Linux kernel 2.5.44) , it has almost all the advantages mentioned before, and is recognized as the best-performing multiplexed I/O ready notification method under Linux2.6.

Compared with select, the biggest advantage of epoll is that it will not reduce efficiency as the number of listening fds increases. Because in the select implementation in the kernel, it is processed by polling. The more fds polled, the more time it will take.

epoll working principle

epoll also only notifies those ready file descriptors, and when we call epoll_wait() to obtain a ready file descriptor, what is returned is not the actual descriptor, but A value representing the number of ready descriptors. You only need to obtain the corresponding number of file descriptors in sequence from an array specified by epoll. Memory mapping (mmap) technology is also used here, so that it is completely This eliminates the overhead of copying these file descriptors during system calls.

Another essential improvement is that epoll adopts an event-based readiness notification method. In select/poll, the kernel scans all monitored file descriptors only after the process calls a certain method, while epoll registers a file descriptor through epoll_ctl() in advance. Once a certain file descriptor is ready, , the kernel will use a callback-like mechanism to quickly activate this file descriptor, and will be notified when the process calls epoll_wait().

As can be seen from the above, epoll is an improvement on the select and poll models, which improves the performance of network programming and is widely used in large-scale concurrent requests. In C/S architecture.

epoll in python1. Triggering method:

Edge trigger/horizontal trigger, only applicable to Unix/Linux operating system

2. Schematic diagram

3. General steps

Create an epoll

object

——Create 1 epollObjectTell the epoll object to monitor specific events on specific sockets——Tell the epoll object to monitor specific events on the specified socket

Ask the epoll object which sockets may have had the specified event since the last query——Ask the epoll object, which sockets have had the specified event since the last query

Perform some

action

on those sockets—— Perform some operations on these socketsTell the epoll object to modify the

list

of sockets and/or events to monitor——Tell the epoll object to modify the socket list and/or events, and MonitorRepeat steps 3 through 5 until finished——Repeat steps 3-5 until finished

Destroy the epoll object——Destroy the epoll object

4, related usage

import select Import select module

epoll = select.epoll() creates an epoll object

epoll.register(file handle, event type) registers the file handle to be monitored and Event

Event type:

select.EPOLLIN Readable event

select.EPOLLOUT Writable event

select.EPOLLERR Error event

Select.EPOLLHUP Client disconnect event

epoll.unregister(file handle) Destroy file handle

epoll.poll(timeout) When the file handle changes, it will be automatically displayed in the form of a list Report to the user process, timeout

is the timeout, the default -1, that is, wait until the file handle changes. If there is no change, it will return empty

epoll.fileno() Returns the epoll control file descriptor (

Return

the epoll control file descriptor)

epoll. modfiy(fineno,event)fineno is the file descriptor and event is the event type. Its function is to modify the event corresponding to the file descriptor

epoll.fromfd(fileno) creates an epoll object from a specified file descriptor

epoll.close() Closes the control file descriptor of the epoll object

5 Example: The client sends data and the server returns the received data to the client

Server code

#!/usr/bin/env python
#-*- coding:utf-8 -*-
import socket
import select
import Queue
#创建socket对象
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#设置IP地址复用
serversocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
#ip地址和端口号
server_address = ("127.0.0.1", 8888)
#绑定IP地址
serversocket.bind(server_address)
#监听，并设置最大连接数
serversocket.listen(10)
print  "服务器启动成功，监听IP：" , server_address
#服务端设置非阻塞
serversocket.setblocking(False)  
#超时时间
timeout = 10
#创建epoll事件对象，后续要监控的事件添加到其中
epoll = select.epoll()
#注册服务器监听fd到等待读事件集合
epoll.register(serversocket.fileno(), select.EPOLLIN)
#保存连接客户端消息的字典，格式为{}
message_queues = {}
#文件句柄到所对应对象的字典，格式为{句柄：对象}
fd_to_socket = {serversocket.fileno():serversocket,}
while True:
  print "等待活动连接......"
  #轮询注册的事件集合，返回值为[(文件句柄，对应的事件)，(...),....]
  events = epoll.poll(timeout)
  if not events:
     print "epoll超时无活动连接，重新轮询......"
     continue
  print "有" , len(events), "个新事件，开始处理......"
  
  for fd, event in events:
     socket = fd_to_socket[fd]
     #如果活动socket为当前服务器socket，表示有新连接
     if socket == serversocket:
            connection, address = serversocket.accept()
            print "新连接：" , address
            #新连接socket设置为非阻塞
            connection.setblocking(False)
            #注册新连接fd到待读事件集合
            epoll.register(connection.fileno(), select.EPOLLIN)
            #把新连接的文件句柄以及对象保存到字典
            fd_to_socket[connection.fileno()] = connection
            #以新连接的对象为键值，值存储在队列中，保存每个连接的信息
            message_queues[connection]  = Queue.Queue()
     #关闭事件
     elif event & select.EPOLLHUP:
        print 'client close'
        #在epoll中注销客户端的文件句柄
        epoll.unregister(fd)
        #关闭客户端的文件句柄
        fd_to_socket[fd].close()
        #在字典中删除与已关闭客户端相关的信息
        del fd_to_socket[fd]
     #可读事件
     elif event & select.EPOLLIN:
        #接收数据
        data = socket.recv(1024)
        if data:
           print "收到数据：" , data , "客户端：" , socket.getpeername()
           #将数据放入对应客户端的字典
           message_queues[socket].put(data)
           #修改读取到消息的连接到等待写事件集合(即对应客户端收到消息后，再将其fd修改并加入写事件集合)
           epoll.modify(fd, select.EPOLLOUT)
     #可写事件
     elif event & select.EPOLLOUT:
        try:
           #从字典中获取对应客户端的信息
           msg = message_queues[socket].get_nowait()
        except Queue.Empty:
           print socket.getpeername() , " queue empty"
           #修改文件句柄为读事件
           epoll.modify(fd, select.EPOLLIN)
        else :
           print "发送数据：" , data , "客户端：" , socket.getpeername()
           #发送数据
           socket.send(msg)
#在epoll中注销服务端文件句柄
epoll.unregister(serversocket.fileno())
#关闭epoll
epoll.close()
#关闭服务器socket
serversocket.close()

Client code:

#!/usr/bin/env python
#-*- coding:utf-8 -*-
import socket
#创建客户端socket对象
clientsocket = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
#服务端IP地址和端口号元组
server_address = ('127.0.0.1',8888)
#客户端连接指定的IP地址和端口号
clientsocket.connect(server_address)
while True:
    #输入数据
    data = raw_input('please input:')
    #客户端发送数据
    clientsocket.sendall(data)
    #客户端接收数据
    server_data = clientsocket.recv(1024)
    print '客户端收到的数据：'server_data
    #关闭客户端socket
    clientsocket.close()

The above is the detailed content of Introduction to epoll about python network programming learning IO multiplexing. For more information, please follow other related articles on the PHP Chinese website!