Let's talk about redis file events and time events

This article brings you relevant knowledge about Redis, which mainly introduces the issues related to file events and time events. File events are the abstraction of socket operations by the server, and time events It is the abstraction of this kind of timing operation by the server. I hope it will be helpful to everyone.

Recommended learning: Redis tutorial

Redis was single-threaded before 6.0. After 6.0, you can enable multiple threads through the configuration file. Threads, multi-threading after 6.0 refers to the use of multi-threads for execution in io to speed up I/O.

The Redis server is an event-driven program. The server needs to handle the following two types of events:

• File event (file event)
  The Redis server communicates with the client through a socket ( Or other Redis servers) to connect, and file events are the abstraction of socket operations by the server. The communication between the server and the client (or other server) will generate corresponding file events, and the server will complete a series of network communication operations by listening and processing these events. The file event handler uses the I/O multiplexing (multiplexing) program to listen to multiple sockets at the same time and associate different event handlers with the sockets based on the tasks currently performed by the sockets. device. When the monitored socket is ready to perform operations such as connection response (accept), read (read), write (write), close (close), etc., the file event corresponding to the operation will be generated. At this time, the file The event handler will call the event handler previously associated with the socket to handle these events.
• Time event (time event)
  Some operations in the Redis server (such as the serverCron function) need to be executed at a given point in time, and the time event is the server's abstraction of such timing operations. Time events are divided into two categories: the first type is timed events (allowing a program to be executed once after a specified time) and the other type is periodic events (allowing a program to be executed once every specified time).

1. File events

1. File event processor

The file event processor is composed of four parts: socket, I/O Multiplexer, file event dispatcher, event handler.

The I/O multiplexer is responsible for listening to multiple sockets and transmitting those sockets that generated events to the file event dispatcher. Although multiple file events may occur concurrently, a single I/O multiplexer will always put all generated event sockets into a queue, and then pass through this queue in an orderly and synchronous manner. The delivery socket is dispatched to the file event one socket at a time. When the event generated by the previous socket is processed (the event processing associated with the socket is completed), the I/O multiplexer will continue to transmit the next socket to the file event dispatcher. The file event dispatcher receives the socket from the I/O multiplexer and uses the corresponding event handler according to the type of event generated by the socket. The server will execute different socket associations for different characters. Event handlers, these handlers define the actions that the server should perform when an event occurs.

2. I/O multiplexing

All functions of Redis’ multiplexing program are packaged by I/O multiplexing select, epoll, evport, and kqueue. Function library to implement

3. File event type

• AE_READABLE event
  When the socket becomes readable (the client performs a write or close operation ) or when a new socket that can respond appears, the socket will generate an AE_READABLE event.

• AE_WAITABLE event
  When the socket becomes writable (the client performs a read operation), the AE_WAITABLE event will be generated
  The I/O multiplexer will simultaneously Listen to the AE_READABLE event and the AE_WAITABLE event. If a socket generates both events at the same time, the event dispatcher will prioritize the AE_READABLE event, which means that the server will read the socket first and then write the socket.

4. File event processor

• Connection response processor
  Connection response processor, this processor is used to connect clients to the server's listening socket The client responds. When the Redis server is initialized, the program will associate this connection response processor with the AE_READABLE event of the server listening socket. When the client connects to the server listening socket, the socket will be generated. The AE_READABLE event triggers the execution of the connection response processor and performs the corresponding socket response operation.
• Command request processor
  When a client successfully connects to the server through the connection response processor, the server will associate the AE_READABLE event of the socket with the command request processor. When the client requests When the server sends a command request, the socket will generate an AE_READABLE event, causing the command request processor to execute and perform operations such as socket reading. During the entire process of the client connecting to the server, the server will always associate the command request handler for the AE_READABEL event of the client socket.
• Command reply processor
  When the server has a command reply that needs to be sent to the client, the server will associate the AE_WRITABLE event of the client socket with the command reply processor. When the client is ready to receive When the server returns a command reply, an AE_WAITABEL event will be generated, causing the command reply processor to execute and perform the corresponding socket write operation. When the command reply is sent, the server will disassociate the command reply handler from the AE_WAITABLE event of the client socket.

5. One-time completed client-server connection event example

First the Redis client initiates a connection to the server, then the listening socket will generate an AE_READABEL event, triggering the connection response processing The processor executes, and the processor responds to the client's connection request, then creates the client socket and client status, and associates the AE_READABEL event of the client socket with the command request processor so that the client can The master server sends a command request.

Later, assuming that the client sends a command request to the main server, the client socket will generate an AE_READABEL event, triggering the execution of the command request processor. The processor reads the client's command and then passes it to the associated program to execute.

Executing the command will generate a corresponding command reply. In order to transmit this line of command reply back to the client, the server will associate the AE_WAITABLE event with the command reply processor. When the client attempts to read the command reply, the client will generate an AE_WAITABLE event, triggering the execution of the command reply processor. When the command reply processor writes the command reply to the socket in the entire server, the server will release the client socket. The AE_WAITABLE event is associated with the command reply handler execution.

2. Time events

1. Composition of time events

• id
  The server creates a globally unique ID for time events. The ID numbers increase in order from small to large.
• when
  A UNIX timestamp with millisecond precision that records the arrival time of a time event.
• timeProc
  Time event processor, a function. When a time event arrives, the server calls the corresponding processor to handle the event.

Whether a time event is a timed event or a periodic event depends on the return value of the time event processor. If the event processor returns ae.h/AE_NOMORE, then the event is a timed event. After arriving once, it will be deleted and will not be reached again. If the event handler returns an integer value other than AE_NOMORE, then the event is a periodic event. When a time event is reached, the server will update the when attribute of the event based on the return value of the event handler, so that the event will continue for a period of time. It arrives again after a certain amount of time and is updated and run in this way.

Implementation

The server puts all time events in an unordered linked list (The unordered does not refer to the id field, but the when field, so it must be traversed every time it is executed Complete the linked list. ), whenever the time event executor runs, it will traverse the entire linked list, find all the events that have arrived, and call the corresponding event handler.
What needs to be explained here is that although it is an unordered linked list, the length of the linked list is not very long. In normal mode, the Redis server only uses serverCron as a time event, so this opportunity degenerates into the role of pointers. In benchmark mode, the server Only two time events are used, so the impact of full traversal on performance is negligible.

serverCron function

The continuously running Redis server needs to regularly check and adjust its own resources and status to ensure that the server can run long-term and stably. These regular operations are performed by redis.c/ The serverCron function is responsible for execution. Its main tasks include:

• Update various statistical information of the server, such as time, memory usage, database usage, etc.
• Clear expired key-value pairs in the database.
• Close and clean up clients with failed connections.
• Try to perform AOF or RDB persistence operation.
• If the server is the master server, then perform regular synchronization on the slave server
• If it is in cluster mode, perform regular synchronization and connection testing on the cluster.

Event Scheduling and Execution

Because there are two event types, file events and time events, in the server, the server must schedule these two events and decide when they should Processing file events, when should time events be processed, how much time should be spent processing them, etc.
The pseudo code of the processing process is as follows:

def aeProcessEvents():
	# 获取到达时间离当前最近的时间事件
	tem_event = aeSearchNearestTimer()
	
	# 计算上一步获得到的事件 距离到达还有多少秒
	remaind_ms = time_event.when - unix_ts_now()
	
	# 如果事件已经到达， 那么remaind_ms的值可能为负数，设置为0
	remaind_ms = max(remaind_ms, 0)
	
	# 阻塞并等待文件事件产生，最大阻塞时间由timeval结构决定，
	# 如果remaind_ms的值为0，那么aeAPiPoll调用之后马上返回，不阻塞
	aeApiPoll(timeval)
	# 处理所有已经产生的文件事件
	processFileEvents()
	# 处理所有已经到达的时间事件
	proccessTimeEvents()

Event scheduling and execution rules:
1) The maximum blocking time of the aeApiPoll function is determined by the time event whose arrival time is closest to the current time. This method is both This can avoid the server from frequently polling (busy waiting) for time events, and also ensure that the aeApiPoll function will not block for too long.

2) Because file events appear randomly, if no time event arrives after waiting and processing a file event, the server will wait and process the file event again. As the file event continues to execute, the time will gradually approach the arrival time set by the time event, and finally reach the arrival time. At this time, the server can start processing the arrival time event.

3) The processing of file events and time events are executed synchronously, orderly, and atomically. The server will not interrupt event processing midway, nor will it preempt events. Therefore, no matter it is a file event The processor or the time event processor will reduce the blocking time of the program as much as possible and actively give up execution rights when necessary, thereby reducing the possibility of event starvation. For example, when the command reply processor writes a command reply to the client socket, if the number of written bytes exceeds a preset constant, the command reply processor will actively use break to break out of the writing loop. Leave the remaining data to be written next time; in addition, time events will also put very time-consuming persistence operations into sub-threads or sub-processes for execution.

4) Because the time event is executed after the file event, and there will be no preemption between events, the actual processing time of the time event is usually slightly later than the arrival time set by the time event.

There is a cooperative relationship between file events and time events. The server will handle these two events in turn, and there will be no preemption during the event processing. The actual processing time of time events is usually later than the set arrival time.

Recommended learning: Redis learning tutorial

The above is the detailed content of Let's talk about redis file events and time events. For more information, please follow other related articles on the PHP Chinese website!