Assembly-based implementation of C/C++ coroutines (for servers)

This article is the implementation of C/C coroutine. We need to achieve these two goals:

1. Have a sequential idea of ​​​​synchronous server programming to facilitate functional design and code debugging - I used the coroutine part in libco

2. has the performance of asynchronous I/O - I used event I/O in libevent apache php mysql

Structurally, it is The functions of libco and libevent are combined, so I named my project libcoevent, which means "synchronous coroutine server programming framework based on libevent". The word co in the name does not mean libco, but coroutine.

As for the programming language, I chose C, mainly because libco only supports Linux based on x86 or x64 architecture, and such architectures are basically PCs, or have sufficient resources and high performance. It's a good embedded system, and there is no problem in learning C. This article explains how the code is implemented.

If you want to use this project, please add the three options -lco -levent -lcoevent to the link options.

Class relationship and basic functions

Class relationship

Class inheritance relationship

The basic inheritance relationship diagram of the class is as follows:

In actual calls, only the classes on the leaf nodes of the inheritance relationship tree will be actually used, and other classes are regarded as virtual classes.

Class affiliation

Instances of various types have affiliations during program running. Except for the top-level Base class, other leaf classes need to be attached to other classes. It can be executed in the operating environment. The dependency diagram is as follows:

• Base class provides the most basic operating environment and manages Server Object;

• Procedure Object ManagementClient Object. In the figure, both the Server and Session objects manage the Client object. The

• Server object is created by the application and initialized to run in the Base object. You can configure the Server object to be automatically destroyed when the server terminates or when its dependent Base object is destroyed.

• Session The object is automatically created by the Server object in session mode, and is run by calling the program entry specified by the application; When the session ends (function call return) or when its subordinate Server object service ends, the Server object is automatically destroyed.

• Client object is created by the application calling the interface of the Procedure object and is used to interact with third-party services. The application can call the interface in advance to request the destruction of the Client object, or it can automatically destroy it when the Procedure service ends.

Base and Event classes

##Base class is used to run various services of libcoevent. Each instance of the Base class should correspond to a thread, and all services run in the Base instance in a coroutine manner. As can be seen from the above figure, the Base class contains a event_base object of the libevent library and a series of Event objects of this coroutine library. The

Event class actually borrows the struct event name from libevent, because every Event class An instance of , corresponding to an event object of libevent. The focus we need to focus on is the Procedure and Client classes.

Procedure class

Procedure class has two key features:

1. Each object has a libco coroutine, That is, it has its own independent context information and can be used to write an independent server process (procedure);

2. Proceure subclasses can create

   Client objects and third parties Server communication and interaction. The

Procedure class has two subclasses, namely Server and Session.

Server Class

The Server class is created by the application and initialized to run in a

Base object. The Server class has three subclasses:

• SubRoutine: It does not actually serve as any server program, but provides the most basic sleep() function and supports the creation of Client objects of the Procedure class. function, so the application can be used as a temporarily created or resident internal program.

• UDPServer: After the application creates and initializes the UDPServer object, the program will automatically bind to a datagram socket interface. Applications can implement network services by sending and receiving data packets in the network interface. UDPServer provides both normal mode and session mode.

• TCPServer: After the application creates and initializes the TCPPServer object, the program will automatically bind and listen to the stream socket. TCPServer only supports session mode.

The so-called "Normal mode" is the behavior in which the application registers the entry function of the Server object and the application operates the Server object.

The so-called "Session mode" refers to the UDPServer or TCPServer object. After receiving incoming data, it automatically distinguishes the client and creates a separate Session Object is processed. Each Session object only serves one client.

Session class

Session objects cannot be actively created by the application, but are automatically created on demand by the Server class in session mode. The characteristic of the Session object is that it can only communicate with a single client (compared to the UDPServer object), so there is no send() function, only reply().

The Session class and its subclasses declared in the header file coevent.h are pure virtual classes in order to prevent the application from explicitly constructing Session object and hide implementation details.

Client Class

Client Objects are created by Procedure objects and recycled by Procedure objects. The role of the Client object is to actively initiate communication with the remote server. Since this action belongs to the client from the perspective of the client-service structure, it is named Client.

DNSClient

One of the more special subclasses of Client is the DNSClient class. This class exists to solve the getaddrinfo( ) Blocking problem. For the implementation principle of DNSClient, please refer to the code and my previous article "DNS Message Structure and Personal DNS Parsing Code Implementation".

As for the DNSClient class, the specific implementation principle is to encapsulate a UDPClient object, use this object to complete the sending and receiving of DNS messages, and implement the parsing of the messages in the class.

UDPServer——Coroutine implementation based on libevent

UDPServer The principle of the common mode is a very typical synchronous coroutine server framework based on libevent. In its code implementation, the core functions are the following functions:

• _libco_routine(), the entry function of the coroutine. Using this function, it is transformed into the unity of liboevent Service entry function

• _libevent_callback(), libevent time callback function, in this function, the recovery of the coroutine context is realized.

• UDPServer::recv_in_timeval(), data receiving function, in this function, the key data waiting function is implemented, and the saving of the coroutine context is also realized

The total amount of code for the above three functions, including blank lines, does not exceed 200 lines. I believe it is still easy to understand. The following explains the implementation principle in detail:

libco coroutine interface

As mentioned before, I use libco as the coroutine library. Coroutines are transparent to the application, but to the library implementation, this is core.

The following explains several interfaces provided by libco's coroutine function (libco's number of documents is simply "touching", which is often complained about on the Internet...):

Creation and destruction Coroutine

Libco Use the structure struct stCoRoutine_t * to save the coroutine, and you can create the coroutine object by calling co_create(); use co_release() Destroy coroutine resources.

Enter the coroutine

After creating the coroutine, call co_resume() to start executing the coroutine from the beginning of the coroutine function.

Pause the coroutine

When the coroutine needs to hand over the CPU usage rights, you can call co_yield() to release the coroutine and switch the context. After the call, the context will be restored to the last coroutine that called co_resume(). The location where co_yield() is called can be regarded as a "breakpoint".

Resume the coroutine

The functions used to restore the coroutine and create the coroutine are co_resume(). Call this function to switch the current stack to the specified coroutine. Context, the coroutine will resume execution from the "breakpoint" mentioned above.

Coroutine Scheduling Implementation

As you can see from the previous section, the libco coroutine function we use includes the coroutine switching function, but when to switch and what happens to the CPU after the switch Distribution, this is what we need to implement and encapsulate.

The time to create and destroy coroutines is naturally when the UDPServer class is initialized and destructed. The following focuses on analyzing the operations of entering, suspending and resuming the coroutine:

Enter the coroutine

The code for entering/resuming the coroutine is in _libevent_callback(), there are Such a line:

// handle control to user application
co_resume(arg->coroutine);

If the current coroutine has not been executed, then after executing this code, the program will switch to the coroutine function specified when creating the libco coroutine and start execution. For UDPServer, that is the _libco_routine() function. This function is very simple, with only three lines:

static void *_libco_routine(void *libco_arg)
{
    struct _EventArg *arg = (struct _EventArg *)libco_arg;
    (arg->worker_func)(arg->fd, arg->event, arg->user_arg);
    return NULL;
}

By passing in parameters, the libco callback function is converted into a server function specified by the application for execution.

But how to implement the first libevent callback? This is still very simple, just set the timeout to 0 when calling libevent's event_add(), which will cause the libevent event to time out immediately. Through this mechanism, we also achieve the purpose of executing each Procedure service function immediately after Base is run.

Pause and resume coroutines

When to call co_yield is the focus of this coroutine implementation. The location of calling co_yield is a possibility The place where context switching occurs is also the key technical point in converting an asynchronous programming framework into a synchronous framework. You can refer to the recv_in_timeval() function of UDPServer here. The basic logic of the function is as follows:

The most important branch is the judgment of the libevent event flag; and the most important logic is event_add() and co_yield() function calls. The function fragment is as follows:

struct timeval timeout_copy;
timeout_copy.tv_sec = timeout.tv_sec;
timeout_copy.tv_usec = timeout.tv_usec;
    ...
event_add(_event, &timeout_copy);
co_yield(arg->coroutine);

Here, we understand the co_yield() function as a breakpoint. When the program is executed here, the right to use the CPU will be handed over and the program will return To the upper-level function that calls co_resume(). Where exactly is this "upper-level function"? In fact, it is the _libevent_callback() function mentioned earlier.

From the perspective of _libevent_callback(), the program will return from the co_resume() function and continue execution. At this point we can understand this: the scheduling of coroutines is actually borrowed from libevent. Here we have to pay attention to the few sentences above co_resume():

// switch into the coroutine
if (arg->libevent_what_ptr) {
    *(arg->libevent_what_ptr) = (uint32_t)what;
}

Here the libevent event flag value is passed to the coroutine, and this is the previous event. important basis for judgment. When the time comes, _libevent_callback() will call co_resume() below, handing the CPU usage rights back to the coroutine.

Destroy the coroutine

In addition to ci_yield(), the coroutine function call return will also result in co_resume() Return, so in _libevent_callback(), we also need to determine whether the coroutine has ended. If the coroutine ends, then the related coroutine resources should be destroyed. See if (is_coroutine_end(arg->coroutine)) {...} The code in the conditional body.

Session Mode

In the implementation of this project, a server design pattern called "Session Mode" is provided. Session mode refers to the UDPServer or TCPServer object. After receiving incoming data, it automatically distinguishes the client and creates a separate Session object for processing. Each Session object only serves one client.

For TCPServer, it is relatively simple to implement the above function, because after monitoring a TCP socket, when there is an incoming connection, just call accept() , you can get a new file descriptor and create a new subclass of Server for this file descriptor - this is the TCPSession class.

But UDPServer is more troublesome, because UDP cannot do this. We can only implement the so-called session by ourselves.

UDPSession achieves

design goals

We need to achieve the following effects of the UDPSession class:

• Class When calling the recv function, only the data sent by the corresponding remote client will be received. The

• class calls the send function (the actual implementation is reply()), you can use the port of UDPServer to reply

recv()

In the project, UDPSession is an abstract class, and the actual implementation is UDPItnlSession. But to be precise, the implementation of UDPItnlSession is closely dependent on UDPServer. For this part, you can refer to the do-while() loop body code in the _session_mode_worker() function of UDPServer. The program idea is as follows:

• UDPServer Maintain a UDPSession dictionary, with a combination of remote IP port names as the key.

• When the data arrives, determine whether the remote IP port combination is in the dictionary. If it is, copy the data to the corresponding session; if it does not exist, create the session

For the code to copy data, see the forward_incoming_data() function implementation of the UDPItnlSession class.

reply()

Sending data is actually very simple, just perform sendto() directly on the fd of UDPServer.

quit

For the Server object in session mode, the code provides a function that can be called by its session and requires the server to exit and destroy resources: quit_session_mode_server(). The implementation principle is to trigger an EV_SIGNAL event to the server. For ordinary I/O events, this should not occur, and we use it here as an exit signal. If the server detects this signal, the exit logic is triggered.

Application Example

The sample code of this project is divided into two parts: server and client. The server uses libcoevent, while the client only uses Python A simple program written. This article will not explain the client part of the code.

Server’s code provides application examples for three subclasses of the Server class. Using logic including blank lines, debugging statements, error judgments, etc., one process and two services were implemented in less than 300 lines. It should be said that the logic is still very clear and a lot of code is saved.

SubRoutine

shows a one-time linear network logic through the function _simple_test_routine(). In the program, the routine first creates a DNSClient object, requests a domain name from the default domain name server, and then connect() the 80 port of the server. After success, return directly.

This function shows the usage scenario of SubRoutine and the usage of Client object, especially the simple usage of DNSClient. The entry function of

UDPServer

UDPServer is _udp_session_routine(), which functions to provide domain name query services for clients. Clients send a string as the domain name to be queried, and then the server returns the query results to the client after requesting it through the DNSClient object.

This function shows the (more complex and complete) usage of UDPSession objects and DNSClient.

TCPServer

The entry function is _tcp_session_routine(), the logic is relatively simple, mainly to show the usage of TCPSession.

Postscript

In principle, libcoevent has been developed and has achieved the necessary functions, and can be used to write server programs. Of course, since this is the first version, a lot of the code still looks a bit messy. The significance of this library is that it can carefully explain the more original implementation principles of C/C coroutines from a teaching perspective, and it can also be used as a usable coroutine server library.

We welcome readers to criticize this library, and we also welcome readers to put forward new requirements-for example, I decided to add a few requirements, which can be regarded as TODO:

1. implementationHTTPServer, as a subclass of TCPServer, provides HTTP fcgi service;

2. implements the SSLClient class to handle external SSL ask.

Related articles:

C# Network Programming Series Articles (8) UdpClient implements synchronized UDP server

C language to implement php server

Related videos:

C# Tutorial

The above is the detailed content of Assembly-based implementation of C/C++ coroutines (for servers). For more information, please follow other related articles on the PHP Chinese website!