Teach you how libco supports huge amounts of data information

ibco is a c/c++ coroutine library used on a large scale in the WeChat backend. It has been running stably on tens of thousands of machines in the WeChat backend since 2013. Libco was open sourced for the first time in 2013 as one of Tencent's six major open source projects. We have recently made a major update, which is synchronized on github.com/tencent/libco. libco supports the backend agile synchronization style programming model while providing high concurrency capabilities of the system.

Features supported by libco

No need to invade business logic, multi-process and multi-thread services are transformed into coroutine services, and concurrency capabilities are improved a hundred times;

Support CGIFramework, easy to build web services (New);

Supports commonly used third libraries such as gethostbyname, mysqlclient, ssl (New);

Optional shared stack mode, easy to stand-alone Access tens of millions of connections (New);

Improve and concise coroutine programmingInterface

　-pthread-like interface design, through simple and clear interfaces such as co_create and co_resume Complete the creation and restoration of the coroutine; – Thread-like coroutine private variables , coroutine semaphore co_signal (New) for inter-coroutine communication; – Non-language level lambda implementation, combined with coroutine in-place Write and execute background asynchronous tasks (New); – A small and light network framework based on epoll/kqueue, a high-performance timer;

generated by libco based on time roulette Background

In the early days of WeChat’s backend, due to complex and changeable business needs and rapid product iteration, most modules adopted a semi-synchronous and semi-asynchronous model. The access layer is an asynchronous model, and the business logic layer is a synchronous multi-process or multi-thread model. The concurrency capability of the business logic is only dozens to hundreds. As WeChat's business grows, the system scale becomes larger and larger, and each module is easily affected by back-end service/network jitter.

The choice of asynchronous transformation

In order to improve the concurrency capability of WeChat's backend, the general approach is to change all services on the existing network to an asynchronous model. This approach requires a huge amount of work, from the framework to the business logic code, which requires a complete transformation, which is time-consuming, labor-intensive and risky. So we started thinking about using coroutines.

However, using coroutines will face the following challenges:

Industry coroutines have no experience in large-scale application in c/c++ environment;

How to control coroutine scheduling;

How to deal with synchronization style API calls, such as Socket, mysqlclient, etc.;

How to deal with the use of existing global variables and thread private variables;

In the end, we solved all the above problems through libco and achieved non-intrusive asynchronous transformation of business logic. We used libco to transform hundreds of WeChat backend modules into coroutines and asynchronous transformations. During the transformation process, the business logic code was basically unchanged. So far, most of the services in the WeChat backend are multi-process or multi-threaded coroutine models. The concurrency capabilities have been qualitatively improved compared to before, and libco has become the cornerstone of the WeChat backend framework.

libco framework

The libco framework is divided into three layers, namely the interface layer, systemfunctionHook layer andeventdriverlayer.

Processing of synchronous style APIs

For synchronous style APIs, which are mainly synchronous network calls, libco’s primary task is to eliminate the occupation of resources by these waits and improve the concurrency performance of the system. For a regular network background service, we may go through steps such as connect, write, and read to complete a complete network interaction. When calling these APIs synchronously, the entire thread will hang waiting for network interaction.

Although the concurrency performance of the synchronous programming style is not good, it has the advantages of clear code logic, easy writing, and can support rapid business iteration and agile development. In order to continue to maintain the advantages of synchronous programming without modifying the existing business logic code online, libco innovatively took over the network call interface (Hook) and registered the surrender and recovery of the coroutine as an event in asynchronous network IO. with callbacks. When business processing encounters a synchronous network request, the libco layer will register the network request as an asynchronous event. This coroutine will give up the CPU occupation, and the CPU will be handed over to other coroutines for execution. Libco will automatically resume coroutine execution when a network event occurs or times out.

We have taken over most of the synchronization style APIs through the Hook method, and libco will schedule the coroutine to resume execution at the appropriate time.

Support for tens of millions of coroutines
By default, libco allows each coroutine to have its own running stack. When the coroutine is created, a fixed-size memory is allocated from the heap memory as the running stack for the coroutine. If we use a coroutine to handle an access connection on the front end, then for a massive access service, the concurrency limit of our service will easily be limited by memory. To this end, libco also provides a stackless coroutine sharing stack mode, which allows you to set up several coroutines to share the same running stack. When switching between coroutines under the same shared stack, the current running stack content needs to be copied to the coroutine's private memory. In order to reduce the number of such memory copies, the memory copy of the shared stack only occurs when switching between different coroutines. When the occupier of the shared stack has not changed, there is no need to copy the running stack.

The shared coroutine stack mode of libco coroutines makes it easy for a single machine to access tens of millions of connections, just by creating enough coroutines. We create 10 million coroutines (E5-2670 v3 @ 2.30GHz * 2, 128G memory) through the libco shared stack mode. Each 100,000 coroutines use 128k memory, and the entire echo service is stable The total memory consumption is about 66G.
Coroutine private variables
When a multi-process program is transformed into a multi-threaded program, we can use thread to quickly modify global variables. In the coroutine environment, we created the coroutine variable ROUTINE_VAR, which greatly simplifies Reduce the workload of coroutine transformation.
Since coroutines are essentially executed serially within a thread, when we define a thread private variable, there may be reentrancy issues. For example, if we define a thread private variable of a thread, we originally wanted each execution logic to have exclusive use of this variable. But when our execution environment is migrated to coroutines, the same thread private variable may be operated by multiple coroutines, which leads to the problem of variable intrusion. For this reason, when we were doing the asynchronous transformation of libco, we changed most of the thread private variables into coroutine-level private variables. Coroutine private variables have the following characteristics: when the code is running in a multi-threaded non-coroutine environment, the variable is thread-private; when the code is running in a coroutine environment, this variable is coroutine-private. The underlying coroutine private variables will automatically determine the running environment and correctly return the required value.
Coroutine private variables play a decisive role in transforming the existing environment from synchronization to asynchronousization. At the same time, we have defined a very simple and convenient method to define coroutine private variables, which is as simple as just one line of declaration code.
　Hook method of gethostbyname
For existing network services, it may be necessary to query DNS through the system's gethostbyname API interface to obtain the real address. During the coroutine transformation, we found that the socket family function of our hook is not applicable to gethostbyname. When a coroutine calls gethostbyname, it will wait for the result synchronously, which causes other coroutines in the same thread to be delayed in execution. We studied the gethostbyname source code of glibc and found that the hook does not take effect mainly because glibc defines a poll method to wait for events instead of a general poll method; at the same time, glibc also defines a thread private variable to switch between different coroutines. Reentrancy may result in inaccurate data. The final asynchronousization of the gethostbyname coroutine is solved through the Hook poll method and the definition of coroutine private variables.
Gethostbyname is a synchronous query dns interface provided by glibc. There are many excellent asynchronous solutions for gethostbyname in the industry, but these implementations require the introduction of a third-party library and require the underlying layer to provide an asynchronous callback notification mechanism. Through the hook method, libco realizes the asynchronousization of gethostbyname without modifying the glibc source code.
Coroutine Semaphore
In a multi-threaded environment, we will have synchronization needs between threads. For example, the execution of one thread needs to wait for the signal of another thread. For this need, we usually use pthread_signal to solve it. of. In libco, we define the coroutine semaphore co_signal to handle the concurrency requirements between coroutines. A coroutine can decide to notify a waiting coroutine or wake up all waiting coroutines through co_cond_signal and co_cond_broadcast.
Summary
Libco is an efficient c/c++ coroutine library that provides a complete coroutine programming interface, commonly used Socket family function Hooks, etc., allowing businesses to use synchronous programming models for rapid iterative development. With stable operation over the past few years, libco has played a pivotal role as the cornerstone of WeChat's backend framework.

【Related recommendations】

1. WeChat public account platform source code download

2. Alizi order system source code free download

The above is the detailed content of Teach you how libco supports huge amounts of data information. For more information, please follow other related articles on the PHP Chinese website!