One article to understand the timeout delivery of microservices

                                                                                                                                               

Why is timeout control needed?

A common problem in many cascading failure scenarios is that the server is consuming a lot of resources to process requests that have already exceeded the client's deadline. As a result, the server consumes a lot of resources without doing any valuable work. , it is meaningless to reply to a request that has timed out.

Timeout control can be said to be an important line of defense to ensure service stability. Its essence is to fail fast. A good timeout control strategy can clear high-latency requests as soon as possible and release resources as soon as possible to avoid requests. of accumulation.

Timeout delivery between services

If a request has multiple stages, such as consisting of a series of RPC calls, then our service should check before the start of each stage The deadline is to avoid wasted work, that is, to check whether there is enough time remaining to process the request.

A common implementation error is to set a fixed timeout in each RPC service. We should pass the timeout between each service. The timeout can be set at the top level of the service call, by the initial request The entire RPC tree triggered will have the same absolute deadline set. For example, set the timeout to 3s at the top level of the service request. Service A requests service B. Service B takes 1s to execute. Service B then requests service C. At this time, the timeout period remains 2s. Service C takes 1s to execute. This is When service C then requests service D, service D takes 500ms to execute, and so on. Ideally, the same timeout delivery mechanism is used in the entire call chain.

If the timeout delivery mechanism is not used, the following situation will occur:

1. Service A sends a request to Service B, and the set timeout The time is 3s
2. Service B takes 2s to process the request, and continues to request service C
3. If timeout delivery is used, the timeout period of service C should be 1s, but no timeout delivery is used here, so The timeout time is 3s programmed in the configuration
4. The continued execution of service C takes 2s. In fact, the timeout set by the top layer has expired at this time, and the following request is meaningless
5. Continue the request Service D

If service B adopts the timeout delivery mechanism, then the request should be abandoned immediately in service C, because the deadline has been reached and the client may have reported an error. When we set the timeout delivery, we generally reduce the deadline for delivery, such as 100 milliseconds, in order to take into account the network transmission time and the processing time after the client receives the reply.

Intra-process timeout transmission

Not only timeout transmission is required between services, but also timeout transmission within the process is required. For example, Mysql, Redis and MySQL are serially called in a process. For service B, set the total request time to 3s. It takes 1s to request Mysql and then request Redis again. The timeout is 2s. It takes 500ms to execute Redis and then request service B. The timeout is 1.5s because each of our The middleware or service will set a fixed timeout in the configuration file. We need to take the minimum value of the remaining time and the set time.

context implements timeout delivery

The context principle is very simple, but the function is very powerful, and the standard library of go has also been implemented In addition to supporting context, various open source frameworks have also implemented support for context. Context has become a standard, and timeout delivery also relies on context.

We usually set the initial context at the top layer of the service for timeout control transfer, such as setting the timeout to 3s

ctx, cancel := context.WithTimeout(context.Background(), time.Second*3)defer cancel()

When context transfer is performed, such as requesting Redis in the above figure, then Get the remaining time in the following way, and then compare the timeout set by Redis to get a smaller time

dl, ok := ctx.Deadline()

timeout := time.Now().Add(time.Second * 3)if ok := dl.Before(timeout); ok {
    timeout = dl}

Timeout transfer between services mainly refers to the timeout transfer when RPC is called. For gRPC, we do not need to do it. For additional processing, gRPC itself supports timeout delivery. The principle is similar to the above. It is delivered through metadata and will eventually be converted into the value of grpc-timeout, as shown in the following code grpc-go/internal/transport/handler_server. go:79

if v := r.Header.Get("grpc-timeout"); v != "" {
        to, err := decodeTimeout(v)
        if err != nil {
            return nil, status.Errorf(codes.Internal, "malformed time-out: %v", err)
        }
        st.timeoutSet = true
        st.timeout = to}

Timeout delivery is an important line of defense to ensure service stability. The principle and implementation are very simple. Has timeout delivery been implemented in your framework? If not, take action quickly.

Timeout delivery in go-zero

go-zero can be configured through Timeout in the configuration fileapi gateway andrpc The timeout of the service and will be automatically passed between services.

The previous article understands how to implement Go timeout control which explains how to use timeout control.

Reference

"SRE: Google Operation and Maintenance Decryption"

Project address

github.com /zeromicro/go-zero

Welcome to use go-zero and star/fork to support us!

Recommended: "golang tutorial"

The above is the detailed content of One article to understand the timeout delivery of microservices. For more information, please follow other related articles on the PHP Chinese website!