nginx analysis of keepalive and pipeline request processing

Original article, please indicate when reprinting: Reprinted from pagefault

The link address of this article: nginx analysis of keepalive and pipeline request processing

This time we mainly look at the processing of keepalive and pipeline in nginx. There is no need to introduce the concepts here. . Let’s look directly at how nginx does it. First let’s look at the processing of keepalive. We know that keepalive is the default in http 1.1, unless the client explicitly specifies the connect header as close. The following is the code for nginx to determine whether keepalive is needed.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

void

ngx_http_handler(ngx_http_request_t * r)

{

…. ....................................

                                                                                                                                       headers_in.connection_type) {

case

0://If the version is greater than 1.0, the default is keepalive

r- >keepalive = (r->http_version > NGX_HTTP_VERSION_10);

//If the connection header is specified as close, keepalive is not required keepalive = 0; "

}................................

}

Then we know that keepalive means that the current connection will not be closed directly after the current http request is executed, so the related processing of nginx's keepalive is also the function to clean up the request. nginx’s function to clean up requst is ngx_http_finalize_request. In this function, ngx_http_finalize_connection is called to release the connection, and the relevant judgment of keepalive is in this function.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

staticvoid

ngx_http_finalize_connection(ngx_http_request_t *r)

{

ngx_http_core_loc_conf_t *clcf;

clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module);

. ..................... ................................................

//You can see that if keepalive is set and timeout is greater than 0, keepalive processing will be entered.

                                                                               r->keepalive   

&& clcf->keepalive_timeout > 0)

{

ngx_http_set_keepalive(r);

} else

if(r->lingering_close && clcf->lingering_timeout > 0) {

                                                                                  return;

}  ngx_http_close_request(r, 0);

}

From the above we can see that keepalive is set through ngx_http_set_keepalive. Next we will look at this function in detail. In this function, pipeline requests will be processed incidentally, so let’s take a look at it together. First, how nginx distinguishes pipeline requests. It will assume that if the data read from the client contains more data, that is, parsing After completing the current request, there is still some data. At this time, it is considered a pipeline request. Another very important point is http_connection. We know from the previous blog that if we need to alloc a large header, it will be taken from hc->free first. If not, it will be created and then handed over to hc-> ;Busy to manage. And this buf will be reused here, because if the large buf is large, it needs to be re-allocated a second time. If the buf is reused here, one allocation will be reduced.

1314

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 2 7 28 29 30 31 32 33 34 35 36 37 38 39 40 41

​ hc = r->http_connection; ​ b = r->header_in; //Generally, after parsing header_in, pos will be set to last. That is to say, the data read is just a complete http request. When pos is less than last, it means it may be a pipeline request. if(b->pos last) { if (b != c->buffer) {                                                       * request processing then we do not use c-> buffer for                                                                                                            ​​ * Now we would move the large header buffers to the free list. */                                                                                                                                                                                                                                                                                     == NULL) { //You can see the number of large_client_headers                                                                                                                                             cscf->large_client_header_buffers.num * sizeof (ngx_buf_t *));                                                                                                                                                                                                                                                                                                                                         ​ <p><code>                } //Then clean up the busy of the current request                                                                                                     1; i++) { f = hc->busy[i];                                           ​​​​​​​ f->pos = f->start;                                                                                                          er_in buf for free use next time .                                                                                                                                             } } Then the next part is the free request and setting the keepalive timer. 12 3 45 6 78 9 1011 12

15

r->keepalive = 0; if

ngx_http_free_request(r, 0); c->data = hc; //Set timer ngx_add_timer(rev, clcf->keepalive_timeout); //Then set readable Event

(ngx_handle_read_event(rev, 0) != NGX_OK) { ngx_http_close_connection(c); ​​​​​​​​ return ; ​​​​​ } wev = c->write; wev->handler = ngx_http_empty_handler; Then the next part is the processing of the pipeline. 12 3 456 7 89 10 1112 13 14 15

if(b->pos last) {

>log

, 0,

"pipelined request"

);

#if (NGX_STAT_STUB)

                 ​_reading, 1);

#endif

//Set tag.

                                                                                                                                                                                                                                 

"reading client pipelined request line"

;

//Then throw it into the post queue and continue processing.

​​​​​ngx_post_event( rev, &ngx_posted_events);

When it reaches the bottom, it means that it is not a pipeline request, so we start to clean up the request and http_connection.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

2 7

28

29

30

                                                                                                    }

b = c->buffer;

if(ngx_pfree(c->pool, b->start) == NGX_OK) {

​​​​

* the special note for ngx_http_keepalive_handler() that

           * c->buffer's memory was freed

                                                                                                                                                                                                                                                       

} "

}

................................................ ................

if(hc->busy) {

for

(i = 0; i nbusy; i++) {                                                  hc ->busy[i] = NULL;

set keepalive handler.

12

3456

789

//This function will be analyzed in detail later

      rev->handler = ngx_http_keepalive_handler;

                                                                                                                                                                                                              0) != NGX_OK) {

                                                                                                                                                          

The last thing is the processing of tcp push. I won’t introduce it here for now. Next, I will have a special blog to introduce nginx’s operation of tcp push. Then let’s look at the ngx_http_keepalive_handler function. This function handles the keepalive connection. When there is a readable event on the connection again, this handler will be called. This handler is relatively simple, it just creates a new buf, and then restarts the execution of an http request (calling ngx_http_init_request).

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

size = b->end - b ->start; ​​​​​​​if (b->pos == NULL) {​​​​​ /*​​​​​​ * The c->buffer's memory was freed by ngx_http_set_keepalive(). .                                                                                                                                                                                                                            ->pos == NULL) {                                                                                                                                                                                        ->pos;                                                           } Then try to read the data, if there is no readable data, the handle will be added to the readable event again 123 4 567 89 n = c->recv(c, b->last, size);c->log_error = NGX_ERR OR_INFO ; if(n == NGX_AGAIN) {                          ​ ngx_http_close_connection(c); } return ;}

Finally, if the data is read, enter request processing.

1

ngx_http_init_request(rev);

Finally, let’s look at the ngx_http_init_request function. This time we mainly look at how nginx reuses the request when the pipeline is requested.
Pay attention to hc->busy[0] here. We know earlier that if it is a pipeline request, we will save the request header_in that has not been parsed before. This is because we may have read the second request of the pipeline request. Some heads.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

//Get the request, here we know that in the pipeline request , we will save the previous request. r = hc->request; if(r) { //If it exists, we reuse the previous request .                                                                             sizeof(ngx_http_request_t));      r->pipeline = hc->pipeline; //If nbusy exists                                                                                                           ;nbusy) { //Save this header_in, and then parse it directly below. GR-& gt; header_in = hc-& gt; busy [0]; }} } else{     r = ngx_pcalloc(c->pool, sizeof (ngx_http_request_t)); _connection(c);                                                                                                                                                  c->data = r; From the above code, and then combined with my previous blog, we know that large header is mainly for pipeline, because in pipeline, if the previous request reads more of the next request header, the next time it is parsed, it may exceed the originally allocated client_header_buffer_size. At this time, we need to reallocate a header, which is a large header, so the httpconnection here is mainly for the pipeline situation, and the keepalive If the connection is not a pipeline request, in order to save memory, the previous request will be released. The above introduces nginx's analysis of keepalive and pipeline request processing, including aspects of the process. I hope it will be helpful to friends who are interested in PHP tutorials.