How to deal with Nginx discarding http packet body

nginx discards the http package body processing example detailed explanation

The http framework discards the http request package body and the previous article http framework receives the package body, both are provided by the http framework Method, called by each http module to decide what to do with the package body. Whether to choose to discard or receive is determined by the module. For example, if the static resource module receives a get request from the browser and requests a file, it can directly return the content of the file to the browser. There is no need to receive the package body data, and the get request will not actually have a package body. Therefore, the static resource module will call the discard packet body function provided by the http framework to perform packet discard processing.

Compared with the process of receiving the package body, the operation of discarding the package body is much simpler. At least there is no need to store the package body in the request_body buffer in the http structure, and there is no need to consider whether the package body is only stored in To the memory, or only to the file, etc., the framework discards it directly after receiving the package body. The discarded packet body consists of three parts:

(1) The http module calls the ngx_http_discard_request_body function provided by the framework for the first time to do some initialization operations. For example, if all packet bodies cannot be discarded in one operation, the read event needs to be registered in epoll again, so that the packet discarding operation can be continued when the execution is scheduled again. Furthermore, call the actual packet discarding function ngx_http_read_discarded_request_body to discard the packet body.

(2) If all the bags cannot be discarded at one operation, when the event is scheduled again, continue to receive the remaining bag data and then discard.

(3) Actual packet loss processing, that is, directly discarding the packet body after receiving it.

It can be seen from the figure that among these three processes, the packet loss process is a common function. That is to say, whether the http module calls the ngx_http_discard_request_body function to start packet loss processing, or when the entire packet body is not received in one schedule, ngx_http_discarded_request_body_handler is responsible for discarding the remaining packet body operations, and the public packet loss function ngx_http_read_discarded_request_body will be called to receive the packet body directly. Discard operation.

1. Packet loss initialization process

ngx_http_discard_request_body is a function called by the http module to discard the packet body. It is a transparent operation for the module. In other words, the module only needs to call this interface to discard the http request packet body, without knowing how the http framework implements this interface. Even if the framework does not discard all packet bodies in one scheduling, the packet discarding operation will be performed again when the next scheduling is executed, but the modules do not know this.

//功能: 丢弃http包体的首次回调函数，如果一次性不能全部接收完成并丢弃，则设置 
//  读事件的回调为ngx_http_discarded_request_body_handler 
ngx_int_t ngx_http_discard_request_body(ngx_http_request_t *r) 
{ 
 
 //需要丢弃的包体不用考虑超时问题 
 if (rev->timer_set) 
 { 
  ngx_del_timer(rev); 
 } 
 
 //包体长度小于等于0，则直接返回。表示丢弃包体 
 //如果已经接收过包体了，这时也不需要在接收。通常情况下get请求没有包体，因此包体长度为0 
 if (r->headers_in.content_length_n <= 0 || r->request_body) 
 { 
  return ngx_ok; 
 } 
 
 size = r->header_in->last - r->header_in->pos; 
 //已经预先接收了部分包体 
 if (size) 
 { 
  //包体未全部接收完成 
  if (r->headers_in.content_length_n > size) 
  { 
   r->header_in->pos += size; 
   r->headers_in.content_length_n -= size; 
 
  } 
  else 
  { 
   //包体已经全部接收 
   r->header_in->pos += (size_t) r->headers_in.content_length_n; 
   r->headers_in.content_length_n = 0; 
   return ngx_ok; 
  } 
 } 
 
 //设置后续读事件的回调 
 r->read_event_handler = ngx_http_discarded_request_body_handler; 
 
 //注册读事件回调，插入到epoll 
 ngx_handle_read_event(rev, 0)); 
  
 //接收包体内容 
 if (ngx_http_read_discarded_request_body(r) == ngx_ok) 
 { 
  //表示已经接收到完整的包体了，将延迟关闭清0 
  r->lingering_close = 0; 
 
 } 
 else 
 { 
  //表示需要多次调度才能完成丢弃包体这个操作，于是把引用计数加1，防止这边在丢弃包体，而其他 
  //事件却已经让请求意外销毁 
  r->count++; 
  //标识为正在丢弃包体 
  r->discard_body = 1; 
 } 
 
 return ngx_ok; 
}

When receiving the http request header, if the http package body data is also received by the way, there is no need to continue to perform the remaining operations at this time. If the package body is discarded successfully, the function returns directly. If all packet bodies are not discarded in one schedule, the read event read_event_handler of the http request structure ngx_http_request_s will be set to: ngx_http_discarded_request_body_handler. This function will be responsible for discarding the remaining packet bodies the next time it is scheduled. Therefore ngx_http_discard_request_body will only be called for the first time by the http module.

The function will also call the actual packet loss function ngx_http_read_discarded_request_body to start receiving the packet body and discard it directly.

2. Packet loss processing

The ngx_http_read_discarded_request_body function is responsible for receiving the packet body data from the client and then discarding it. Therefore, for the module, it is the operation of discarding the package body, but for the framework, the operation of discarding the package body is actually the operation of receiving the package body, but the received package body data is not given to the module for use. Why does the framework need to receive the package body and then discard it directly? Isn't it unnecessary? It’s not, there’s a reason why it’s done this way. Assume that an unrobust client browser uses a blocking method to send http packet body data to the nginx server. If the nginx framework does not receive it, it will cause the client browser to time out and not respond, causing the client browser to close the connection. . Therefore, the http framework of nginx must first receive the package body data from the client from the kernel, but this data is useless to the module, so the received data will be discarded directly.

//功能: 从内核中读取数据到nginx中，nginx不对收到的数据进行处理。相当于丢弃包体 
static ngx_int_t ngx_http_read_discarded_request_body(ngx_http_request_t *r) 
{ 
 //用于接收包体的临时缓冲区 
 u_char buffer[ngx_http_discard_buffer_size]; 
 
 for ( ;; ) 
 { 
  //已经全部丢弃成功 
  if (r->headers_in.content_length_n == 0) 
  { 
   //设置丢弃后的读事件回调，再有读事件时，不做任何处理 
   r->read_event_handler = ngx_http_block_reading; 
   return ngx_ok; 
  } 
   
  //从内核中接收包体到临时缓冲区 
  n = r->connection->recv(r->connection, buffer, size); 
 
  //更新剩余需要接收的包体大小 
  r->headers_in.content_length_n -= n; 
 } 
}

        函数内部只是使用一个临时的缓冲区变量存放每次接收来自内核的包体数据。并没有把这部分数据保存到http请求结构中的request_body缓冲区。因此包体数据没有交给http模块，相当于被丢弃了。在所有包体从内核中接收完成时，设置http请求结构ngx_http_request_s的读事件read_event_handler回调设置为: ngx_http_block_reading， 表示再收到来自客户端的数据，则不进行任何处理了。因为已经接收完所有的包体数据，也就不需要理会来自客户端浏览器的其它数据。

三、丢弃剩余的包体

        ngx_http_discarded_request_body_handler用于在一次调度中没有丢弃完所有包体，则该函数会表调用，用于丢弃剩余的包体。函数内部也会调用实际的丢弃包体函数，进行接收包体然后丢弃操作。nginx服务器做了一个优化处理，会设置一个总超时时间，如果超过这个时间都还没有丢弃完全部的包体，则会关闭这个连接。这是一种对服务器保护的措施，避免长时间的丢包操作占用服务器资源。

//功能: 第1次未能全部丢弃包体时，该函数被调用。之后有读事件时，该函数被调用 
void ngx_http_discarded_request_body_handler(ngx_http_request_t *r) 
{ 
 //检测延迟关闭时间,如果总时长超过了lingering_time，则不再接收任何包体，这是一个总时间。 
 //总超时后，将直接光比连接 
 if (r->lingering_time) 
 { 
  timer = (ngx_msec_t) (r->lingering_time - ngx_time()); 
  //已经到达了延迟关闭时间 
  if (timer <= 0) 
  { 
   //清空丢弃包体标识，表示包体已经丢弃 
   r->discard_body = 0;  
   //延迟关闭开关清0 
   r->lingering_close = 0; 
   ngx_http_finalize_request(r, ngx_error); 
   return; 
  } 
 
 } 
 
 //接收包体后丢弃 
 rc = ngx_http_read_discarded_request_body(r); 
 //表示包体已经全部丢弃 
 if (rc == ngx_ok) 
 { 
  r->discard_body = 0;  //包体已经全部接收完 
  r->lingering_close = 0;  //清空延迟关闭标志 
  ngx_http_finalize_request(r, ngx_done); 
  return; 
 } 
}

        ngx_http_discarded_request_body_handler这个函数是怎么被事件对象调用的呢? 在前面的文章已经分析了，ngx_connection_s读事件的回调设置为ngx_http_request_handler。   因此在读事件发生时，会回调请求结构的读回调。如果还不是不清楚这个调用过程，可以参考:

static void ngx_http_request_handler(ngx_event_t *ev) 
{ 
 //如果同时发生读写事件，则只有写事件才会触发。写事件优先级更高 
 if (ev->write)  
 { 
  r->write_event_handler(r); //在函数ngx_http_handler设置为ngx_http_core_run_phases 
 } 
 else 
 { 
  r->read_event_handler(r); //在函数ngx_http_process_request设置为ngx_http_block_reading 
 } 
}

The above is the detailed content of How to deal with Nginx discarding http packet body. For more information, please follow other related articles on the PHP Chinese website!