Nginx link tracking for microservice architecture

In most microservice architectures, Nginx is basically a commonly used access layer facility, so we hope that the request ID will be verified and populated from the Nginx layer and printed in the Nginx request log.

Reading Tips: This article does not provide a complete solution for link tracking, but only provides a support solution for link tracking at the Nginx layer!

1 Background introduction

The birth of microservices has solved many problems of traditional monolithic applications, such as poor maintainability, poor scalability, and poor flexibility (coarse-grained comparison). Although the microservice architecture is good, it also brings many challenges, among which Troubleshooting is one of the challenges that needs to be solved. So, how do you find the root cause of a failure across many applications and instances?

Based on the above requirements, we can centrally collect and display all the logs generated by each transaction in each application (but only if you have: Log Center). This way you can quickly see at which step the transaction failed. If done well, you can also directly carry out secondary development and data analysis. After analyzing the collected logs and faults, they can be displayed intuitively with a graphical interface.

For example, you can display the topology diagram of microservice calls and use colors to distinguish faults (such as commonly used red: indicates abnormality, green: normal, yellow: warning). Then you can classify common faults or exceptions and make a friendly display (to put it bluntly, there is no need to go directly to the stack), such as: NullPointerException: The interface directly and friendly prompts which line of code threw a null pointer, and the input parameter is What... (This is not the focus of this article, so I won’t go into too much nonsense. I will introduce it in detail later when I have the opportunity).

To do link tracking of the entire microservice architecture, we definitely hope that there will be a global transaction ID to associate all logs from the first point when a transaction enters the microservice center (for link tracking, such an ID is definitely not enough , but only this is introduced here). Of course, the most ideal thing is to plan the front-end logs (such as operation logs, data flows, etc.).

2 Nginx

In most microservice architectures, Nginx is basically a commonly used access layer facility, so we hope that the request ID will be verified and populated from the Nginx layer and printed in the Nginx request log. Here are only three ways to implement the transaction ID production method of the Nginx layer.

2.1 Option 2: Splicing based on built-in variables

In versions prior to 1.11.0, we can use splicing to assemble the request ID. The reference configuration is as follows:

<span class="hljs-section">server</span> {
    <span class="hljs-comment"># 定义$request_trace_id的值，在1.11.0之前，我们可以使用类似的方式声明</span>
    <span class="hljs-comment"># 只要能确保其值出现重复的可能性尽可能的小即可。 </span>
    <span class="hljs-attribute">set</span> <span class="hljs-variable">$request_trace_id</span> trace-id-<span class="hljs-variable">$pid</span>-<span class="hljs-variable">$connection</span>-<span class="hljs-variable">$bytes_sent</span>-<span class="hljs-variable">$msec</span>;
    <span class="hljs-attribute">location</span> / {
        <span class="hljs-comment"># ……</span>
        <span class="hljs-comment"># 将此trace_id传递给后端的server，通过header方式，此后我们既可以在环境中获取此header  </span>
        <span class="hljs-attribute">proxy_set_header</span> X-Request-Id <span class="hljs-variable">$request_trace_id</span>;  
    }
}

Parameter description:

• $pid: nginx worker process ID
• $connection: The number of link ids to the upstream server
• $bytes_sent: Number of bytes sent
• $msec: The current time, that is, the time obtained by this variable, including seconds and milliseconds (divided by .)

2.2 Option 3: Implementation based on LUA script

Use the random UUID generated by the system /dev/urandom. The reference script is as follows:

<span class="hljs-comment">---</span>
 <span class="hljs-comment">--- UUID</span>
 <span class="hljs-comment">--- Created by lry.</span>
 <span class="hljs-comment">--- DateTime: 2018/2/25 下午7:38</span>
 <span class="hljs-comment">--- Describe: 用系统/dev/urandom生成的随机uuid</span>
 <span class="hljs-comment">---</span>
 <span class="hljs-keyword">local</span> template =<span class="hljs-string">"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"</span>
 <span class="hljs-keyword">local</span> d = <span class="hljs-built_in">io</span>.open(<span class="hljs-string">"/dev/urandom"</span>, <span class="hljs-string">"r"</span>):read(<span class="hljs-number">4</span>)
 <span class="hljs-built_in">math</span>.randomseed(<span class="hljs-built_in">os</span>.time() + d:byte(<span class="hljs-number">1</span>) + (d:byte(<span class="hljs-number">2</span>) * <span class="hljs-number">256</span>) + (d:byte(<span class="hljs-number">3</span>) * <span class="hljs-number">65536</span>) + (d:byte(<span class="hljs-number">4</span>) * <span class="hljs-number">4294967296</span>))
<span class="hljs-keyword">local</span> uuid=<span class="hljs-built_in">string</span>.gsub(template, <span class="hljs-string">"x"</span>,
  <span class="hljs-function"><span class="hljs-keyword">function</span> <span class="hljs-params">(c)</span></span>
    <span class="hljs-keyword">local</span> v = (c == <span class="hljs-string">"x"</span>) <span class="hljs-keyword">and</span> <span class="hljs-built_in">math</span>.random(<span class="hljs-number">0</span>, <span class="hljs-number">0xf</span>) <span class="hljs-keyword">or</span> <span class="hljs-built_in">math</span>.random(<span class="hljs-number">8</span>, <span class="hljs-number">0xb</span>)
    <span class="hljs-keyword">return</span> <span class="hljs-built_in">string</span>.format(<span class="hljs-string">"%x"</span>, v)
  <span class="hljs-keyword">end</span>)
<span class="hljs-keyword">return</span> uuid

2.3 Solution 1: Based on $request_id Implementation

Nginx在 1.11.0版本中就提供了内置变量 $request_id ，其原理就是生成32位的随机字符串，虽不能比拟UUID的概率，但32位的随机字符串的重复概率也是微不足道了，所以一般可视为UUID来使用即可。参考配置如下：

<span class="hljs-comment"># Nnginx代理默认会把header中参数的 "_" 下划线去掉，所以后台服务器后就获取不到带"_"线的参数名</span>
<span class="hljs-attribute">underscores_<span class="hljs-keyword">in</span>_headers</span> <span class="hljs-literal">on</span>;

<span class="hljs-comment"># 设定日志格式</span>
<span class="hljs-attribute"><span class="hljs-built_in">log</span>_format</span> main  \<span class="hljs-string">'<span class="hljs-variable">$remote_addr</span> - <span class="hljs-variable">$remote_user</span> [<span class="hljs-variable">$time_local</span>] "<span class="hljs-variable">$request</span>" \'
                 \'<span class="hljs-variable">$status</span> <span class="hljs-variable">$body_bytes_sent</span> "<span class="hljs-variable">$http_referer</span>" <span class="hljs-variable">$upstream_http_request_id</span> \'
                 \'"<span class="hljs-variable">$http_user_agent</span>" "<span class="hljs-variable">$http_x_forwarded_for</span>"\';

server {
    location / {
        <span class="hljs-comment"># 如果请求头中已有该参数,则获取即可;如果没有,则使用</span><span class="hljs-variable"><span class="hljs-comment">$request_id</span></span><span class="hljs-comment">进行填充</span>
        <span class="hljs-built_in">set</span> <span class="hljs-variable">$temp_request_id</span> <span class="hljs-variable">$http_x_request_id</span>;
        <span class="hljs-keyword">if</span> (<span class="hljs-variable">$temp_request_id</span> = "") {
            <span class="hljs-built_in">set</span> <span class="hljs-variable">$temp_request_id</span> <span class="hljs-variable">$request_id</span>;
        }
        <span class="hljs-comment"># 屏蔽掉原来的请求头参数</span>
        proxy_<span class="hljs-built_in">set</span>_header  x_request_id        "";
        <span class="hljs-comment"># 设置向后转发的请求头参数</span>
        proxy_<span class="hljs-built_in">set</span>_header  X-Request-Id        <span class="hljs-variable">$temp_request_id</span>;
    }
}
</span>

3 最佳实践

生成交易ID的方式有很多种，但希望使用者结合自身实际情况进行合理取舍，而不要盲目的追求ID的唯一性、可读性和时序性等等。

比如，ID具有时序性虽然有一定的好处，但实际的架构根本没有去使用该时序性，则没必要花大量的精力和做出大量的开发，去实现一个有时序性的交易ID。又比如，觉得UUID可读性太差，从而花了很多成本去开发一个具有一定含义的交易ID（如前几位表示什么意思，多少位到多少位又表示什么意思之类的），开发出来后，实际架构根本没有去解读该ID的地方，则浪费了成本。

但也不是所有人都直接使用UUID就能满足的，比如我需要考虑日志的容量，则可以考虑适当缩减ID的长度（每个ID缩减10个字符串，每笔交易就可能少几百或几千个字符串，再往上规划，还是可以减少一些日志容量的）。

最后，如果有考虑想收集前端的日志的童鞋，建议交易ID就不要使用Long型，因为前端可能会有损失精度的问题。同时也建议使用 $request_id  来填充交易ID。

The above is the detailed content of Nginx link tracking for microservice architecture. For more information, please follow other related articles on the PHP Chinese website!