Describe in detail how to optimize nginx logs by day

This article brings you relevant knowledge about nginx, which mainly introduces how to optimize nginx logs on a daily basis. Friends who are interested can take a look at it together. I hope it will be helpful to everyone.

Foreword:

Following the previous article "Remember an nginx interception crawler", I found that the server nginx log is very ugly, and it has been a long time A lot of them are in one file (even several years old), which is not friendly to a Linux novice like me, so I want to divide the logs into files, so that at least it will be much more convenient for us to check the logs:

Files by date

Directly configure nginx.conf

user nginx;
worker_processes 2;

error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;


events {
    worker_connections  1024;
}

http {
    include  /etc/nginx/mime.types;
    default_type application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_iso8601] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for" $request_time $upstream_response_time';

    #引入time_iso8601模块 设置时间 日期变量
    map $time_iso8601 $logdate {
        #&#39;~^(?<ymdh>\d{4}-\d{2}-\d{2}T\d{2})&#39; $ymdh;
        &#39;~^(?<ymd>\d{4}-\d{2}-\d{2})&#39; $ymd;
        default &#39;date-not-found&#39;;
    }

    #日志存放目录
    access_log /var/log/nginx/access-$logdate.log main;

    #日志缓存,将多个日志进行积累,达到一定量级后写入到磁盘,可以减少磁盘旋转,从而降低磁盘i/o,提升nginx能效
    open_log_file_cache max=10;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;

    #隐藏http版本号
    server_tokens off;
}

This will generate log files according to date; the most important thing here is to introduce $time_iso8601 (this is a time format that comes with nginx) is used to customize variables and does not require compilation.

Date format optimization

• nginx comes with two time formats:
  $time_iso8601 shaped like 2023-03-29T16:58:49 08:00
$time_local is shaped like 14/Nov/2022:08:28:14 0000
  It can be seen that the format of $time_local does not look very beautiful. For me, it is more pleasing to the eye in terms of 'year, month, day, hour, minute and second', so when we customize the log format, the time The format also selected is $time_iso8601.

• How can the time be consistent with our local time?
  The time format is pleasing to the eye, but I find that the time is still in the UTC time zone. How to deal with it:
  I am using docker-compose, you can configure environment variables directly.

version: '3'
services:
  d_nginx:
    container_name: c_nginx
    environment:
      TZ: 'Asia/Shanghai'

The second is to compile in Dockerfile, once and for all.

FROM nginx:1.20.1-alpine
#定义时区参数
ENV TZ=Asia/Shanghai

There are some low-level versions of nginx that still fail to do the above two steps. It may be because the tzdata time zone data package is missing.

FROM nginx:1.12.1-alpine
#将alpine的源更换成阿里云的源
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories
#定义时区参数
ENV TZ=Asia/Shanghai
#安装时区数据包 
RUN apk add --update tzdata
#设置时区 
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo '$TZ' > /etc/timezone

After the above steps, the format and time in the nginx log should be what I want.

Share with you an nginx log analysis tool

《goaccess》
goaccess nginx.log -a > nginx .html
This tool can easily and quickly analyze nginx logs, and can visually display a lot of information to help us quickly locate problems; such as the number of interface calls (specific to a certain interface), the number of visitors, etc.

Recommended study: "Nginx usage tutorial"

The above is the detailed content of Describe in detail how to optimize nginx logs by day. For more information, please follow other related articles on the PHP Chinese website!