Golang implements log auditing

With the continuous development of business, the importance of system logs and audit logs is becoming higher and higher. The long-term development of the logging system requires an efficient and reliable technology, as well as sufficient flexibility and scalability. In recent years, golang, as an efficient programming language, has also shown unique advantages in log auditing. This article will introduce how golang implements log auditing.

1. Advantages of golang in log auditing

1. Coroutine

Golang has the characteristics of coroutine (goroutine) and can easily create a large number of concurrent execution programs , which enables it to ensure data security and integrity under high concurrent traffic and avoid various cumbersome and dangerous factors under multi-threading.

2. Garbage collection mechanism

golang has its own garbage collection mechanism that can automatically manage memory. This mechanism not only solves the problem of memory leaks, but also reduces the programmer's memory management and processing, improving the efficiency of programming.

3. Cross-platform capability

Golang’s code can be compiled across platforms, which is very convenient. At the same time, golang also supports a variety of operating systems, including mainstream operating systems such as Windows, Linux, macOS, etc., which can meet the needs of different platforms.

4. Performance Optimization

The performance of golang is very excellent, and its speed and execution efficiency are higher than other languages. This means that system performance can be maintained without reducing system performance under high concurrency, ensuring the response speed of the interface, and improving system availability and stability.

2. Specific methods for implementing log auditing in golang

1. Design of log system

Golang generally needs to consider the following aspects when implementing log auditing:

• Log record formatting: The log system needs to be able to parse log records, convert and store them according to the specified format.
• Database operation: By using golang's ORM framework, we can easily operate the database and store and read data.
• Authentication: In the log audit system, authentication is required to audit certain operations.
• Log query: For the query requirements of the log system, it is necessary to consider how to design reasonable query conditions and query methods.
• Log storage: You need to choose an appropriate storage method to ensure the integrity and security of log data.

2. Use the logrus library to implement logging

In golang, logrus is a very popular logging framework that supports multiple log output methods and provides a rich API interface. . Simply put, logrus is a logging library that can provide highly customizable logging solutions.

logrus supports the following levels of log output:

• trace
• debug
• info
• warn
• error
• fatal
• panic

Using logrus can easily achieve the recording, output and management of logs. For example, we can customize the output format, Output level, output to file, output to another machine, etc. This flexibility makes golang an ideal choice for log auditing.

3. Implement the audit log system

Let’s implement a simple audit log system. This system can record user operation logs and store them in the database to facilitate auditing by administrators.

First, to build the golang web service environment, we can use the third-party library gorilla/mux to handle routing. After implementing routing, we need to define different handler functions for each interface, such as login, logout, query, etc.

After processing each interface, let's implement audit log recording. We can use the logrus library to record logs. First define a log processor, which can output the logs to the console or other external files.

logrus.SetLevel(logrus.TraceLevel)
logrus.SetOutput(os.Stdout)

Then, during the process of processing the interface, we need to record the operations of each user, specifically How should it be recorded? You can record user operations by defining a log format template and log object. When we define the log, we will record the user name and IP address of the user's login, as well as the accessed interface and request method. This makes it very convenient to record audit logs.

Log object definition:

type AuditLog struct {

ID uint64
Username string
IPAddress string
Method string
Path string
CreatedAt time.Time

}

Log format:

func (auditLog *AuditLog) String() string {

// format the log into json format
buf := bytes.NewBuffer(nil)

// begin log format
buf.WriteString("{")
buf.WriteString(fmt.Sprintf(""id":"%d",", auditLog.ID))
buf.WriteString(fmt.Sprintf(""username":"%s",", auditLog.Username))
buf.WriteString(fmt.Sprintf(""ip_address":"%s",", auditLog.IPAddress))
buf.WriteString(fmt.Sprintf(""method":"%s",", auditLog.Method))
buf.WriteString(fmt.Sprintf(""path":"%s",", auditLog.Path))
buf.WriteString(fmt.Sprintf(""created_at":%d", auditLog.CreatedAt.Unix()))
// end log format
buf.WriteString("}")

return buf.String()

}

Record audit log:

func AuditingLogMiddleware(next http.Handler) http.Handler {

return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    start := time.Now()

    var username string
    var ok bool

    // 根据实际场景修改，这里以token作为身份认证方式
    if token := r.Header.Get("Authorization"); token != "" {
        tokenParts := strings.Split(token, " ")

        if len(tokenParts) == 2 {
            tokenType := tokenParts[0]
            tokenContent := tokenParts[1]

            // 根据实际场景修改，这里假设发放的Token为JWT
            if tokenType == "Bearer" {
                claims, err := util.ParseJwtToken(util.JwtKey, tokenContent)
                if err == nil {
                    username, ok = claims["username"].(string)
                }
            }
        }
    }

    // 添加审计日志
    auditLog := &AuditLog{
        Username:  username, // 登录的用户名
        IPAddress: r.RemoteAddr, // 客户端IP地址
        Method:    r.Method, // http请求类型（GET、POST、PUT、DELETE……）
        Path:      r.URL.Path, // 请求的URL路径
        CreatedAt: time.Now(), // 日志创建时间
    }

    logrus.Trace(auditLog)

    // Call the next handler, which can be another middleware in the chain, or the final handler.
    next.ServeHTTP(w, r)

    // 审计log耗时
    end := time.Now()

    diff := end.Sub(start)
    logrus.Tracef("log time to response: %dms", int64(diff/time.Millisecond))
})

}

After recording the log, we need to write the log to the database. Here we assume that gorm is used as the ORM framework to operate the database.

GORM is a lightweight ORM library for SQL databases, supporting MySQL, PostgreSQL, Sqlite3 and other databases.

func DatabaseUri(user, password, database, host, port string) string {

return fmt.Sprintf("%s:%s@tcp(%s:%s)/%s?charset=utf8mb4&parseTime=True&loc=Local", user, password, host, port, database)

}

func SetupDB() error {

var err error

dbUri := DatabaseUri("root", "", "log_audit", "localhost", "3306")
db, err = gorm.Open(mysql.Open(dbUri), &gorm.Config{
    Logger: &logger.Default{},
})

if err != nil {
    panic("failed to connect database")
}

// 定义migration 操作等

return nil

}

Remember to call SetupDB first in the main function, because it is the initialization function for operating the database, and you need to make sure to connect to it before operating the database.

The function that calls AuditLogMiddleware needs to be registered in the service route. It meets different audit requirements as needed. It is judged that the business needs need to be synchronously written to reliable message queues such as kafka, nats, nsq, etc. for follow-up. Asynchronous processing.

At this point, golang’s implementation of log auditing is complete. Through the above methods, we can implement an efficient and reliable logging system and audit module in golang, laying a solid foundation for our business development and better ensuring the reliability and security of the system.

The above is the detailed content of Golang implements log auditing. For more information, please follow other related articles on the PHP Chinese website!