Using go-zero to implement distributed configuration center

With the development of the Internet, the scale of enterprise applications has gradually increased, and the configurations required for different business scenarios have become more and more complex. The management and maintenance of configurations are often a tedious and error-prone process. In order to solve these problems, the distributed configuration center came into being.

The distributed configuration center is a modular design that centralizes the configuration information of all applications and provides a friendly operation interface to facilitate managers to modify and publish configuration information. By centrally managing configuration information, system failures caused by configuration issues can be effectively reduced.

This article will introduce how to use go-zero to implement a simple distributed configuration center.

What is go-zero?

Go-Zero is a Go language microservice framework. It has the characteristics of high performance, easy scalability and ease of use. It is a way for Go language developers to quickly build high-performance, scalable and reliable microservice applications. One of the preferred frameworks.

In addition to providing microservice-related functions such as service registration, health check, current limiting circuit breaker, long connection management and service governance, Go-Zero also provides many tools to assist development, such as Rpc generation tools, http API generation tools, configuration center, log library, cache library, etc.

Implementation principle of distributed configuration center

The implementation of distributed configuration center needs to take into account the following aspects:

1. Database storage: a relational database is required (such as MySQL, PostgreSQL, etc.) to store configuration information to ensure persistent storage of configuration information.
2. Backend management: It is necessary to provide a backend management system through which administrators can add, delete, modify, check, and publish configuration information.
3. Configuration file loading: An interface for loading configuration files needs to be provided for application calls to ensure that applications can obtain the latest configuration information.
4. Scheduled refresh: It is necessary to implement the function of regularly refreshing configuration information to ensure timely update of data.
5. Distributed consistency: When deploying multiple nodes, the consistency of configuration information needs to be considered to avoid errors caused by node out-of-synchronization.

Use go-zero to implement a distributed configuration center

This article will briefly introduce the process of using the go-zero framework to implement a distributed configuration center. The specific steps are as follows:

1. Install go-zero

To use go-zero, you need to install the relevant dependencies first:

go get -u github.com/tal-tech/go-zero

2. Create the database

First create the database and create the table and table structure As follows:

CREATE TABLE `config` (
    `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
    `app_name` varchar(255) DEFAULT '',
    `key_name` varchar(255) DEFAULT '',
    `value` varchar(1024) DEFAULT '',
    `create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
    `update_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

3. Create a backend management system

The backend management system is mainly used for adding, deleting, modifying, checking and publishing operations on configuration information. In the go-zero framework, you can use the goctl tool to quickly generate management system-related code:

goctl api new -api config -dir config/api

The generated code is located in the config/api directory and needs to be adjusted according to actual needs.

4. Implement configuration file loading

Generate an rpc service named config through the goctl tool, and load the configuration file by calling its interface.

The service interface is defined as follows:

type Config interface {
    GetConfig(ctx context.Context, req *model.GetConfigReq) (*model.GetConfigResp, error)
    WatchConfig(ctx context.Context, req *model.GetConfigReq) (*model.GetConfigResp, error)
}

5. Implementing scheduled refresh

In order to implement the scheduled refresh function, you can use etcd related tools in the go-zero framework.

First you need to install etcd:

go get -u go.etcd.io/etcd/client/v3

Then set the address and port of etcd in the configuration file:

[etcd]
  null=127.0.0.1:2379

Finally, implement the scheduled refresh logic in the code:

func RefreshConfig() {
    etcdCli, err := clientv3.New(clientv3.Config{
        Endpoints:   *conf.Etcd,
        DialTimeout: time.Second * 3,
    })
    if err != nil {
        logx.Errorf("err: %v", err)
        return
    }
    defer etcdCli.Close()

    for {
        ctx, cancel := context.WithTimeout(context.Background(), time.Second*5)
        resp, err := etcdCli.Get(ctx, *conf.EtcdKey)
        if err != nil {
            logx.Errorf("err: %v", err)
            cancel()
            continue
        }
        if len(resp.Kvs) == 1 {
            var configMap map[string]string
            err = json.Unmarshal(resp.Kvs[0].Value, &configMap)
            if err != nil {
                logx.Errorf("err: %v", err)
            } else {
                cacheConfigMap.Lock()
                cacheConfigMap.data = configMap
                cacheConfigMap.Unlock()
                logx.Info("Refresh config success")
            }
        }
        cancel()
        time.Sleep(time.Second * 10)
    }
}

6. Achieve distributed consistency

In order to achieve distributed consistency, etcd related tools need to be used in the go-zero framework.

First you need to install etcd:

go get -u go.etcd.io/etcd/client/v3

Then implement etcd-related distributed lock logic in the code:

func Lock() error {
    etcdCli, err := clientv3.New(clientv3.Config{
        Endpoints:   *conf.Etcd,
        DialTimeout: time.Second * 3,
    })
    if err != nil {
        logx.Errorf("err: %v", err)
        return err
    }
    defer etcdCli.Close()

    var s *concurrency.Session
    var m *concurrency.Mutex
    for {
        opTimeoutCtx, cancel := context.WithTimeout(context.Background(), time.Second)
        s, err = concurrency.NewSession(etcdCli,
            concurrency.WithContext(opTimeoutCtx),
            concurrency.WithTTL(int32(*conf.LockTtl)))
        if err != nil {
            logx.Errorf("create etcd session error: %v", err)
            cancel()
            time.Sleep(time.Second)
            continue
        }

        opTimeoutCtx, cancel = context.WithTimeout(context.Background(), time.Second)
        m = concurrency.NewMutex(s, *conf.EtcdKey)
        err = m.Lock(opTimeoutCtx)
        if err != nil {
            logx.Errorf("etcd lock failed: %v", err)
            cancel()
            time.Sleep(time.Second)
            continue
        }
        break
    }

    cacheConfigMap.Lock()
    defer cacheConfigMap.Unlock()

    defer func() {
        if m != nil {
            err = m.Unlock(context.Background())
            if err != nil {
                logx.Errorf("etcd unlock failed: %v", err)
            }
        }
    }()
    defer func() {
        if s != nil {
            s.Close()
        }
    }()
    return nil
}

Conclusion

This article introduces how to use The go-zero framework implements a simple distributed configuration center. By using go-zero's high performance, easy scalability, and ease of use, we can quickly build a highly available distributed configuration center in a short time, effectively helping us reduce system failures caused by configuration issues.

The above is the detailed content of Using go-zero to implement distributed configuration center. For more information, please follow other related articles on the PHP Chinese website!