MySQL replication instance detailed explanation of GTID replication

This article brings you relevant knowledge about mysql, which mainly introduces issues related to GTID replication. GTID is the number of a submitted transaction and is a globally unique Number, let’s take a look at it together, I hope it will be helpful to everyone.

Recommended learning: mysql video tutorial

Starting from MySQL 5.6.5, a new GTID-based replication method has been added. GTID ensures that each transaction submitted on the main database has a unique ID in the cluster. This method strengthens the primary and secondary consistency, fault recovery and fault tolerance capabilities of the database.

What is GTID

GTID (Global Transaction ID) is the number of a submitted transaction and is a globally unique number. GTID is actually composed of UUID TID. The UUID is the unique identifier of a MySQL instance. TID represents the number of transactions that have been committed on this instance, and increases monotonically as transactions are committed.

The following is the specific form of a GTID: 3E11FA47-71CA-11E1-9E33-C80AA9429562:23. The colon separates the uuid in front and the TID in the back.

The GTID collection can contain transactions from multiple MySQL instances, separated by commas.

If the transaction sequence number from the same MySQL instance has multiple range intervals, use colons to separate each group of ranges. For example: e6954592-8dba-11e6-af0e-fa163e1cf111:1-5:11-18,e6954592-8dba-11e6-af0e-fa163e1cf3f2:1-27.

What are the GTID improvements?

In the original binary log-based replication, the slave library needs to tell the master library which offset to perform incremental synchronization from. If specified Errors can cause data to be omitted, resulting in data inconsistencies. With the help of GTID, in the event of a master-slave switchover, MySQL's other slave databases can automatically find the correct replication location on the new master database, which greatly simplifies the maintenance of the cluster under complex replication topologies and reduces the occurrence of manual setting of replication locations. Risk of misuse. In addition, GTID-based replication can ignore transactions that have already been executed, reducing the risk of data inconsistency.

Based on gtid set, the master database can accurately know what data is missing from the slave database, and will not provide more or less data to the slave database to avoid wasting network bandwidth.

Mysql master-slave structure has no advantage for GTID in the case of one master and one slave. However, the advantages of the structure with more than two masters are extremely obvious, and the new master can be switched without losing data.

Note: Before building master-slave replication, perform some operations (such as data cleaning, etc.) on an instance that will become the master, and replicate through GTID. These are before the master-slave is established. The operation will also be copied to the slave server, causing the replication to fail. That is to say, replication through GTID always starts from the earliest transaction log, even if these operations are performed before replication. For example, if you perform some drop and delete cleanup operations on server1, and then perform a change operation on server2, server2 will also perform cleanup operations on server1.

How GTID works

1. When a transaction is executed and submitted on the main library side, a GTID is generated and recorded in the binlog log.
2. After the binlog is transferred to the slave and stored in the slave's relaylog, read the value of the GTID and set the gtid_next variable, which tells the slave the next GTID value to be executed.
3. The sql thread obtains the GTID from the relay log, and then compares the binlog on the slave side to see whether the GTID exists.
4. If there is a record, it means that the transaction with the GTID has been executed and the slave will ignore it.
5. If there is no record, the slave will execute the GTID transaction and record the GTID to its own binlog. Before reading and executing the transaction, it will first check that other sessions hold the GTID to ensure that it is not executed repeatedly.

One master and one slave GTID copy setup

Host planning:

• master: docker, port 3312
• slave: docker , Port 3313

Master configuration

The content of the configuration file my.cnf is as follows:

$ cat /home/mysql/docker-data/3313/conf/my.cnf
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html

[mysqld]
#
# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
# innodb_buffer_pool_size = 128M
#
# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin
#
# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
# join_buffer_size = 128M
# sort_buffer_size = 2M
# read_rnd_buffer_size = 2M
#datadir=/home/mysql/docker-data/3307/data
#socket=/home/mysql/docker-data/3307/mysql.sock

character_set_server=utf8
init_connect='SET NAMES utf8'

# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

#log-error=/home/mysql/docker-data/3307/logs/mysqld.log
#pid-file=/home/mysql/docker-data/3307/mysqld.pid
lower_case_table_names=1
server-id=1403311
log-bin=mysql-bin
binlog-format=ROW
auto_increment_increment=1
auto_increment_offset=1
# 开启gtid
gtid_mode=ON
enforce-gtid-consistency=true

#rpl_semi_sync_master_enabled=1
#rpl_semi_sync_master_timeout=10000

Create a docker instance:

$ docker run --name mysql3312 -p 3312:3306 --privileged=true -ti -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE=order -e MYSQL_USER=user -e MYSQL_PASSWORD=pass -v /home/mysql/docker-data/3312/conf:/etc/mysql/conf.d -v /home/mysql/docker-data/3312/data/:/var/lib/mysql -v /home/mysql/docker-data/3312/logs/:/var/log/mysql -d mysql:5.7

Add for Copied user and authorization:

mysql> GRANT REPLICATION SLAVE,FILE,REPLICATION CLIENT ON *.* TO 'repluser'@'%' IDENTIFIED BY '123456';
Query OK, 0 rows affected, 1 warning (0.01 sec)

mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.01 sec)

slave configuration

The content of the configuration file my.cnf is consistent with the master. Be careful to modify the server-id to keep it unique.

Create docker instance:

$ docker run --name mysql3313 -p 3313:3306 --privileged=true -ti -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE=order -e MYSQL_USER=user -e MYSQL_PASSWORD=pass -v /home/mysql/docker-data/3313/conf:/etc/mysql/conf.d -v /home/mysql/docker-data/3313/data/:/var/lib/mysql -v /home/mysql/docker-data/3313/logs/:/var/log/mysql -d mysql:5.7

Enable GTID synchronization:

mysql> change master to master_host='172.23.252.98',master_port=3310,master_user='repluser',master_password='123456',master_auto_position=1;
Query OK, 0 rows affected, 2 warnings (0.02 sec)

mysql> start slave;
Query OK, 0 rows affected (0.02 sec)

View status:

mysql> show master status;
+------------------+----------+--------------+------------------+----------------------------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set                      |
+------------------+----------+--------------+------------------+----------------------------------------+
| mysql-bin.000008 |      154 |              |                  | cd2eaa0a-7a59-11ec-b3b4-0242ac110002:1 |
+------------------+----------+--------------+------------------+----------------------------------------+
1 row in set (0.00 sec)

mysql> show slave status\G;
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 172.23.252.98
                  Master_User: repluser
                  Master_Port: 3312
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000006
          Read_Master_Log_Pos: 419
               Relay_Log_File: 5dfbef024732-relay-bin.000003
                Relay_Log_Pos: 632
        Relay_Master_Log_File: mysql-bin.000006
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB:
          Replicate_Ignore_DB:
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table:
  Replicate_Wild_Ignore_Table:
                   Last_Errno: 0
                   Last_Error:
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 419
              Relay_Log_Space: 846
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error:
               Last_SQL_Errno: 0
               Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
             Master_Server_Id: 1403311
                  Master_UUID: cd2eaa0a-7a59-11ec-b3b4-0242ac110002
             Master_Info_File: /var/lib/mysql/master.info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind:
      Last_IO_Error_Timestamp:
     Last_SQL_Error_Timestamp:
               Master_SSL_Crl:
           Master_SSL_Crlpath:
           Retrieved_Gtid_Set: cd2eaa0a-7a59-11ec-b3b4-0242ac110002:1
            Executed_Gtid_Set: cd2eaa0a-7a59-11ec-b3b4-0242ac110002:1
                Auto_Position: 1
         Replicate_Rewrite_DB:
                 Channel_Name:
           Master_TLS_Version:
1 row in set (0.00 sec)

Insert data in the master.order table:

mysql> insert into t_order values(4,"V");

Found that the data has been synchronized to the slave:

mysql> select * from order.t_order;
+------+------+
| id   | name |
+------+------+
|    4 | V    |
+------+------+
3 rows in set (0.00 sec)

Stop the slave first, and then insert data into the master.order table:

mysql> insert into t_order values(5,"X");

Then start the slave, and find that the data has been automatically synchronized:

mysql> stop slave;
Query OK, 0 rows affected (0.01 sec)

mysql> select * from order.t_order;
+------+------+
| id   | name |
+------+------+
|    4 | V    |
+------+------+
3 rows in set (0.00 sec)

mysql> start slave;
Query OK, 0 rows affected (0.02 sec)

mysql> select * from order.t_order;
+------+------+
| id   | name |
+------+------+
|    4 | V    |
|    5 | X    |
+------+------+
4 rows in set (0.00 sec)

Problems encountered

Show slave status on the slave server:

Fatal error: The slave I/O thread stops because master and slave have equal MySQL server UUIDs; these UUIDs must be different for replication to work.

First check whether the server_id of the master and slave are consistent. If they are consistent, modify the server_id in the my.cnf file. Field:

mysql> show variables like 'server_id';

Then check whether the uuid of master and slave are consistent:

mysql> show variables like '%uuid%';

If the uuid is consistent, modify the auto.cnf file in the data directory, copy the entire data directory, and replace the auto.cnf file I also copied it, and it records the uuid of the database. The uuid of each library should be different.

Recommended learning: mysql video tutorial

The above is the detailed content of MySQL replication instance detailed explanation of GTID replication. For more information, please follow other related articles on the PHP Chinese website!