search
HomeDatabaseMysql TutorialFailover with the MySQL Utilities: Part 2 – mysqlfailover_MySQL

In theprevious postof this series we saw how you could usemysqlrpladminto perform manual failover/switchover when GTID replication is enabled in MySQL 5.6. Now we will reviewmysqlfailover(version 1.4.3), another tool from the MySQL Utilities that can be used for automatic failover.

Summary

  • mysqlfailovercan perform automatic failover if MySQL 5.6′s GTID-replication is enabled.
  • All slaves must use--master-info-repository=TABLE.
  • The monitoring node is a single point of failure: don’t forget to monitor it!
  • Detection of errant transactions works well, but you have to use the--pedanticoption to make sure failover will never happen if there is an errant transaction.
  • There are a few limitations such as the inability to only fail over once, or excessive CPU utilization, but they are probably not showstoppers for most setups.

Setup

We will use the same setup as last time: one master and two slaves, all using GTID replication. We can see the topology usingmysqlfailoverwith thehealthcommand:

$ mysqlfailover --master=root@localhost:13001 --discover-slaves-login=root health[...]MySQL Replication Failover UtilityFailover Mode = auto Next Interval = Tue Jul1 10:01:22 2014Master Information------------------Binary Log File PositionBinlog_Do_DBBinlog_Ignore_DBmysql-bin.000003700GTID Executed Seta9a396c6-00f3-11e4-8e66-9cebe8067a3f:1-3Replication Health Status+------------+--------+---------+--------+------------+---------+| host | port | role| state| gtid_mode| health|+------------+--------+---------+--------+------------+---------+| localhost| 13001| MASTER| UP | ON | OK|| localhost| 13002| SLAVE | UP | ON | OK|| localhost| 13003| SLAVE | UP | ON | OK|+------------+--------+---------+--------+------------+---------+

$mysqlfailover--master=root@localhost:13001--discover-slaves-login=roothealth

[...]

MySQLReplicationFailoverUtility

FailoverMode=auto    NextInterval=TueJul  110:01:222014

MasterInformation

------------------

BinaryLogFile  Position  Binlog_Do_DB  Binlog_Ignore_DB  

mysql-bin.000003  700                                      

GTIDExecutedSet

a9a396c6-00f3-11e4-8e66-9cebe8067a3f:1-3

ReplicationHealthStatus

+------------+--------+---------+--------+------------+---------+

|host      |port  |role    |state  |gtid_mode  |health  |

+------------+--------+---------+--------+------------+---------+

|localhost  |13001  |MASTER  |UP    |ON        |OK      |

|localhost  |13002  |SLAVE  |UP    |ON        |OK      |

|localhost  |13003  |SLAVE  |UP    |ON        |OK      |

+------------+--------+---------+--------+------------+---------+

Note that--master-info-repository=TABLEneeds to be configured on all slaves or the tool will exit with an error message:

2014-07-01 10:18:55 AM CRITICAL Failover requires --master-info-repository=TABLE for all slaves.ERROR: Failover requires --master-info-repository=TABLE for all slaves.

2014-07-0110:18:55AMCRITICALFailoverrequires--master-info-repository=TABLEforallslaves.

ERROR:Failoverrequires--master-info-repository=TABLEforallslaves.

Failover

You can use 2 commands to trigger automatic failover:

  • auto: the tool tries to find a candidate in the list of servers specified with--candidates, and if no good server is found in this list, it will look at the other slaves to see if one can be a good candidate. This is the default command
  • elect: same asauto, but if no good candidate is found in the list of candidates, other slaves will not be checked and the tool will exit with an error.

Let’s start the tool withauto:

$ mysqlfailover --master=root@localhost:13001 --discover-slaves-login=root auto

$mysqlfailover--master=root@localhost:13001--discover-slaves-login=rootauto

The monitoring console is visible and is refreshed every--intervalseconds (default: 15). Its output is similar to what you get when using thehealthcommand.

Then let’s kill -9 the master to see what happens once the master is detected as down:

Failed to reconnect to the master after 3 attemps.Failover starting in 'auto' mode...# Candidate slave localhost:13002 will become the new master.# Checking slaves status (before failover).# Preparing candidate for failover.# Creating replication user if it does not exist.# Stopping slaves.# Performing STOP on all slaves.# Switching slaves to new master.# Disconnecting new master as slave.# Starting slaves.# Performing START on all slaves.# Checking slaves for errors.# Failover complete.# Discovering slaves for master at localhost:13002Failover console will restart in 5 seconds.MySQL Replication Failover UtilityFailover Mode = auto Next Interval = Tue Jul1 10:59:47 2014Master Information------------------Binary Log File PositionBinlog_Do_DBBinlog_Ignore_DBmysql-bin.000005191GTID Executed Seta9a396c6-00f3-11e4-8e66-9cebe8067a3f:1-3Replication Health Status+------------+--------+---------+--------+------------+---------+| host | port | role| state| gtid_mode| health|+------------+--------+---------+--------+------------+---------+| localhost| 13002| MASTER| UP | ON | OK|| localhost| 13003| SLAVE | UP | ON | OK|+------------+--------+---------+--------+------------+---------+

Failedtoreconnecttothemasterafter3attemps.

Failoverstartingin'auto'mode...

# Candidate slave localhost:13002 will become the new master.

# Checking slaves status (before failover).

# Preparing candidate for failover.

# Creating replication user if it does not exist.

# Stopping slaves.

# Performing STOP on all slaves.

# Switching slaves to new master.

# Disconnecting new master as slave.

# Starting slaves.

# Performing START on all slaves.

# Checking slaves for errors.

# Failover complete.

# Discovering slaves for master at localhost:13002

Failoverconsolewillrestartin5seconds.

MySQLReplicationFailoverUtility

FailoverMode=auto    NextInterval=TueJul  110:59:472014

MasterInformation

------------------

BinaryLogFile  Position  Binlog_Do_DB  Binlog_Ignore_DB  

mysql-bin.000005  191                                      

GTIDExecutedSet

a9a396c6-00f3-11e4-8e66-9cebe8067a3f:1-3

ReplicationHealthStatus

+------------+--------+---------+--------+------------+---------+

|host      |port  |role    |state  |gtid_mode  |health  |

+------------+--------+---------+--------+------------+---------+

|localhost  |13002  |MASTER  |UP    |ON        |OK      |

|localhost  |13003  |SLAVE  |UP    |ON        |OK      |

+------------+--------+---------+--------+------------+---------+

Looks good! The tool is then ready to fail over to another slave if the new master becomes unavailable.

You can also run custom scripts at several points of execution with the--exec-before,--exec-after,--exec-fail-check,--exec-post-failoveroptions.

However it would be great to have a--failover-and-exitoption to avoid flapping: the tool would detect master failure, promote one of the slaves, reconfigure replication and then exit (this is what MHA does for instance).

Tool registration

When the tool is started, it registers itself on the master by writing a few things in the specific table:

mysql> SELECT * FROM mysql.failover_console;+-----------+-------+| host| port|+-----------+-------+| localhost | 13001 |+-----------+-------+

mysql>SELECT*FROMmysql.failover_console;

+-----------+-------+

|host      |port  |

+-----------+-------+

|localhost|13001|

+-----------+-------+

This is nice as it avoids that you start several instances ofmysqlfailoverto monitor the same master. If we try, this is what we get:

$ mysqlfailover --master=root@localhost:13001 --discover-slaves-login=root auto[...]Multiple instances of failover console found for master localhost:13001.If this is an error, restart the console with --force.Failover mode changed to 'FAIL' for this instance.Console will start in 10 seconds..........starting Console.

$mysqlfailover--master=root@localhost:13001--discover-slaves-login=rootauto

[...]

Multipleinstancesoffailoverconsolefoundformasterlocalhost:13001.

Ifthisisanerror,restarttheconsolewith--force.

Failovermodechangedto'FAIL'forthisinstance.

Consolewillstartin10seconds..........startingConsole.

With thefailcommand,mysqlfailoverwill monitor replication health and exit in the case of a master failure, without actually performing failover.

Running in the background

In all previous examples,mysqlfailoverwas running in the foreground. This is very good for demo, but in a production environment you are likely to prefer running it in the background. This can be done with the--daemonoption:

$ mysqlfailover --master=root@localhost:13001 --discover-slaves-login=root auto --daemon=start --log=/var/log/mysqlfailover.log

$mysqlfailover--master=root@localhost:13001--discover-slaves-login=rootauto--daemon=start--log=/var/log/mysqlfailover.log

and it can be stopped with:

$ mysqlfailover --daemon=stop

$mysqlfailover--daemon=stop

Errant transactions

If we create an errant transaction on one of the slaves, it will be detected:

MySQL Replication Failover UtilityFailover Mode = auto Next Interval = Tue Jul1 16:29:44 2014[...]WARNING: Errant transaction(s) found on slave(s).Replication Health Status[...]

MySQLReplicationFailoverUtility

FailoverMode=auto    NextInterval=TueJul  116:29:442014

[...]

WARNING:Erranttransaction(s)foundonslave(s).

ReplicationHealthStatus

[...]

However this does not prevent failover from occurring! You have to use--pedantic:

$ mysqlfailover --master=root@localhost:13001 --discover-slaves-login=root --pedantic auto[...]# WARNING: Errant transaction(s) found on slave(s).#- For slave 'localhost@13003': db906eee-012d-11e4-8fe1-9cebe8067a3f:12014-07-01 16:44:49 PM CRITICAL Errant transaction(s) found on slave(s). Note: If you want to ignore this issue, please do not use the --pedantic option.ERROR: Errant transaction(s) found on slave(s). Note: If you want to ignore this issue, please do not use the --pedantic option.

$mysqlfailover--master=root@localhost:13001--discover-slaves-login=root--pedanticauto

[...]

# WARNING: Errant transaction(s) found on slave(s).

#  - For slave 'localhost@13003': db906eee-012d-11e4-8fe1-9cebe8067a3f:1

2014-07-0116:44:49PMCRITICALErranttransaction(s)foundonslave(s).Note:Ifyouwanttoignorethisissue,pleasedonotusethe--pedanticoption.

ERROR:Erranttransaction(s)foundonslave(s).Note:Ifyouwanttoignorethisissue,pleasedonotusethe--pedanticoption.

Limitations

  • Like formysqlrpladmin, the slave election process is not very sophisticated and it cannot be tuned.
  • The server on whichmysqlfailoveris running is a single point of failure.
  • Excessive CPU utilization: once it is running,mysqlfailoverhogs one core. This is quite surprising.

Conclusion

mysqlfailoveris a good tool to automate failover in clusters using GTID replication. It is flexible and looks reliable. Its main drawback is that there is no easy way to make it highly available itself: ifmysqlfailovercrashes, you will have to manually restart it.

Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
How does MySQL's licensing compare to other database systems?How does MySQL's licensing compare to other database systems?Apr 25, 2025 am 12:26 AM

MySQL uses a GPL license. 1) The GPL license allows the free use, modification and distribution of MySQL, but the modified distribution must comply with GPL. 2) Commercial licenses can avoid public modifications and are suitable for commercial applications that require confidentiality.

When would you choose InnoDB over MyISAM, and vice versa?When would you choose InnoDB over MyISAM, and vice versa?Apr 25, 2025 am 12:22 AM

The situations when choosing InnoDB instead of MyISAM include: 1) transaction support, 2) high concurrency environment, 3) high data consistency; conversely, the situation when choosing MyISAM includes: 1) mainly read operations, 2) no transaction support is required. InnoDB is suitable for applications that require high data consistency and transaction processing, such as e-commerce platforms, while MyISAM is suitable for read-intensive and transaction-free applications such as blog systems.

Explain the purpose of foreign keys in MySQL.Explain the purpose of foreign keys in MySQL.Apr 25, 2025 am 12:17 AM

In MySQL, the function of foreign keys is to establish the relationship between tables and ensure the consistency and integrity of the data. Foreign keys maintain the effectiveness of data through reference integrity checks and cascading operations. Pay attention to performance optimization and avoid common errors when using them.

What are the different types of indexes in MySQL?What are the different types of indexes in MySQL?Apr 25, 2025 am 12:12 AM

There are four main index types in MySQL: B-Tree index, hash index, full-text index and spatial index. 1.B-Tree index is suitable for range query, sorting and grouping, and is suitable for creation on the name column of the employees table. 2. Hash index is suitable for equivalent queries and is suitable for creation on the id column of the hash_table table of the MEMORY storage engine. 3. Full text index is used for text search, suitable for creation on the content column of the articles table. 4. Spatial index is used for geospatial query, suitable for creation on geom columns of locations table.

How do you create an index in MySQL?How do you create an index in MySQL?Apr 25, 2025 am 12:06 AM

TocreateanindexinMySQL,usetheCREATEINDEXstatement.1)Forasinglecolumn,use"CREATEINDEXidx_lastnameONemployees(lastname);"2)Foracompositeindex,use"CREATEINDEXidx_nameONemployees(lastname,firstname);"3)Forauniqueindex,use"CREATEU

How does MySQL differ from SQLite?How does MySQL differ from SQLite?Apr 24, 2025 am 12:12 AM

The main difference between MySQL and SQLite is the design concept and usage scenarios: 1. MySQL is suitable for large applications and enterprise-level solutions, supporting high performance and high concurrency; 2. SQLite is suitable for mobile applications and desktop software, lightweight and easy to embed.

What are indexes in MySQL, and how do they improve performance?What are indexes in MySQL, and how do they improve performance?Apr 24, 2025 am 12:09 AM

Indexes in MySQL are an ordered structure of one or more columns in a database table, used to speed up data retrieval. 1) Indexes improve query speed by reducing the amount of scanned data. 2) B-Tree index uses a balanced tree structure, which is suitable for range query and sorting. 3) Use CREATEINDEX statements to create indexes, such as CREATEINDEXidx_customer_idONorders(customer_id). 4) Composite indexes can optimize multi-column queries, such as CREATEINDEXidx_customer_orderONorders(customer_id,order_date). 5) Use EXPLAIN to analyze query plans and avoid

Explain how to use transactions in MySQL to ensure data consistency.Explain how to use transactions in MySQL to ensure data consistency.Apr 24, 2025 am 12:09 AM

Using transactions in MySQL ensures data consistency. 1) Start the transaction through STARTTRANSACTION, and then execute SQL operations and submit it with COMMIT or ROLLBACK. 2) Use SAVEPOINT to set a save point to allow partial rollback. 3) Performance optimization suggestions include shortening transaction time, avoiding large-scale queries and using isolation levels reasonably.

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

PhpStorm Mac version

PhpStorm Mac version

The latest (2018.2.1) professional PHP integrated development tool

Atom editor mac version download

Atom editor mac version download

The most popular open source editor

WebStorm Mac version

WebStorm Mac version

Useful JavaScript development tools

SecLists

SecLists

SecLists is the ultimate security tester's companion. It is a collection of various types of lists that are frequently used during security assessments, all in one place. SecLists helps make security testing more efficient and productive by conveniently providing all the lists a security tester might need. List types include usernames, passwords, URLs, fuzzing payloads, sensitive data patterns, web shells, and more. The tester can simply pull this repository onto a new test machine and he will have access to every type of list he needs.

EditPlus Chinese cracked version

EditPlus Chinese cracked version

Small size, syntax highlighting, does not support code prompt function