阅读xtrabackup代码的一点笔记-Mysql Tutorial-php.cn

Home

Database

Mysql Tutorial

阅读xtrabackup代码的一点笔记

WBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWB

Jun 01, 2016 pm 01:16 PM

blogtechnologyModel

xtrabackup binary最重要的两个过程是backup和prepare，对应的函数分别是xtrabackup_backup_func()和xtrabackup_prepare_func()，这里做一些阅读代码时的笔记。

xtrabackup backup的线程模型：

1. 一个log拷贝线程；

2. n个ibd文件拷贝线程；

3. 一个io监控线程；

4. 通过suspend_start/suspend_end文件来标注是否启动终止线程；

typedef struct {

datafiles_iter_t *it;

uint num;

uint *count;

os_ib_mutex_t count_mutex;

os_thread_id_t id;

}data_thread_ctxt_t;

数据线程上下文切换工作目录；

116 /** Set if InnoDB must operate in read-only mode. We don't do any

117 recovery and open all tables in RO mode instead of RW mode. We don't

118 sync the max trx id to disk either. */

xb_set_innodb_read_only() 将innodb设成只读模式

srv_backup_mode=TRUE; 将innodb设成backup模式；

设置innodb的一系列参数

innodb_init_param()

xb_normalize_init_values(void)

修改srv_unix_file_flush_method

根据bp大小，调整srv_max_n_threads参数

1017 /*********************************************************************//**

1018 Initializes the synchronization primitives, memory system, and the thread

1019 local storage. */

srv_general_init()

ut_crc32_init()

xb_filters_init()

2567 /************************************************************************

2568 Initializes the I/O and tablespace cache subsystems. */

xb_fil_io_init(void)

838 /******************************************************//**

839 Initializes the log. */

log_init(void)

585 /*********************************************************************//**

586 Creates the lock system at database start. */

lock_sys_create()

open_or_create_log_file

创建xtrabackup_extra_lsndir/extrabackup_traget_dir

表空间memory cache

fil_system_t* f_system = fil_system;

recv_find_max_checkpoint(&max_cp_group, &max_cp_field)

log_group_read_checkpoint_info(max_cp_group, max_cp_field)

checkpoint_lsn_start/checkpoint_no_start

确认一致的checkpoint状态；

创建XB_LOG_FILENAME文件，写入文件头信息；

创建io_watching_thread;

从checkpoint位置开始copy log文件；

xtrabackup_copy_logfile(checkpoint_lsn_start, FALSE)

log_copying/log_copying_stop

创建日志copy线程

os_thread_create(log_copying_thread, NULL, &log_copying_thread_id);

2591 /****************************************************************************

2592 Populates the tablespace memory cache by scanning for and opening data files.

2593 @returns DB_SUCCESS or error code.*/

xb_load_tablespaces()

挂起，等待XB_FN_SUSPENDED_AT_START文件被删除

xtrabackup_suspend

xb_page_bitmap_init()

根据xtrabackup_parallel设置，创建data_copy_thread_func线程

等待所有data_copy_thread_func线程退出

挂起，等待XB_FN_SUSPENDED_AT_END文件被删除

xtrabackup_suspend

读取最新的checkpoint，记录在metadata的to_lsn字段；

通过设置log_copying=FALSE && set log_copying_stop，停止log_copying_thread；

创建一个文件XB_FN_LOG_COPIED，通知外部脚本，log_copying_thread已经结束；

写metadata；prepare_func

1. 切换到xtrabackup_real_target_dir

2. 读取XTRABACKUP_METADATA_FILENAME，获取原信息；

metadata_type

3. xtrabackup_init_temp_log()

4. innodb_init_param()

2670 /************************************************************************

2671 Initialize the tablespace memory cache and populate it by scanning for and

2672 opening data files.

2673 @returns DB_SUCCESS or error code.*/

xb_data_files_init()

应用增量到全量

xtrabackup_apply_deltas()

重设innodb初始化参数

innodb_init_param()

innodb_init()遍历文件

mtr_start -> mtr_commit

trx_sys_print_mysql_binlog_offset()

将binlog位置信息输出到 xtrabackup_binlog_pos_innodb文件中

xtrabackup_close_temp_log(TRUE)

输出记录metadata_log

backup

$./xtrabackup_56 --defaults-file=/u01/my3928/my.cnf --backup --target_dir=/u01/xianlin.lh/backup_dir/

./xtrabackup_56 version 2.1.8 for MySQL server 5.6.15 Linux (x86_64) (revision id: undefined)

xtrabackup: uses posix_fadvise().

xtrabackup: cd to /u01/my3928/data

xtrabackup: using the following InnoDB configuration:

xtrabackup: innodb_data_home_dir = /u01/my3928/data

xtrabackup: innodb_data_file_path = ibdata1:4G;ibdata2:16M:autoextend

xtrabackup: innodb_log_group_home_dir = /u01/my3928/data

xtrabackup: innodb_log_files_in_group = 4

xtrabackup: innodb_log_file_size = 1073741824

2014-05-05 17:29:35 2ac06bc4a2c0 InnoDB: Using Linux native AIO

xtrabackup: using O_DIRECT

>> log scanned up to (1451746590)

[01] Copying /u01/my3928/data/ibdata1 to /u01/xianlin.lh/backup_dir/ibdata1

>> log scanned up to (1451746590)

[01] ...done

[01] Copying /u01/my3928/data/ibdata2 to /u01/xianlin.lh/backup_dir/ibdata2

[01] ...done

[01] Copying ./test/t2.ibd to /u01/xianlin.lh/backup_dir/test/t2.ibd

[01] ...done

[01] Copying ./test/t4.ibd to /u01/xianlin.lh/backup_dir/test/t4.ibd

[01] ...done

[01] Copying ./test/t3.ibd to /u01/xianlin.lh/backup_dir/test/t3.ibd

[01] ...done

[01] Copying ./test/sbtest1.ibd to /u01/xianlin.lh/backup_dir/test/sbtest1.ibd

>> log scanned up to (1451746590)

[01] ...done

[01] Copying ./test/t1.ibd to /u01/xianlin.lh/backup_dir/test/t1.ibd

[01] ...done

[01] Copying ./mysql/innodb_index_stats.ibd to /u01/xianlin.lh/backup_dir/mysql/innodb_index_stats.ibd

[01] ...done

[01] Copying ./mysql/slave_worker_info.ibd to /u01/xianlin.lh/backup_dir/mysql/slave_worker_info.ibd

[01] ...done

[01] Copying ./mysql/innodb_table_stats.ibd to /u01/xianlin.lh/backup_dir/mysql/innodb_table_stats.ibd

[01] ...done

[01] Copying ./mysql/slave_relay_log_info.ibd to /u01/xianlin.lh/backup_dir/mysql/slave_relay_log_info.ibd

[01] ...done

[01] Copying ./mysql/slave_master_info.ibd to /u01/xianlin.lh/backup_dir/mysql/slave_master_info.ibd

[01] ...done

>> log scanned up to (1451746590)

xtrabackup: The latest check point (for incremental): '1451746590'

xtrabackup: Stopping log copying thread.

.>> log scanned up to (1451746590)

xtrabackup: Transaction log of lsn (1451746590) to (1451746590) was copied.

创建备份的流程

http://www.percona.com/doc/percona-xtrabackup/2.1/xtrabackup_bin/creating_a_backup.html

1 backup_type = full-backuped

2 from_lsn = 0

3 to_lsn = 1451746590

4 last_lsn = 1451746590

5 compact = 0prepare

$./xtrabackup_56 --defaults-file=/u01/my3928/my.cnf --prepare --target-dir=/u01/xianlin.lh/backup_dir/

./xtrabackup_56 version 2.1.8 for MySQL server 5.6.15 Linux (x86_64) (revision id: undefined)

xtrabackup: cd to /u01/xianlin.lh/backup_dir/

xtrabackup: This target seems to be not prepared yet.

xtrabackup: xtrabackup_logfile detected: size=2097152, start_lsn=(1451746590)

xtrabackup: using the following InnoDB configuration for recovery:

xtrabackup: innodb_data_home_dir = ./

xtrabackup: innodb_data_file_path = ibdata1:4G;ibdata2:16M:autoextend

xtrabackup: innodb_log_group_home_dir = ./

xtrabackup: innodb_log_files_in_group = 1

xtrabackup: innodb_log_file_size = 2097152

2014-05-05 18:38:32 2b7f2f5202c0 InnoDB: Using Linux native AIO

xtrabackup: using the following InnoDB configuration for recovery:

xtrabackup: innodb_data_home_dir = ./

xtrabackup: innodb_data_file_path = ibdata1:4G;ibdata2:16M:autoextend

xtrabackup: innodb_log_group_home_dir = ./

xtrabackup: innodb_log_files_in_group = 1

xtrabackup: innodb_log_file_size = 2097152

2014-05-05 18:38:32 2b7f2f5202c0 InnoDB: Using Linux native AIO

xtrabackup: Starting InnoDB instance for recovery.

xtrabackup: Using 104857600 bytes for buffer pool (set by --use-memory parameter)

InnoDB: The InnoDB memory heap is disabled

InnoDB: Mutexes and rw_locks use GCC atomic builtins

InnoDB: Compressed tables use zlib 1.2.3

InnoDB: Using Linux native AIO

InnoDB: Using CPU crc32 instructions

InnoDB: Initializing buffer pool, size = 100.0M

InnoDB: Completed initialization of buffer pool

InnoDB: Highest supported file format is Barracuda.

InnoDB: The log sequence numbers 552524932 and 552524932 in ibdata files do not match the log sequence number 1451746590 in the ib_logfiles!

InnoDB: Database was not shutdown normally!

InnoDB: Starting crash recovery.

InnoDB: Reading tablespace information from the .ibd files...

InnoDB: Restoring possible half-written data pages

InnoDB: from the doublewrite buffer...

InnoDB: Last MySQL binlog file position 0 26767310, file name mysql-bin.000023

InnoDB: 128 rollback segment(s) are active.

InnoDB: Waiting for purge to start

2014-05-05 18:38:33 2b7f49056700 InnoDB: Warning: table 'test/sbtest1'

InnoDB: in InnoDB data dictionary has unknown flags 50.

InnoDB: 5.6.15 started; log sequence number 1451746590

[notice (again)]

If you use binary log and don't use any hack of group commit,

the binary log position seems to be:

InnoDB: Last MySQL binlog file position 0 26767310, file name mysql-bin.000023

xtrabackup: starting shutdown with innodb_fast_shutdown = 1

InnoDB: FTS optimize thread exiting.

InnoDB: Starting shutdown...

InnoDB: Shutdown completed; log sequence number 1451747281

prepare的流程：

http://www.percona.com/doc/percona-xtrabackup/2.1/xtrabackup_bin/preparing_the_backup.html

1 backup_type = full-prepared

2 from_lsn = 0

3 to_lsn = 1451746590

4 last_lsn = 1451746590

5 compact = 0

Statement

The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

MySQL's Role: Databases in Web ApplicationsApr 17, 2025 am 12:23 AM

The main role of MySQL in web applications is to store and manage data. 1.MySQL efficiently processes user information, product catalogs, transaction records and other data. 2. Through SQL query, developers can extract information from the database to generate dynamic content. 3.MySQL works based on the client-server model to ensure acceptable query speed.

MySQL: Building Your First DatabaseApr 17, 2025 am 12:22 AM

The steps to build a MySQL database include: 1. Create a database and table, 2. Insert data, and 3. Conduct queries. First, use the CREATEDATABASE and CREATETABLE statements to create the database and table, then use the INSERTINTO statement to insert the data, and finally use the SELECT statement to query the data.

MySQL: A Beginner-Friendly Approach to Data StorageApr 17, 2025 am 12:21 AM

MySQL is suitable for beginners because it is easy to use and powerful. 1.MySQL is a relational database, and uses SQL for CRUD operations. 2. It is simple to install and requires the root user password to be configured. 3. Use INSERT, UPDATE, DELETE, and SELECT to perform data operations. 4. ORDERBY, WHERE and JOIN can be used for complex queries. 5. Debugging requires checking the syntax and use EXPLAIN to analyze the query. 6. Optimization suggestions include using indexes, choosing the right data type and good programming habits.

Is MySQL Beginner-Friendly? Assessing the Learning CurveApr 17, 2025 am 12:19 AM

MySQL is suitable for beginners because: 1) easy to install and configure, 2) rich learning resources, 3) intuitive SQL syntax, 4) powerful tool support. Nevertheless, beginners need to overcome challenges such as database design, query optimization, security management, and data backup.

Is SQL a Programming Language? Clarifying the TerminologyApr 17, 2025 am 12:17 AM

Yes,SQLisaprogramminglanguagespecializedfordatamanagement.1)It'sdeclarative,focusingonwhattoachieveratherthanhow.2)SQLisessentialforquerying,inserting,updating,anddeletingdatainrelationaldatabases.3)Whileuser-friendly,itrequiresoptimizationtoavoidper

Explain the ACID properties (Atomicity, Consistency, Isolation, Durability).Apr 16, 2025 am 12:20 AM

ACID attributes include atomicity, consistency, isolation and durability, and are the cornerstone of database design. 1. Atomicity ensures that the transaction is either completely successful or completely failed. 2. Consistency ensures that the database remains consistent before and after a transaction. 3. Isolation ensures that transactions do not interfere with each other. 4. Persistence ensures that data is permanently saved after transaction submission.

MySQL: Database Management System vs. Programming LanguageApr 16, 2025 am 12:19 AM

MySQL is not only a database management system (DBMS) but also closely related to programming languages. 1) As a DBMS, MySQL is used to store, organize and retrieve data, and optimizing indexes can improve query performance. 2) Combining SQL with programming languages, embedded in Python, using ORM tools such as SQLAlchemy can simplify operations. 3) Performance optimization includes indexing, querying, caching, library and table division and transaction management.

MySQL: Managing Data with SQL CommandsApr 16, 2025 am 12:19 AM

MySQL uses SQL commands to manage data. 1. Basic commands include SELECT, INSERT, UPDATE and DELETE. 2. Advanced usage involves JOIN, subquery and aggregate functions. 3. Common errors include syntax, logic and performance issues. 4. Optimization tips include using indexes, avoiding SELECT* and using LIMIT.

See all articles