search
HomeDatabaseMysql Tutorial利用MySQL日志模拟数据变化轨迹

利用MySQL日志模拟数据变化轨迹

Jun 07, 2016 pm 04:34 PM
mysqluseVarietydatalogsimulationtrajectory

题记 先吐槽~~,这周末给屌丝哥(@应元)给废了。 有这么一个需求,希望通过binlog去恢复数据库某个时间段内数据的变化情况。 正文: 先说一下为什么用binlog可以做这么一件事情。 由于我们的binlog采用ROW模式,那么binlog会记录每一条数据所有列的变化信息

题记

先吐槽~~,这周末给屌丝哥(@应元)给废了。

有这么一个需求,希望通过binlog去恢复数据库某个时间段内数据的变化情况。

正文:

先说一下为什么用binlog可以做这么一件事情。

由于我们的binlog采用ROW模式,那么binlog会记录每一条数据所有列的变化信息,这些信息,我们就可以认为是一个数据源。

首先,我们先看一下binlog,通过命令行

mysqlbinlog –no-defaults -v –start-datetime=”2012-10-01 00:00:00″? –stop-datetime=”2012-10-1 02:00:00″ mysql-bin.000001 > tmp.log

去解析binlog。INSERT、UPDATE、DELETE三种操作如下图:

图中的@1、@2就是表示表a的列名,等号后面的信息就是该列的值。

我们打算通过拼装这些信息,将所有的操作都转换成INSERT操作,重新插入到数据库中,这样就可以看到一个数据的变化轨迹。

当然,我们需要注意一点,将表结构中的主键替换成普通索引,将唯一约束去除,保证每一条拼装出来的sql都能顺利被执行。

对于INSERT和DELETE两个操作,其数据项是唯一的,而UPDATE则有两部分。由于是顺序操作,所以我们需要的是UPDATE中SET之后的部分,即变化后的数据。

我们看一下列子:

首先我们有一个表a,结构如下:

然后我们对其做了一些操作,如下:

假设这些操作的时间在2012-11-25 16:20:00?至 2012-11-25 16:21:00内操作。在之后的时间内也被操作过。

现在我们就想看到在2012-11-25 16:20:00?至 2012-11-25 16:21:00内的操作内容。

通过该思路的方法,恢复后的数据库内容如下:

这样我们就可以观察到表a中id=1的num列的变化过程。

经典场景:商品减库存。

最后,方法还有不足之处:

如上图中红色方框内这两条数据,其实应该表示一个是UPDATE之后的结果,一个是DELETE的结果。我们在考虑时候对表结构进行变更,增加新的一列,表示是什么操作引起数据变化,这样就更加直观的看到数据变化的轨迹了。

PS:下篇文章我们会给出在实现过程中碰到的问题以及实现工具。

Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
MySQL: An Introduction to the World's Most Popular DatabaseMySQL: An Introduction to the World's Most Popular DatabaseApr 12, 2025 am 12:18 AM

MySQL is an open source relational database management system, mainly used to store and retrieve data quickly and reliably. Its working principle includes client requests, query resolution, execution of queries and return results. Examples of usage include creating tables, inserting and querying data, and advanced features such as JOIN operations. Common errors involve SQL syntax, data types, and permissions, and optimization suggestions include the use of indexes, optimized queries, and partitioning of tables.

The Importance of MySQL: Data Storage and ManagementThe Importance of MySQL: Data Storage and ManagementApr 12, 2025 am 12:18 AM

MySQL is an open source relational database management system suitable for data storage, management, query and security. 1. It supports a variety of operating systems and is widely used in Web applications and other fields. 2. Through the client-server architecture and different storage engines, MySQL processes data efficiently. 3. Basic usage includes creating databases and tables, inserting, querying and updating data. 4. Advanced usage involves complex queries and stored procedures. 5. Common errors can be debugged through the EXPLAIN statement. 6. Performance optimization includes the rational use of indexes and optimized query statements.

Why Use MySQL? Benefits and AdvantagesWhy Use MySQL? Benefits and AdvantagesApr 12, 2025 am 12:17 AM

MySQL is chosen for its performance, reliability, ease of use, and community support. 1.MySQL provides efficient data storage and retrieval functions, supporting multiple data types and advanced query operations. 2. Adopt client-server architecture and multiple storage engines to support transaction and query optimization. 3. Easy to use, supports a variety of operating systems and programming languages. 4. Have strong community support and provide rich resources and solutions.

Describe InnoDB locking mechanisms (shared locks, exclusive locks, intention locks, record locks, gap locks, next-key locks).Describe InnoDB locking mechanisms (shared locks, exclusive locks, intention locks, record locks, gap locks, next-key locks).Apr 12, 2025 am 12:16 AM

InnoDB's lock mechanisms include shared locks, exclusive locks, intention locks, record locks, gap locks and next key locks. 1. Shared lock allows transactions to read data without preventing other transactions from reading. 2. Exclusive lock prevents other transactions from reading and modifying data. 3. Intention lock optimizes lock efficiency. 4. Record lock lock index record. 5. Gap lock locks index recording gap. 6. The next key lock is a combination of record lock and gap lock to ensure data consistency.

What are common causes of poor MySQL query performance?What are common causes of poor MySQL query performance?Apr 12, 2025 am 12:11 AM

The main reasons for poor MySQL query performance include not using indexes, wrong execution plan selection by the query optimizer, unreasonable table design, excessive data volume and lock competition. 1. No index causes slow querying, and adding indexes can significantly improve performance. 2. Use the EXPLAIN command to analyze the query plan and find out the optimizer error. 3. Reconstructing the table structure and optimizing JOIN conditions can improve table design problems. 4. When the data volume is large, partitioning and table division strategies are adopted. 5. In a high concurrency environment, optimizing transactions and locking strategies can reduce lock competition.

When should you use a composite index versus multiple single-column indexes?When should you use a composite index versus multiple single-column indexes?Apr 11, 2025 am 12:06 AM

In database optimization, indexing strategies should be selected according to query requirements: 1. When the query involves multiple columns and the order of conditions is fixed, use composite indexes; 2. When the query involves multiple columns but the order of conditions is not fixed, use multiple single-column indexes. Composite indexes are suitable for optimizing multi-column queries, while single-column indexes are suitable for single-column queries.

How to identify and optimize slow queries in MySQL? (slow query log, performance_schema)How to identify and optimize slow queries in MySQL? (slow query log, performance_schema)Apr 10, 2025 am 09:36 AM

To optimize MySQL slow query, slowquerylog and performance_schema need to be used: 1. Enable slowquerylog and set thresholds to record slow query; 2. Use performance_schema to analyze query execution details, find out performance bottlenecks and optimize.

MySQL and SQL: Essential Skills for DevelopersMySQL and SQL: Essential Skills for DevelopersApr 10, 2025 am 09:30 AM

MySQL and SQL are essential skills for developers. 1.MySQL is an open source relational database management system, and SQL is the standard language used to manage and operate databases. 2.MySQL supports multiple storage engines through efficient data storage and retrieval functions, and SQL completes complex data operations through simple statements. 3. Examples of usage include basic queries and advanced queries, such as filtering and sorting by condition. 4. Common errors include syntax errors and performance issues, which can be optimized by checking SQL statements and using EXPLAIN commands. 5. Performance optimization techniques include using indexes, avoiding full table scanning, optimizing JOIN operations and improving code readability.

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. How to Fix Audio if You Can't Hear Anyone
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
WWE 2K25: How To Unlock Everything In MyRise
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Dreamweaver Mac version

Dreamweaver Mac version

Visual web development tools

MantisBT

MantisBT

Mantis is an easy-to-deploy web-based defect tracking tool designed to aid in product defect tracking. It requires PHP, MySQL and a web server. Check out our demo and hosting services.

SAP NetWeaver Server Adapter for Eclipse

SAP NetWeaver Server Adapter for Eclipse

Integrate Eclipse with SAP NetWeaver application server.

VSCode Windows 64-bit Download

VSCode Windows 64-bit Download

A free and powerful IDE editor launched by Microsoft

PhpStorm Mac version

PhpStorm Mac version

The latest (2018.2.1) professional PHP integrated development tool