search
HomeDatabaseMysql TutorialMySQL字符集和copy_and_convert_MySQL

bitsCN.com

MySQL字符集和copy_and_convert

 

关于copy_and_convert

    在对MySQL做业务压力测试的时候,我们在perf结果中发现 copy_and_convert 是一个耗费cpu的操作。这个函数的意思,就是在字符集之间做内容转换。

    如果源和目标的字符集相同,就可以直接用memcpy,这显然比做字符集转换(按字节或字长拷贝更快,和节省cpu)

 当整个系统是CPU瓶颈时,我们希望能够减少这种cpu消耗。

     

 一次查询涉及的拷贝

    如果我们执行一个简单的select语句,会涉及到两部分的内容:metadata和data。MySQL的返回包是能够“自解析”的,也就是说不仅有内容,还有描述信息。这些描述信息我们称为metadata。

    在将这两部分信息返回给客户端的时候,就涉及到字符串拷贝,如果源字符集和目标字符集不同,就需要作转换。

 

字符集里面可以定义的部分

1、  表的字符集

    比如我们创建如下的表    

 

CREATE TABLE `t` (

  `c` int(11) DEFAULT NULL,

  `d` varchar(50) DEFAULT NULL

) ENGINE=InnoDB DEFAULT CHARSET=gbk;

    这就定义了这个表的字符集,要求存入数据按照gbk格式存储。在我们查询的时候,从表中读出的数据,就是我们上文说到的“源字符集”,在这里是gbk.

比如

Insert t values(2012, ‘南方很冷’);

 

2、连接字符集

    在连接的时候我们可以在mysql客户端指定参数 mysql -default-character-set=gbk ,表示我们接收的内容希望是gbk编码,这个就是上文说的“目标字符集”。

    一般我们会被建议:连接字符集和表字符集相同。

    有的文章说防止乱码—-乱码倒是不会,实际的原因就是在内容拷贝时避免字符集转换计算。

 

 本文要说的“但是”

 

但是即使这么做,在压力测试(或者你gdb)时,还是会发现会调用到 copy_and_convert。

     

原因有二:

    首先,metadata部分,是固定的使用utf8,这个值可以从show variables like ‘character_set_system’;的结果看到,hardcode,配置无法修改。

    其次,虽然我们定义表字符集是gbk, 但是整型字段是是Latin1存储,

    #define my_charset_numeric      my_charset_latin1,配置无法修改。

 因此在我们上面的设置中,一次查询,在传回metadata的时候需要utf8->gbk, 返回结果中c这个字段需要latin1->gbk.

 

问题和建议

从上面分析知道,这存在的问题就是,无论用户怎么设置,都无法完全避免这种转换。

 由于数字类型转成字符串后,使用什么字符集都占用一样的空间。

 

1)       在代码上最简单的修改可以如下:

将 #define my_charset_numeric      my_charset_latin1 改成

#define my_charset_numeric      my_charset_utf8_general_ci

 这样跟metadata部分相同,然后建表和客户端都是使用utf-8。

 

 2)      更灵活一点是把character_set_system改成可配置,然后整型存储按照用户定义的字符集。

 

bitsCN.com
Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
How does MySQL differ from SQLite?How does MySQL differ from SQLite?Apr 24, 2025 am 12:12 AM

The main difference between MySQL and SQLite is the design concept and usage scenarios: 1. MySQL is suitable for large applications and enterprise-level solutions, supporting high performance and high concurrency; 2. SQLite is suitable for mobile applications and desktop software, lightweight and easy to embed.

What are indexes in MySQL, and how do they improve performance?What are indexes in MySQL, and how do they improve performance?Apr 24, 2025 am 12:09 AM

Indexes in MySQL are an ordered structure of one or more columns in a database table, used to speed up data retrieval. 1) Indexes improve query speed by reducing the amount of scanned data. 2) B-Tree index uses a balanced tree structure, which is suitable for range query and sorting. 3) Use CREATEINDEX statements to create indexes, such as CREATEINDEXidx_customer_idONorders(customer_id). 4) Composite indexes can optimize multi-column queries, such as CREATEINDEXidx_customer_orderONorders(customer_id,order_date). 5) Use EXPLAIN to analyze query plans and avoid

Explain how to use transactions in MySQL to ensure data consistency.Explain how to use transactions in MySQL to ensure data consistency.Apr 24, 2025 am 12:09 AM

Using transactions in MySQL ensures data consistency. 1) Start the transaction through STARTTRANSACTION, and then execute SQL operations and submit it with COMMIT or ROLLBACK. 2) Use SAVEPOINT to set a save point to allow partial rollback. 3) Performance optimization suggestions include shortening transaction time, avoiding large-scale queries and using isolation levels reasonably.

In what scenarios might you choose PostgreSQL over MySQL?In what scenarios might you choose PostgreSQL over MySQL?Apr 24, 2025 am 12:07 AM

Scenarios where PostgreSQL is chosen instead of MySQL include: 1) complex queries and advanced SQL functions, 2) strict data integrity and ACID compliance, 3) advanced spatial functions are required, and 4) high performance is required when processing large data sets. PostgreSQL performs well in these aspects and is suitable for projects that require complex data processing and high data integrity.

How can you secure a MySQL database?How can you secure a MySQL database?Apr 24, 2025 am 12:04 AM

The security of MySQL database can be achieved through the following measures: 1. User permission management: Strictly control access rights through CREATEUSER and GRANT commands. 2. Encrypted transmission: Configure SSL/TLS to ensure data transmission security. 3. Database backup and recovery: Use mysqldump or mysqlpump to regularly backup data. 4. Advanced security policy: Use a firewall to restrict access and enable audit logging operations. 5. Performance optimization and best practices: Take into account both safety and performance through indexing and query optimization and regular maintenance.

What are some tools you can use to monitor MySQL performance?What are some tools you can use to monitor MySQL performance?Apr 23, 2025 am 12:21 AM

How to effectively monitor MySQL performance? Use tools such as mysqladmin, SHOWGLOBALSTATUS, PerconaMonitoring and Management (PMM), and MySQL EnterpriseMonitor. 1. Use mysqladmin to view the number of connections. 2. Use SHOWGLOBALSTATUS to view the query number. 3.PMM provides detailed performance data and graphical interface. 4.MySQLEnterpriseMonitor provides rich monitoring functions and alarm mechanisms.

How does MySQL differ from SQL Server?How does MySQL differ from SQL Server?Apr 23, 2025 am 12:20 AM

The difference between MySQL and SQLServer is: 1) MySQL is open source and suitable for web and embedded systems, 2) SQLServer is a commercial product of Microsoft and is suitable for enterprise-level applications. There are significant differences between the two in storage engine, performance optimization and application scenarios. When choosing, you need to consider project size and future scalability.

In what scenarios might you choose SQL Server over MySQL?In what scenarios might you choose SQL Server over MySQL?Apr 23, 2025 am 12:20 AM

In enterprise-level application scenarios that require high availability, advanced security and good integration, SQLServer should be chosen instead of MySQL. 1) SQLServer provides enterprise-level features such as high availability and advanced security. 2) It is closely integrated with Microsoft ecosystems such as VisualStudio and PowerBI. 3) SQLServer performs excellent in performance optimization and supports memory-optimized tables and column storage indexes.

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

Atom editor mac version download

Atom editor mac version download

The most popular open source editor

MinGW - Minimalist GNU for Windows

MinGW - Minimalist GNU for Windows

This project is in the process of being migrated to osdn.net/projects/mingw, you can continue to follow us there. MinGW: A native Windows port of the GNU Compiler Collection (GCC), freely distributable import libraries and header files for building native Windows applications; includes extensions to the MSVC runtime to support C99 functionality. All MinGW software can run on 64-bit Windows platforms.

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

WebStorm Mac version

WebStorm Mac version

Useful JavaScript development tools