搜尋
首頁資料庫mysql教程MySQL 5.7 supports the GB18030 Chinese Character Set_MySQL

My former boss at MySQL sent out a notice that MySQL 5.7.4 nowsupports theGB18030character set, thus responding to requests that have been appearing since2005. This is a big deal because the Chinese government demands GB18030 support, and because the older simplified-Chinese character sets (gbk and gb2312) have a much smaller repertoire (that is, they have too few characters). And this is real GB18030 support -- I can define columns and variables with CHARACTER SET GB18030. That's rare --Oracle 12candSQL Server 2012andPostsgreSQL 9.3can't do it. (They allow input from GB18030 clients but they convert it immediately to Unicode.) Among big-time DBMSs, until now,only DB2has treated GB18030 as a first-class character set.

Standard Adherence

We're talking about the current version of the standard, GB18030-2005 "IT Chinese coded character set", especially its description of 70,244 Chinese characters. I couldn't puzzle out the Chinese wording inthe official document, all I could do was use translate.google.comon some excerpts. But I've been told that the MySQL person who coded this feature is Chinese, so they'll have had better luck. What I could understand was what are the difficult characters, what are the requirements for a claim of support, and what the encoding should look like. From the coder's comments, it's clear that part was understood. I did not check whether there was adherence for non-mandatory parts, such as Tibetan script.

Conversions

The repertoire of GB18030 ought to be the same as the Unicode repertoire. So I took a list of every Unicode character, converted to GB18030, and converted back to Unicode. The result in every case was the same Unicode character that I'd started with. That's called "perfect round tripping". As I explained in an earlier blog post"The UTF-8 World Is Not Enough", storing Chinese characters with a Chinese character set has certain advantages. Well, now the biggest disadvantage has disappeared.

Hold on -- how is perfect round tripping possible, given that MySQLfrequently refers to Unicode 4.0, and some of the characters in GB18030-2005 are only in Unicode 4.1? Certainly that ought to be a problem according to theUnicode FAQandthis extract from Ken Lunde's book. But it turns out to be okay because MySQL doesn't actually disallow those characters -- it accepts encodings which are not assigned to characters. Of course I believe that MySQL should have upgraded the Unicode support first, and added GB18030 support later. But the best must not be an enemy of the good.

Also the conversions to and from gb2312 work fine, so I expect that eventually gb2312 will become obsolete. It's time for mainland Chinese users to consider switching over to gb18030 once MySQL 5.7 is GA.

Collations

The new character set comes with three collations: one trivial, one tremendous, one tsk, tsk.

The trivial collation is gb18030_bin. As always the bin stands for binary. I expect that as always this will be the most performant collation, and the only one that guarantees that no two characters will ever have the same weight.

The tremendous collation is gb18030_unicode_520_ci. The "unicode_520" part of the name really does mean that the collation table comes from"Unicode 5.2"and this is the first time that MySQL has taken to heart the maxim: what applies to the superset can apply to the subset. In fact all MySQL character sets should have Unicode collations, because all their characters are in Unicode. So to test this, I went through all the Unicode characters and their GB18030 equivalents, and compared their weights withWEIGHT_STRING:
WEIGHT_STRING(utf32_char COLLATE utf32_unicode_520_ci) to
WEIGHT_STRING(gb18030_char COLLATE gb18030_unicode_520_ci).
Every utf32 weight was exactly the same as the gb18030 weight.

The tsk, tsk collation is gb18030_chinese_ci.

The first bad thing is the suffix chinese_ci, which will make some people think that this collation is like gb2312_chinese_ci. (Such confusion has happened before for the general_ci suffix.) In fact there are thousands of differences between gb2312_chinese_ci and gb18030_chinese_ci. Here's an example.

mysql> CREATE TABLE t5	->(gb2312 CHAR CHARACTER SET gb2312 COLLATE gb2312_chinese_ci,	-> gb18030 CHAR CHARACTER SET gb18030 COLLATE gb18030_chinese_ci);Query OK, 0 rows affected (0.22 sec)mysql> INSERT INTO t5 VALUES ('[','['),(']',']');Query OK, 2 rows affected (0.01 sec)Records: 2Duplicates: 0Warnings: 0mysql> SELECT DISTINCT gb2312 from t5 ORDER BY gb2312;+--------+| gb2312 |+--------+| ]	|| [	|+--------+2 rows in set (0.00 sec)mysql> SELECT DISTINCT gb18030 from t5 ORDER BY gb18030;+---------+| gb18030 |+---------+| [	 || ]	 |+---------+2 rows in set (0.00 sec)

See the difference? The gb18030 order is obviously better -- ']' should be greater than '[' -- but when two collations are wildly different they shouldn't both be called "chinese_ci".

The second bad thing is the algorithm. The new chinese_ci collation is based onpinyinfor Chinese characters, and binary comparisons of the UPPER() values for non-Chinese characters. This is pretty well useless for non-Chinese. I can bet that somebody will observe "well, duh, it's a Chinese character set" -- but I can't see why one would use an algorithm for Latin/Greek/Cyrillic/etc. characters that's so poor. There's aCommon Locale Data Repositoryfor tailoring for Chinese, there are MySQL worklog tasks that explain the brave new world, there's no need to invent an idiolect when there's a received dialect.

Documentation

The documentation isn't up to date yet -- there's no attempt to explain what the new character set and its collations are about, and no mention at all inthe FAQ.

But the worklog taskWL#4024: gb18030 Chinese character setgives a rough idea of what the coder had in mind before starting. It looks as if WL#4024 was partly copied fromhttp://icu-project.org/docs/papers/unicode-gb18030-faq.htmlso that's also worth a look.

For developers who just need to know what's going on now, just re-read this blog post. What I've described should be enough for people who care about Chinese.

I didn't look for bugs with full-text or LIKE searches, and I didn't look at speed. But I did look hard for problems with the essentials, and found none. Congratulations are due.

陳述
本文內容由網友自願投稿,版權歸原作者所有。本站不承擔相應的法律責任。如發現涉嫌抄襲或侵權的內容,請聯絡admin@php.cn
MySQL中有哪些不同的存儲引擎?MySQL中有哪些不同的存儲引擎?Apr 26, 2025 am 12:27 AM

mysqloffersvariousStorageengines,每個suitedfordferentusecases:1)InnodBisidealForapplicationsNeedingingAcidComplianCeanDhighConcurncurnency,supportingtransactionsancions and foreignkeys.2)myisamisbestforread-Heavy-Heavywyworks,lackingtransactionsactionsacupport.3)記憶

MySQL中有哪些常見的安全漏洞?MySQL中有哪些常見的安全漏洞?Apr 26, 2025 am 12:27 AM

MySQL中常見的安全漏洞包括SQL注入、弱密碼、權限配置不當和未更新的軟件。 1.SQL注入可以通過使用預處理語句防止。 2.弱密碼可以通過強制使用強密碼策略避免。 3.權限配置不當可以通過定期審查和調整用戶權限解決。 4.未更新的軟件可以通過定期檢查和更新MySQL版本來修補。

您如何確定MySQL中的慢速查詢?您如何確定MySQL中的慢速查詢?Apr 26, 2025 am 12:15 AM

在MySQL中識別慢查詢可以通過啟用慢查詢日誌並設置閾值來實現。 1.啟用慢查詢日誌並設置閾值。 2.查看和分析慢查詢日誌文件,使用工具如mysqldumpslow或pt-query-digest進行深入分析。 3.優化慢查詢可以通過索引優化、查詢重寫和避免使用SELECT*來實現。

如何監視MySQL Server的健康和性能?如何監視MySQL Server的健康和性能?Apr 26, 2025 am 12:15 AM

要監控MySQL服務器的健康和性能,應關注系統健康、性能指標和查詢執行。 1)監控系統健康:使用top、htop或SHOWGLOBALSTATUS命令查看CPU、內存、磁盤I/O和網絡活動。 2)追踪性能指標:監控查詢每秒數、平均查詢時間和緩存命中率等關鍵指標。 3)確保查詢執行優化:啟用慢查詢日誌,記錄並優化執行時間超過設定閾值的查詢。

比較和對比Mysql和Mariadb。比較和對比Mysql和Mariadb。Apr 26, 2025 am 12:08 AM

MySQL和MariaDB的主要區別在於性能、功能和許可證:1.MySQL由Oracle開發,MariaDB是其分支。 2.MariaDB在高負載環境中性能可能更好。 3.MariaDB提供了更多的存儲引擎和功能。 4.MySQL採用雙重許可證,MariaDB完全開源。選擇時應考慮現有基礎設施、性能需求、功能需求和許可證成本。

MySQL的許可與其他數據庫系統相比如何?MySQL的許可與其他數據庫系統相比如何?Apr 25, 2025 am 12:26 AM

MySQL使用的是GPL許可證。 1)GPL許可證允許自由使用、修改和分發MySQL,但修改後的分發需遵循GPL。 2)商業許可證可避免公開修改,適合需要保密的商業應用。

您什麼時候選擇InnoDB而不是Myisam,反之亦然?您什麼時候選擇InnoDB而不是Myisam,反之亦然?Apr 25, 2025 am 12:22 AM

選擇InnoDB而不是MyISAM的情況包括:1)需要事務支持,2)高並發環境,3)需要高數據一致性;反之,選擇MyISAM的情況包括:1)主要是讀操作,2)不需要事務支持。 InnoDB適合需要高數據一致性和事務處理的應用,如電商平台,而MyISAM適合讀密集型且無需事務的應用,如博客系統。

在MySQL中解釋外鍵的目的。在MySQL中解釋外鍵的目的。Apr 25, 2025 am 12:17 AM

在MySQL中,外鍵的作用是建立表與表之間的關係,確保數據的一致性和完整性。外鍵通過引用完整性檢查和級聯操作維護數據的有效性,使用時需注意性能優化和避免常見錯誤。

See all articles

熱AI工具

Undresser.AI Undress

Undresser.AI Undress

人工智慧驅動的應用程序,用於創建逼真的裸體照片

AI Clothes Remover

AI Clothes Remover

用於從照片中去除衣服的線上人工智慧工具。

Undress AI Tool

Undress AI Tool

免費脫衣圖片

Clothoff.io

Clothoff.io

AI脫衣器

Video Face Swap

Video Face Swap

使用我們完全免費的人工智慧換臉工具,輕鬆在任何影片中換臉!

熱工具

mPDF

mPDF

mPDF是一個PHP庫,可以從UTF-8編碼的HTML產生PDF檔案。原作者Ian Back編寫mPDF以從他的網站上「即時」輸出PDF文件,並處理不同的語言。與原始腳本如HTML2FPDF相比,它的速度較慢,並且在使用Unicode字體時產生的檔案較大,但支援CSS樣式等,並進行了大量增強。支援幾乎所有語言,包括RTL(阿拉伯語和希伯來語)和CJK(中日韓)。支援嵌套的區塊級元素(如P、DIV),

SecLists

SecLists

SecLists是最終安全測試人員的伙伴。它是一個包含各種類型清單的集合,這些清單在安全評估過程中經常使用,而且都在一個地方。 SecLists透過方便地提供安全測試人員可能需要的所有列表,幫助提高安全測試的效率和生產力。清單類型包括使用者名稱、密碼、URL、模糊測試有效載荷、敏感資料模式、Web shell等等。測試人員只需將此儲存庫拉到新的測試機上,他就可以存取所需的每種類型的清單。

VSCode Windows 64位元 下載

VSCode Windows 64位元 下載

微軟推出的免費、功能強大的一款IDE編輯器

SublimeText3漢化版

SublimeText3漢化版

中文版,非常好用

WebStorm Mac版

WebStorm Mac版

好用的JavaScript開發工具