search
HomeWeb Front-endJS TutorialCoding algorithm of utf8 reprint_basic knowledge

unicode字符集是我们世界上最完善最全面的字符集,几乎包含了世界上所有的字符。其实可以这么理解,unicode字符集是一张巨大的表格,把世界上各种语言的字符和标点符号都编排到里面,然后按照一定的顺序给每个字符排号(很遗憾的是对于中文来说,这个顺序不是按照汉语拼音的顺序)。有了这张巨大的表格,世界上大部分字符就都有一个unicode内码(整数)来对应。计算机就通过记录字符的unicode码来表示这个字符,然后交给操作系统,操作系统通过unicode码到字符字体点阵的映射把这个内码转换成字体点阵显示到我们的屏幕上。

         utf8是我们常用的编码方式,在web开发中使用utf8编码能完全解决字符集问题。其实utf8是unicode字符集的一种物理实现,它描述了如何高效的存储unicode的内码(就是上面说的字符在字符集的顺序码),RFC2044文档(http: //www.ietf.org/rfc/rfc2044.txt?number=2044)描述了如何从一个内码转换成utf8格式的算法。英文不好没关系,看这个转换表就会马上明白了:

   UCS-4 range (hex.)           UTF-8 octet sequence (binary)
   0000 0000-0000 007F   0xxxxxxx
   0000 0080-0000 07FF   110xxxxx 10xxxxxx
   0000 0800-0000 FFFF   1110xxxx 10xxxxxx 10xxxxxx
   0001 0000-001F FFFF   11110xxx 10xxxxxx 10xxxxxx 10xxxxxx
   0020 0000-03FF FFFF   111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx
   0400 0000-7FFF FFFF   1111110x 10xxxxxx ... 10xxxxxx

         上面的表格左边是16进制表示的unicode内码,最后一行的16进制数“7FFF FFFF”是utf8所能表示的内码的最大值,换成10进制是这样的一个数:2147483647(够大吧:))[对不起,此文最初这儿写错了,已改正]。上面表格中右边一列就是utf8的二进制格式,转换规则可谓一目了然。我直接给出算法吧(js代码):


    function toUtf8(code)
    {
     var iByte=0;
     var i=0;
     result="";
     while(code>0x7f)
     {
      iByte=code%0x40;
      code=(code-iByte)/0x40;
      result="%"+(iByte|0x80).toString(16).toUpperCase()+result;
      i++;
     }
     prefix=[0x0,0xc0,0xe0,0xf0,0xf8,0xfc];
     if(i>prefix.length)
     {
      i=5;
     }
     result=""+(code|prefix[i]).toString(16).toUpperCase()+result;
     return result;
    }
        


          例如字符"汉"的unicode是6C49,把这个unicode字符表示为一个大整数,然后转变成多字节编码110110001001001:
          观察这个整数的二进制码序列(110,110001,001001)
          从后往前取
          如果这个二进制序列只有后7位(小于128,也就是ascii字符)则直接取后7位二进制数形成一个utf8字符。
          上面的字符“汉”二进制序列大于7位,所以取后6位(1001001),加10形成一个utf8字节(10 001001 ,16进制89)。
          剩下的二进制序列(110,110001)从后向前取6位,加10形成一个utf8字节(10 110001,16进制B1)。
          剩下的二进制序列(110)从后向前取6位,由于不足6位,将这个数和1110000相或,得到字符11100110,16进制E6
          最后,就得到了utf8编码,16进制表示为E6B189

【Application Areas】
Although most of this standard algorithm has been implemented by development tool providers or libraries, we still need to implement this algorithm ourselves at some point:
                                                                                                                          character, the server uses the above algorithm to convert the unicode serial number after u into utf8 character Use the above algorithm combined with escape to directly implement the encodeURI function on the client (this solution is recommended)

Implement json-rpc service

in the form of "uXXXX". So we need to convert the server-side characters into json. For php, there are now two open source projects
JSON-PHP
and
PHP-JSON.                                                                                                                                                                                                                                                                                                    .

Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Javascript Data Types : Is there any difference between Browser and NodeJs?Javascript Data Types : Is there any difference between Browser and NodeJs?May 14, 2025 am 12:15 AM

JavaScript core data types are consistent in browsers and Node.js, but are handled differently from the extra types. 1) The global object is window in the browser and global in Node.js. 2) Node.js' unique Buffer object, used to process binary data. 3) There are also differences in performance and time processing, and the code needs to be adjusted according to the environment.

JavaScript Comments: A Guide to Using // and /* */JavaScript Comments: A Guide to Using // and /* */May 13, 2025 pm 03:49 PM

JavaScriptusestwotypesofcomments:single-line(//)andmulti-line(//).1)Use//forquicknotesorsingle-lineexplanations.2)Use//forlongerexplanationsorcommentingoutblocksofcode.Commentsshouldexplainthe'why',notthe'what',andbeplacedabovetherelevantcodeforclari

Python vs. JavaScript: A Comparative Analysis for DevelopersPython vs. JavaScript: A Comparative Analysis for DevelopersMay 09, 2025 am 12:22 AM

The main difference between Python and JavaScript is the type system and application scenarios. 1. Python uses dynamic types, suitable for scientific computing and data analysis. 2. JavaScript adopts weak types and is widely used in front-end and full-stack development. The two have their own advantages in asynchronous programming and performance optimization, and should be decided according to project requirements when choosing.

Python vs. JavaScript: Choosing the Right Tool for the JobPython vs. JavaScript: Choosing the Right Tool for the JobMay 08, 2025 am 12:10 AM

Whether to choose Python or JavaScript depends on the project type: 1) Choose Python for data science and automation tasks; 2) Choose JavaScript for front-end and full-stack development. Python is favored for its powerful library in data processing and automation, while JavaScript is indispensable for its advantages in web interaction and full-stack development.

Python and JavaScript: Understanding the Strengths of EachPython and JavaScript: Understanding the Strengths of EachMay 06, 2025 am 12:15 AM

Python and JavaScript each have their own advantages, and the choice depends on project needs and personal preferences. 1. Python is easy to learn, with concise syntax, suitable for data science and back-end development, but has a slow execution speed. 2. JavaScript is everywhere in front-end development and has strong asynchronous programming capabilities. Node.js makes it suitable for full-stack development, but the syntax may be complex and error-prone.

JavaScript's Core: Is It Built on C or C  ?JavaScript's Core: Is It Built on C or C ?May 05, 2025 am 12:07 AM

JavaScriptisnotbuiltonCorC ;it'saninterpretedlanguagethatrunsonenginesoftenwritteninC .1)JavaScriptwasdesignedasalightweight,interpretedlanguageforwebbrowsers.2)EnginesevolvedfromsimpleinterpreterstoJITcompilers,typicallyinC ,improvingperformance.

JavaScript Applications: From Front-End to Back-EndJavaScript Applications: From Front-End to Back-EndMay 04, 2025 am 12:12 AM

JavaScript can be used for front-end and back-end development. The front-end enhances the user experience through DOM operations, and the back-end handles server tasks through Node.js. 1. Front-end example: Change the content of the web page text. 2. Backend example: Create a Node.js server.

Python vs. JavaScript: Which Language Should You Learn?Python vs. JavaScript: Which Language Should You Learn?May 03, 2025 am 12:10 AM

Choosing Python or JavaScript should be based on career development, learning curve and ecosystem: 1) Career development: Python is suitable for data science and back-end development, while JavaScript is suitable for front-end and full-stack development. 2) Learning curve: Python syntax is concise and suitable for beginners; JavaScript syntax is flexible. 3) Ecosystem: Python has rich scientific computing libraries, and JavaScript has a powerful front-end framework.

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

VSCode Windows 64-bit Download

VSCode Windows 64-bit Download

A free and powerful IDE editor launched by Microsoft

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SAP NetWeaver Server Adapter for Eclipse

SAP NetWeaver Server Adapter for Eclipse

Integrate Eclipse with SAP NetWeaver application server.

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

ZendStudio 13.5.1 Mac

ZendStudio 13.5.1 Mac

Powerful PHP integrated development environment