Introduction to python character encoding file method

Character encoding development

ASCII 255 only supports English letters and numbers, special characters 1bytes

Unicode Chinese and English Unified 2bytes

utf-8 Chinese>3bytes English >1bytes

bytes type

Text data is always unicode represented by str type, binary data is represented by bytes type

Binary data is used in video, audio files and sending Socket network transmission data, etc.

Convert string to binary str.encode("encoding=utf-8")

Convert binary to string b'\xe2\x82'.decode(" encoding=utf-8")

The file handle is the memory address of the file object

Character encoding and transcoding

Chinese characters cannot be stored in the ASCII code table, which is the default system of Windows The character encoding is GBK.

Unicode character encoding can store all characters in the world, but all characters occupy two bytes. It turns out that a 2M

English file needs to use Unicode 4M storage space

Convert utf-8 string to gbk character

Conversion of any two encoding strings must be done by first converting to Unicode encoding

Unicode (Unicode, Universal Code, Unicode) is a character encoding used on computers. Unicode was created to solve the limitations of traditional character encoding schemes. It sets a unified and unique binary encoding for each character in each language.

There are basically two situations where garbled characters appear. :

1. There is no character encoding

2. The character encoding conflicts. The character set specified by others when writing this program is in the wrong position with the character set we use

In the 2.x version of Python, when Pyton interprets the .py file, the default encoding is ASCII code

The default in Python3 is Unicode encoding

Because in python2. The default encoding in

str ="Hello" //This string is encoded in utf-8

new_str=str.decode('utf-8') / / Pass the original encoding format of str to the decode function to convert it into Unicode encoding

ret = new_str.encode('GBK') // Convert the unicode encoding into a GBK encoded string

str = u"Hello" Adding a u letter in front of the string indicates that the string is set to Unicode encoding

Encoding

When the python interpreter loads the code in the .py file, it will The content is encoded (default ascill), so if the encoding type is not specified,

If there is Chinese, an error will be reported.

Python’s working process

python Read the code into memory 2. Lexical and syntactic analysis 3. Put it in the compiler ---》 Generate bytecode 4. Execute bytecode --- 》Generate machine code CPU execution

Variable

The value of the string cannot be modified. It is continuous in the memory. If you want to change it, you must reserve it later, so modification is not supported. !
String characteristics, once modified, recreate it

The above is the detailed content of Introduction to python character encoding file method. For more information, please follow other related articles on the PHP Chinese website!