search
HomeBackend DevelopmentPython TutorialDetailed explanation of string operations and encoding Unicode in Python_

In fact, strings are also a data type. However, what is special about strings is that there is an encoding problem. The following article mainly introduces you to the relevant information on string operations and encoding Unicode in Python. Friends in need can refer to it. Let’s take a look together.

This article mainly introduces you to some knowledge about string operations and encoding Unicode in Python. I won’t say much below. Friends who need it can come and learn together.

String type

str: Unicode string. Strings constructed using '' or r'' are all str, and single quotes can be replaced with double quotes or triple quotes. Regardless of how you specify it, there is no difference when it is stored internally in Python.

bytes: Binary string. Since files in other formats such as jpg cannot be displayed using str, they are represented by bytes. Each byte of bytes is a number from 0 to 255. When printing, Python will display the part that can be expressed in ASCII as ASCII, which makes it easier to read. bytes supports almost all str methods except formatting, even including the re module

bytearray() : Binary string that can be changed in place.

utf-8 encoding range

##RangeNumber of bytesStorage format##0x0000~0x007F (0 ~ 127)0x0080~0x07FF(128 ~ 2047)##0x0800~FFFF(2048 ~ 65535) 3 bytes1110xxxx 10xxxxxx 10xxxxxx4 bytes##0x2000000~0x3FFFFFF5 bytes111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx 0x4000000~0x7FFFFFFF) 6 bytes1111110x 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxxByte order mark BOM
1 byte 0xxxxxxx
2 bytes 110xxxxx 10xxxxxx
##0x10000~1FFFFFF(65536 ~ 2097152)
11110xxx 10xxxxxx 10xxxxxx 10xxxxxx

BOM is the abbreviation of byte order marker,
Specifies the rules when encoding is written

Python is using 'utf-8 'Encoding will not write a BOM header when writing a file, but specifying encoding 'utf-8-sig' will force Python to write a BOM header.

Using 'utf-16-be' will not write a BOM header, but using 'utf-16' will write a BOM header.


>>> open('h.txt','w',encoding='utf-8-sig').write('aaa')
3
>>> open('h.txt','rb').read()
b'\xef\xbb\xbfaaa'
>>> open('h.txt','w',encoding='utf-16').write('bbb')
3
>>> open('h.txt','rb').read()
b'\xff\xfeb\x00b\x00b\x00'
>>> open('hh.txt','w',encoding='utf-16-be').write('ccc')
3
>>> open('hh.txt','rb').read()
b'\x00c\x00c\x00c'
>>> open('h.txt','w',encoding='utf-8').write('ddd')
3
>>> open('h.txt','rb').read()
b'ddd'

Rules when reading

If the correct encoding is specified, the BOM will be ignored, otherwise the BOM It will be displayed as garbled characters or an exception will be returned.

>>> open('h.txt','r').read()
'锘縟dd'
>>> open('h.txt','r',encoding='utf-8-sig').read()
'ddd'

Encoding and decoding

chr and ord

  • >>> ord('中') #20013
    >>> chr(20013) #'中'

Hardcode Unicode into the string.

  • '\xhh': Use 2-digit hexadecimal to represent a character

  • '\uhhhh': Use 4-digit hexadecimal to represent a character Represent a character:


'\Uhhhhhhhh': Use 8-digit hexadecimal to represent a character


>>> s = ' py\x74h\u4e2don' #'pyth in on'


str and bytes, bytearray conversion

str.encode( encoding='utf-8')

bytes(s,encoding='utf-8')<br>

bytes.decode(encoding='utf-8')<br>

str(B, encoding='utf-8')<br>

bytearray(string, encoding='utf-8')

bytearray(bytes)

Document encoding statement

Python uses utf-8 encoding by default.

# -*- coding: latin-1 -*-

: Indicates that the declaration document is latin-1 encoding.

Help function

sys.platform  #&#39;win32&#39;
sys.getdefaultencoding() # &#39;utf-8&#39;
sys.byteorder  #&#39;little&#39;
s.isalnum()  #s表示字符串
s.isalpha()
s.isdecimal
s.isdigit()
s.isnumeric()
s.isprintable()
s.isspace()
s.isidentifier() #如果字符串可以用作变量名,那么返回True
s.islower()
s.isupper()
s.istitle()

More articles related to detailed explanation of string operations and encoding Unicode in Python Please pay attention to PHP Chinese website!

Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Python vs. C  : Understanding the Key DifferencesPython vs. C : Understanding the Key DifferencesApr 21, 2025 am 12:18 AM

Python and C each have their own advantages, and the choice should be based on project requirements. 1) Python is suitable for rapid development and data processing due to its concise syntax and dynamic typing. 2)C is suitable for high performance and system programming due to its static typing and manual memory management.

Python vs. C  : Which Language to Choose for Your Project?Python vs. C : Which Language to Choose for Your Project?Apr 21, 2025 am 12:17 AM

Choosing Python or C depends on project requirements: 1) If you need rapid development, data processing and prototype design, choose Python; 2) If you need high performance, low latency and close hardware control, choose C.

Reaching Your Python Goals: The Power of 2 Hours DailyReaching Your Python Goals: The Power of 2 Hours DailyApr 20, 2025 am 12:21 AM

By investing 2 hours of Python learning every day, you can effectively improve your programming skills. 1. Learn new knowledge: read documents or watch tutorials. 2. Practice: Write code and complete exercises. 3. Review: Consolidate the content you have learned. 4. Project practice: Apply what you have learned in actual projects. Such a structured learning plan can help you systematically master Python and achieve career goals.

Maximizing 2 Hours: Effective Python Learning StrategiesMaximizing 2 Hours: Effective Python Learning StrategiesApr 20, 2025 am 12:20 AM

Methods to learn Python efficiently within two hours include: 1. Review the basic knowledge and ensure that you are familiar with Python installation and basic syntax; 2. Understand the core concepts of Python, such as variables, lists, functions, etc.; 3. Master basic and advanced usage by using examples; 4. Learn common errors and debugging techniques; 5. Apply performance optimization and best practices, such as using list comprehensions and following the PEP8 style guide.

Choosing Between Python and C  : The Right Language for YouChoosing Between Python and C : The Right Language for YouApr 20, 2025 am 12:20 AM

Python is suitable for beginners and data science, and C is suitable for system programming and game development. 1. Python is simple and easy to use, suitable for data science and web development. 2.C provides high performance and control, suitable for game development and system programming. The choice should be based on project needs and personal interests.

Python vs. C  : A Comparative Analysis of Programming LanguagesPython vs. C : A Comparative Analysis of Programming LanguagesApr 20, 2025 am 12:14 AM

Python is more suitable for data science and rapid development, while C is more suitable for high performance and system programming. 1. Python syntax is concise and easy to learn, suitable for data processing and scientific computing. 2.C has complex syntax but excellent performance and is often used in game development and system programming.

2 Hours a Day: The Potential of Python Learning2 Hours a Day: The Potential of Python LearningApr 20, 2025 am 12:14 AM

It is feasible to invest two hours a day to learn Python. 1. Learn new knowledge: Learn new concepts in one hour, such as lists and dictionaries. 2. Practice and exercises: Use one hour to perform programming exercises, such as writing small programs. Through reasonable planning and perseverance, you can master the core concepts of Python in a short time.

Python vs. C  : Learning Curves and Ease of UsePython vs. C : Learning Curves and Ease of UseApr 19, 2025 am 12:20 AM

Python is easier to learn and use, while C is more powerful but complex. 1. Python syntax is concise and suitable for beginners. Dynamic typing and automatic memory management make it easy to use, but may cause runtime errors. 2.C provides low-level control and advanced features, suitable for high-performance applications, but has a high learning threshold and requires manual memory and type safety management.

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

WebStorm Mac version

WebStorm Mac version

Useful JavaScript development tools

ZendStudio 13.5.1 Mac

ZendStudio 13.5.1 Mac

Powerful PHP integrated development environment

SecLists

SecLists

SecLists is the ultimate security tester's companion. It is a collection of various types of lists that are frequently used during security assessments, all in one place. SecLists helps make security testing more efficient and productive by conveniently providing all the lists a security tester might need. List types include usernames, passwords, URLs, fuzzing payloads, sensitive data patterns, web shells, and more. The tester can simply pull this repository onto a new test machine and he will have access to every type of list he needs.

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)