search
HomeBackend DevelopmentPython TutorialPython uses struct to handle binary (pack and unpack usage)

Sometimes you need to use python to process binary data, such as when accessing files and socket operations. At this time, you can use python's struct module to complete. You can use struct to process structures in the C language.

In the struct module The three most important functions are pack(), unpack(), calcsize()

pack(fmt, v1, v2, ...) according to the given format (fmt), encapsulate the data into a string (actually It is similar to the byte flow of the C structure)

UnPACK (FMT, String) based on the given format (FMT) to analyze the byte running string, and return the parsed Tuple

calcsize (FMT) calculation given (FMT). fmt) occupies how many bytes of memory

The supported formats in struct are as follows:

Format C Type Python Number of bytes

x pad byte no value 1

c char string of length 1 1

b signed char integer 1

B unsigned char integer 1

? _Bool bool 1

h short integer 2

H unsigned short integer 2

i int integer 4

I unsigned int integer or long 4

l long integer 4

L unsigned long long 4

q long long long 8

Q unsigned long long long 8

f float float 4

d double float 8

s char[] string 1

p char[] string 1

P void * long

Note 1.q and Q are only interesting when the machine supports 64-bit operation

Note 2. There can be a number before each format to indicate the number

Note 3.s format represents a certain length of characters String, 4s represents a string of length 4, but p represents a pascal string

Note 4. P is used to convert a pointer, its length is related to the machine word length

Note 5. The last one can be used to represent a pointer Type, occupies 4 bytes

In order to exchange data with the structure in c, it is also necessary to consider that some c or c++ compilers use byte alignment, usually 32-bit systems with 4 bytes as the unit. Therefore, the struct is converted according to the byte order of the local machine. The first character in the format can be used to change the alignment. The definition is as follows:

Character Byte order Size and alignment

@ native native Enough 4 bytes

= native standard                                                                                     use using byte using

– little-endian                                                                                                    .

The method of use is to put it in the first position of fmt, just like '@5s6sif'

Example 1:

The structure is as follows:

struct Header
{
    unsigned short id;
    char[4] tag;
    unsigned int version;
    unsigned int count;
}

The above structure data is received through socket.recv, and there is a string s, now you need to parse it out, you can use the unpack() function:

import struct
id, tag, version, count = struct.unpack("!H4s2I", s)

In the above format string, ! means that we want to use network byte order to parse, because our data is received from the network, When transmitted over the network, it is in network byte order. The following H represents an unsigned short id, 4s represents a 4-byte long string, and 2I represents two unsigned int type data.

Just pass one unpack, now our information has been saved in id, tag, version, and count.

Similarly, it is also very convenient to pack local data into struct format:

ss = struct.pack("!H4s2I", id, tag, version, count);

The pack function puts id, tag, version, count is converted into a structure Header according to the specified format. ss is now a string (actually a byte stream similar to a c structure). This string can be sent out through socket.send(ss).

Example 2:

import struct
a=12.34
#将a变为二进制
bytes=struct.pack('i',a)

At this time, bytes is a string string, and the string is the same as the binary storage content of a in bytes.

Then perform the reverse operation, and convert the existing binary data bytes (actually a string) into the python data type:

#Note that unpack returns a tuple!!

a,=struct.unpack('i',bytes)

If it is composed of multiple data, it can be like this:

a='hello'
b='world!'
c=2
d=45.123
bytes=struct.pack('5s6sif',a,b,c,d)

The bytes at this time are data in binary form, and can be written directly to a file such as binfile.write(bytes)Then, we can read it out when we need it , bytes=binfile.read()

and then decoded into python variables through struct.unpack():

a,b,c,d=struct.unpack('5s6sif',bytes)

'5s6sif' is called fmt, which is a formatted string, consisting of numbers and characters, 5s means 5 characters The string, 2i, represents 2 integers, etc. The following are the available characters and types. The ctype representation can correspond to the types in python one-to-one.

Note: Problems encountered when processing binary files

When we process binary files, we need to use the following method:

binfile=open(filepath,'rb')    
#读二进制文件
binfile=open(filepath,'wb')   
#写二进制文件

So what is the difference between the result of binfile=open(filepath,'r')?

There are two differences:

First, if you encounter '0x1A' when using 'r', it will be regarded as the end of the file, which is EOF. Using 'rb' does not have this problem. That is, if you write in binary and read out in text, only part of the file will be read out if '0X1A' is present. When using 'rb', it will read to the end of the file.

Second, for the string x=’abcndef’, we can use len(x) to get its length to be 7. We call n the newline character, which is actually ‘0X0A’. When we write in 'w', which is text mode, '0X0A' will be automatically changed into two characters '0X0D', '0X0A' on the Windows platform, that is, the file length actually becomes 8. When reading in 'r' text mode, it is automatically converted to the original newline character. If you change to 'wb' binary mode to write, one character will remain unchanged, and it will be read as it is when reading. So if you write in text mode and read in binary mode, you have to consider this extra byte. '0X0D' is also called the carriage return character. It will not change under Linux. Because linux only uses '0X0A' to represent line breaks.

Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
How do you create multi-dimensional arrays using NumPy?How do you create multi-dimensional arrays using NumPy?Apr 29, 2025 am 12:27 AM

Create multi-dimensional arrays with NumPy can be achieved through the following steps: 1) Use the numpy.array() function to create an array, such as np.array([[1,2,3],[4,5,6]]) to create a 2D array; 2) Use np.zeros(), np.ones(), np.random.random() and other functions to create an array filled with specific values; 3) Understand the shape and size properties of the array to ensure that the length of the sub-array is consistent and avoid errors; 4) Use the np.reshape() function to change the shape of the array; 5) Pay attention to memory usage to ensure that the code is clear and efficient.

Explain the concept of 'broadcasting' in NumPy arrays.Explain the concept of 'broadcasting' in NumPy arrays.Apr 29, 2025 am 12:23 AM

BroadcastinginNumPyisamethodtoperformoperationsonarraysofdifferentshapesbyautomaticallyaligningthem.Itsimplifiescode,enhancesreadability,andboostsperformance.Here'showitworks:1)Smallerarraysarepaddedwithonestomatchdimensions.2)Compatibledimensionsare

Explain how to choose between lists, array.array, and NumPy arrays for data storage.Explain how to choose between lists, array.array, and NumPy arrays for data storage.Apr 29, 2025 am 12:20 AM

ForPythondatastorage,chooselistsforflexibilitywithmixeddatatypes,array.arrayformemory-efficienthomogeneousnumericaldata,andNumPyarraysforadvancednumericalcomputing.Listsareversatilebutlessefficientforlargenumericaldatasets;array.arrayoffersamiddlegro

Give an example of a scenario where using a Python list would be more appropriate than using an array.Give an example of a scenario where using a Python list would be more appropriate than using an array.Apr 29, 2025 am 12:17 AM

Pythonlistsarebetterthanarraysformanagingdiversedatatypes.1)Listscanholdelementsofdifferenttypes,2)theyaredynamic,allowingeasyadditionsandremovals,3)theyofferintuitiveoperationslikeslicing,but4)theyarelessmemory-efficientandslowerforlargedatasets.

How do you access elements in a Python array?How do you access elements in a Python array?Apr 29, 2025 am 12:11 AM

ToaccesselementsinaPythonarray,useindexing:my_array[2]accessesthethirdelement,returning3.Pythonuseszero-basedindexing.1)Usepositiveandnegativeindexing:my_list[0]forthefirstelement,my_list[-1]forthelast.2)Useslicingforarange:my_list[1:5]extractselemen

Is Tuple Comprehension possible in Python? If yes, how and if not why?Is Tuple Comprehension possible in Python? If yes, how and if not why?Apr 28, 2025 pm 04:34 PM

Article discusses impossibility of tuple comprehension in Python due to syntax ambiguity. Alternatives like using tuple() with generator expressions are suggested for creating tuples efficiently.(159 characters)

What are Modules and Packages in Python?What are Modules and Packages in Python?Apr 28, 2025 pm 04:33 PM

The article explains modules and packages in Python, their differences, and usage. Modules are single files, while packages are directories with an __init__.py file, organizing related modules hierarchically.

What is docstring in Python?What is docstring in Python?Apr 28, 2025 pm 04:30 PM

Article discusses docstrings in Python, their usage, and benefits. Main issue: importance of docstrings for code documentation and accessibility.

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

SublimeText3 Linux new version

SublimeText3 Linux new version

SublimeText3 Linux latest version

SecLists

SecLists

SecLists is the ultimate security tester's companion. It is a collection of various types of lists that are frequently used during security assessments, all in one place. SecLists helps make security testing more efficient and productive by conveniently providing all the lists a security tester might need. List types include usernames, passwords, URLs, fuzzing payloads, sensitive data patterns, web shells, and more. The tester can simply pull this repository onto a new test machine and he will have access to every type of list he needs.

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

VSCode Windows 64-bit Download

VSCode Windows 64-bit Download

A free and powerful IDE editor launched by Microsoft

PhpStorm Mac version

PhpStorm Mac version

The latest (2018.2.1) professional PHP integrated development tool