Detailed explanation of Python's method of using struct to process binary data

This article mainly introduces relevant information about Python's use of struct to process binary examples. I hope that everyone can master this part of the content through this article. Friends in need can refer to it

Python uses struct to process Detailed explanation of binary examples

Sometimes you need to use python to process binary data, for example, when accessing files and socket operations. At this time, you can use python's struct module to complete it. You can use struct to process it. Structure in c language.

• The three most important functions in the struct module are pack(), unpack(), calcsize()

• pack(fmt, v1, v2, ...) Pack the data into a string according to the given format (fmt) (actually a byte stream similar to a c structure)

• unpack(fmt, string) Parses the byte stream string according to the given format (fmt) and returns the parsed tuple

• calcsize(fmt) Calculation How many bytes of memory does a given format (fmt) occupy?
  

The supported formats in struct are as follows:

##Qunsigned long longlong8 ffloatfloat4##dsp##Pvoid *longNote 1.q and Q are only interesting when the machine supports 64-bit operationNote 2.Every There can be a number before each format, indicating the number

Format	C Type	Python	Number of bytes
x	pad byte	no value	1
c	char	string of length 1	1
b	signed char	integer	1
B	unsigned char	integer	1
?	_Bool	bool	1
h	short	integer	2
H	unsigned short	integer	2
i	int	integer	4
I	unsigned int	integer or long	4
l	long	integer	4
L	unsigned long	long	4
q	long long	long	8
double	float	8
char[]	string	1
char[]	string	1

Note 3. The s format represents a string of a certain length, 4s represents a string of length 4, but p represents a pascal string

Note 4.P Used to convert a pointer, its length is related to the machine word length
Note 5. The last one can be used to represent the pointer type, occupying 4 bytes


In order to be the same as the structure in c When exchanging data, you should also consider that some c or c++ compilers use byte alignment, usually in 32-bit systems with 4 bytes, so the struct is converted according to the local machine byte order. You can use the first character in the format characters to change the alignment. The definition is as follows:

##CharacterByte order@nativenativebig-endianstandard Based on the original number of bytes##! standard Based on the original number of bytes

The method of use is to put it in the first position of fmt, like '@5s6sif'

Example 1:

For example A structure

struct Header
{
  unsigned short id;
  char[4] tag;
  unsigned int version;
  unsigned int count;
}

received the above structure data through socket.recv, which is stored in the string s. Now it needs to be parsed out. You can use unpack() function.

import struct id, tag, version, count = struct.unpack("!H4s2I", s)

In the above format string, ! indicates that we want to use network byte order parsing, because our data is from the network When received, it is in network byte order when transmitted on the network. The following H represents an unsigned short id, 4s represents a 4-byte long string, and 2I represents two unsigned int type data.

Just through an unpack, now our information has been saved in id, tag, version, and count.

Similarly, it is also very convenient to pack local data into a struct Format.

ss = struct.pack("!H4s2I", id, tag, version, count);

The pack function converts id, tag, version, count into the structure Header according to the specified format. ss is now a string ( It is actually a byte stream similar to a c structure), and this string can be sent out through socket.send(ss).

Example 2:

import struct
a=12.34
#将a变为二进制
bytes=struct.pack('i',a)

At this time bytes is a string string, and the string is the same as the binary storage content of a in bytes.

Then reverse the operation

Existing binary data bytes, (actually a string), convert it in turn into the python data type:

a,=struct.unpack('i',bytes)

Note that unpack returns a tuple

So if there is only one variable:

bytes=struct.pack('i',a)

Then, you need to do this when decoding

a,=struct.unpack('i',bytes) 或者 (a,)=struct.unpack('i',bytes)

If you use a=struct.unpack('i',bytes) directly, then a=( 12.34,) is a tuple instead of the original floating point number.

If it is composed of multiple data, it can be like this:

a='hello'
b='world!'
c=2
d=45.123
bytes=struct.pack('5s6sif',a,b,c,d)

The bytes at this time are data in binary form, and you can directly Write a file such as binfile.write(bytes)

Then, when we need it, we can read it out, bytes=binfile.read()

and then pass struct. unpack() is decoded into a python variable

a,b,c,d=struct.unpack('5s6sif',bytes)

'5s6sif' is called fmt, which is a format string, consisting of numbers and characters, 5s means 5 A string of characters, 2i, represents 2 integers, etc. The following are the available characters and types. The ctype representation can correspond to the types in Python one-to-one.

Note: Problems encountered when processing binary files

When we process binary files, we need to use the following methods

binfile=open(filepath,'rb')  读二进制文件

binfile=open(filepath,'wb')  写二进制文件

So what is the difference from the result of binfile=open(filepath,'r')?

There are two differences:

First, if you encounter '0x1A' when using 'r', it will be regarded as the end of the file, which is EOF. Using 'rb' does not have this problem. That is, if you write in binary and read out in text, only part of the file will be read out if '0X1A' is present. When using 'rb', it will read to the end of the file.

Second, for the string x='abc\ndef', we can use len(x) to get its length to be 7, \n we call it a newline character, which is actually '0X0A'. When we write in 'w', which is text mode, '0X0A' will be automatically changed into two characters '0X0D', '0X0A' on the Windows platform, that is, the file length actually becomes 8. When reading in 'r' text mode, it is automatically converted to the original newline character. If you change to 'wb' binary mode to write, one character will remain unchanged, and it will be read as it is when reading. So if you write in text mode and read in binary mode, you have to consider this extra byte. '0X0D' is also called the carriage return character. It will not change under Linux. Because linux only uses '0X0A' to represent line breaks.

Size and alignment
native Make up enough 4 bytes		=
standard                                                                                                                            can be based on the original number of bytes		7c849ecf228f306c811b984665de01f1
network (= big-endian)

The above is the detailed content of Detailed explanation of Python's method of using struct to process binary data. For more information, please follow other related articles on the PHP Chinese website!