In some low-level libraries, you often see the use of unsafe packages. This article will take you to understand the unsafe package in Golang, introduce the role of the unsafe package and how to use Pointer. I hope it will be helpful to you!
The unsafe package provides some operations that can bypass go's type safety check, thereby directly operating memory addresses and doing some tricky operations. The running environment of the sample code is
go version go1.18 darwin/amd64
Memory alignment
The unsafe package provides the Sizeof method to obtain the memory size occupied by variables. "Does not include the memory size of the variable pointed by the pointer", Alignof obtains the memory alignment coefficient, the specific memory alignment rules can be googled by yourself.
type demo1 struct { a bool // 1 b int32 // 4 c int64 // 8 } type demo2 struct { a bool // 1 c int64 // 8 b int32 // 4 } type demo3 struct { // 64 位操作系统, 字长 8 a *demo1 // 8 b *demo2 // 8 } func MemAlign() { fmt.Println(unsafe.Sizeof(demo1{}), unsafe.Alignof(demo1{}), unsafe.Alignof(demo1{}.a), unsafe.Alignof(demo1{}.b), unsafe.Alignof(demo1{}.c)) // 16,8,1,4,8 fmt.Println(unsafe.Sizeof(demo2{}), unsafe.Alignof(demo2{}), unsafe.Alignof(demo2{}.a), unsafe.Alignof(demo2{}.b), unsafe.Alignof(demo2{}.c)) // 24,8,1,4,8 fmt.Println(unsafe.Sizeof(demo3{})) // 16 } // 16}复制代码
From the above case, you can see that demo1 and demo2 contain the same Attributes, only the order of the defined attributes is different, but the memory size of the variables is different. This is because memory alignment occurs.
When the computer is processing tasks, it will process data in units of specific word lengths "For example: 32-bit operating system, word length is 4; 64-bit operating system, word length is 8". Then, when reading data, the unit is also based on word length. For example: For 64-bit operating systems, the number of bytes read by the program at one time is a multiple of 8. The following is the layout of demo1 under non-memory alignment and memory alignment:
Non-memory alignment:
The variable c will be placed in different word lengths, and the CPU needs to read at the same time Read twice and process the results of both times at the same time to get the value of c. Although this method saves memory space, it will increase processing time.
Memory alignment:
Memory alignment adopts a scheme that can avoid the situation of the same non-memory alignment, but it will take up some extra space "space for time". You can google for specific memory alignment rules.
Unsafe Pointer
You can declare a pointer type in go. The type here is safe pointer, that is, it is necessary to make it clear that the pointer points to Type, if the types do not match, an error will occur during compilation. As in the following example, the compiler will think that MyString and string are different types and cannot be assigned.
func main() { type MyString string s := "test" var ms MyString = s // Cannot use 's' (type string) as the type MyString fmt.Println(ms) }
Is there a type that can point to variables of any type? You can use unsfe.Pointer, which can point to any type of variable. Through the declaration of Pointer, we can know that it is a pointer type, pointing to the address of the variable. The value corresponding to the specific address can be converted through uinptr. Pointer has the following four special operations:
- Any type of pointer can be converted into a Pointer type
- Pointer type variables can be converted into any type of pointer
- uintptr type variables can be converted to Pointer type
- Pointer type variables can be converted to uintprt type
type Pointer *ArbitraryType // uintptr is an integer type that is large enough to hold the bit pattern of // any pointer. type uintptr uintptr func main() { d := demo1{true, 1, 2} p := unsafe.Pointer(&d) // 任意类型的指针可以转换为 Pointer 类型 pa := (*demo1)(p) // Pointer 类型变量可以转换成 demo1 类型的指针 up := uintptr(p) // Pointer 类型的变量可以转换成 uintprt 类型 pu := unsafe.Pointer(up) // uintptr 类型的变量可以转换成 Pointer 类型; 当 GC 时, d 的地址可能会发生变更, 因此, 这里的 up 可能会失效 fmt.Println(d.a, pa.a, (*demo1)(pu).a) // true true true }
Six ways to use Pointer
The official documentation provides six usage postures of Pointer.
Convert *T1 to *T2 through Pointer
Pointer directly points to a piece of memory, so this piece of memory can be Convert the memory address to any type. It should be noted here that T1 and T2 need to have the same memory layout and there will be abnormal data.
func main() { type myStr string ms := []myStr{"1", "2"} //ss := ([]string)(ms) Cannot convert an expression of the type '[]myStr' to the type '[]string' ss := *(*[]string)(unsafe.Pointer(&ms)) // 将 pointer 指向的内存地址直接转换成 *[]string fmt.Println(ms, ss) }
What happens if the memory layout of T1 and T2 is different? In the example below, although demo1 and demo2 contain the same structure, they have different memory layouts due to memory alignment. When converting Pointer, 24 "sizeof" bytes will be read starting from the address of demo1, and the conversion will be performed according to the memory alignment rules of demo2. The first byte will be converted to a:true, and 8-16 bytes will be converted to c. :2, 16-24 bytes are beyond the range of demo1, but can still be read directly, and the unexpected value b:17368000 is obtained.
type demo1 struct { a bool // 1 b int32 // 4 c int64 // 8 } type demo2 struct { a bool // 1 c int64 // 8 b int32 // 4 } func main() { d := demo1{true, 1, 2} pa := (*demo2)(unsafe.Pointer(&d)) // Pointer 类型变量可以转换成 demo2 类型的指针 fmt.Println(pa.a, pa.b, pa.c) // true, 17368000, 2, }
Convert Pointer type to uintptr type "Uinptr should not be converted to Pointer"
Pointer is a pointer type that can point to any variable. You can print the address of the variable pointed by Pointer by converting Pointer to uintptr. Additionally: uintptr should not be converted to Pointer. Take the following example: When GC occurs, the address of d may change, then up points to the wrong memory due to unsynchronized updates.
func main() { d := demo1{true, 1, 2} p := unsafe.Pointer(&d) up := uintptr(p) fmt.Printf("uintptr: %x, ptr: %p \n", up, &d) // uintptr: c00010c010, ptr: 0xc00010c010 fmt.Println(*(*demo1)(unsafe.Pointer(up))) // 不允许 }
通过算数计算将 Pointer 转换为 uinptr 再转换回 Pointer
当 Piointer 指向一个结构体时, 可以通过此方式获取到结构体内部特定属性的 Pointer。
func main() { d := demo1{true, 1, 2} // 等同于 unsafe.Pointer(&d.b); unsafe.Add(unsafe.Pointer(&d), unsafe.Offsetof(d.b)) pb := unsafe.Pointer(uintptr(unsafe.Pointer(&d)) + unsafe.Offsetof(d.b)) fmt.Println(pb) }
当调用 syscall.Syscall 的时候, 可以讲 Pointer 转换为 uintptr
前面说过, 由于 GC 会导致变量的地址发生变更, 因此不可以直接处理 uintptr。但是, 在调用 syscall.Syscall 时候可以允许传递一个 uintptr, 这里可以简单理解为是编译器做了特殊处理, 来保证 uintptr 是安全的。
- 调用方式:
- syscall.Syscall(SYS_READ, uintptr( fd ), uintptr(unsafe.Pointer(p)), uintptr(n))
下面这种方式是不允许的:
u := uintptr(unsafe.Pointer(p)) // 不应该保存到一个变量上 syscall.Syscall(SYS_READ, uintptr( fd ), u, uintptr(n))
可以将 reflect.Value.Pointer 或 reflect.Value.UnsafeAddr 的结果「uintptr」转换为 Pointer
在 reflect 包中的 Value.Pointer 和 Value.UnsafeAddr 直接返回了地址对应的值「uintptr」, 可以直接将其结果转为 Pointer
func main() { d := demo1{true, 1, 2} // 等同于 unsafe.Pointer(&d.b); unsafe.Add(unsafe.Pointer(&d), unsafe.Offsetof(d.b)) pb := unsafe.Pointer(uintptr(unsafe.Pointer(&d)) + unsafe.Offsetof(d.b)) // up := reflect.ValueOf(&d.b).Pointer(), pc := unsafe.Pointer(up); 不安全, 不应存储到变量中 pc := unsafe.Pointer(reflect.ValueOf(&d.b).Pointer()) fmt.Println(pb, pc) }
可以将 reflect.SliceHeader 或者 reflect.StringHeader 的 Data 字段与 Pointer 相互转换
SliceHeader 和 StringHeader 其实是 slice 和 string 的内部实现, 里面都包含了一个字段 Data「uintptr」, 存储的是指向 []T 的地址, 这里之所以使用 uinptr 是为了不依赖 unsafe 包。
func main() { s := "a" hdr := (*reflect.StringHeader)(unsafe.Pointer(&s)) // *string to *StringHeader fmt.Println(*(*[1]byte)(unsafe.Pointer(hdr.Data))) // 底层存储的是 utf 编码后的 byte 数组 arr := [1]byte{65} hdr.Data = uintptr(unsafe.Pointer(&arr)) hdr.Len = len(arr) ss := *(*string)(unsafe.Pointer(hdr)) fmt.Println(ss) // A arr[0] = 66 fmt.Println(ss) //B }
应用
string、byte 转换
在业务上, 经常遇到 string 和 []byte 的相互转换。我们知道, string 底层其实也是存储的一个 byte 数组, 可以通过 reflect 直接获取 string 指向的 byte 数组, 赋值给 byte 切片, 避免内存拷贝。
func StrToByte(str string) []byte { return []byte(str) } func StrToByteV2(str string) (b []byte) { bh := (*reflect.SliceHeader)(unsafe.Pointer(&b)) sh := (*reflect.StringHeader)(unsafe.Pointer(&str)) bh.Data = sh.Data bh.Cap = sh.Len bh.Len = sh.Len return b } // go test -bench . func BenchmarkStrToArr(b *testing.B) { for i := 0; i < b.N; i++ { StrToByte(`{"f": "v"}`) } } func BenchmarkStrToArrV2(b *testing.B) { for i := 0; i < b.N; i++ { StrToByteV2(`{"f": "v"}`) } } //goos: darwin //goarch: amd64 //pkg: github.com/demo/lsafe //cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz //BenchmarkStrToArr-12 264733503 4.311 ns/op //BenchmarkStrToArrV2-12 1000000000 0.2528 ns/op
通过观察 string 和 byte 的内存布局我们可以知道, 无法直接将 string 转为 []byte 「确实 cap 字段」, 但是可以直接将 []byte 转为 string
func ByteToStr(b []byte) string { return string(b) } func ByteToStrV2(b []byte) string { return *(*string)(unsafe.Pointer(&b)) } // go test -bench . func BenchmarkArrToStr(b *testing.B) { for i := 0; i < b.N; i++ { ByteToStr([]byte{65}) } } func BenchmarkArrToStrV2(b *testing.B) { for i := 0; i < b.N; i++ { ByteToStrV2([]byte{65}) } } //goos: darwin //goarch: amd64 //pkg: github.com/demo/lsafe //cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz //BenchmarkArrToStr-12 536188455 2.180 ns/op //BenchmarkArrToStrV2-12 1000000000 0.2526 ns/op
总结
本文介绍了如何使用 unsafe 包绕过类型检查, 直接操作内存。正如 go 作者对包的命名一样, 它是 unsafe 的, 随着 go 版本的迭代, 有些机制可能会发生变更。如无必要, 不应使用这个包。如果要使用 unsafe 包, 一定要理解清楚Pointer、uinptr、对齐系数等概念。
推荐学习:Golang教程
The above is the detailed content of Learn about the unsafe package in Golang. For more information, please follow other related articles on the PHP Chinese website!

Gooffersrobustfeaturesforsecurecoding,butdevelopersmustimplementsecuritybestpracticeseffectively.1)UseGo'scryptopackageforsecuredatahandling.2)Manageconcurrencywithsynchronizationprimitivestopreventraceconditions.3)SanitizeexternalinputstoavoidSQLinj

Go's error interface is defined as typeerrorinterface{Error()string}, allowing any type that implements the Error() method to be considered an error. The steps for use are as follows: 1. Basically check and log errors, such as iferr!=nil{log.Printf("Anerroroccurred:%v",err)return}. 2. Create a custom error type to provide more information, such as typeMyErrorstruct{MsgstringDetailstring}. 3. Use error wrappers (since Go1.13) to add context without losing the original error message,

ToeffectivelyhandleerrorsinconcurrentGoprograms,usechannelstocommunicateerrors,implementerrorwatchers,considertimeouts,usebufferedchannels,andprovideclearerrormessages.1)Usechannelstopasserrorsfromgoroutinestothemainfunction.2)Implementanerrorwatcher

In Go language, the implementation of the interface is performed implicitly. 1) Implicit implementation: As long as the type contains all methods defined by the interface, the interface will be automatically satisfied. 2) Empty interface: All types of interface{} types are implemented, and moderate use can avoid type safety problems. 3) Interface isolation: Design a small but focused interface to improve the maintainability and reusability of the code. 4) Test: The interface helps to unit test by mocking dependencies. 5) Error handling: The error can be handled uniformly through the interface.

Go'sinterfacesareimplicitlyimplemented,unlikeJavaandC#whichrequireexplicitimplementation.1)InGo,anytypewiththerequiredmethodsautomaticallyimplementsaninterface,promotingsimplicityandflexibility.2)JavaandC#demandexplicitinterfacedeclarations,offeringc

Toensureinitfunctionsareeffectiveandmaintainable:1)Minimizesideeffectsbyreturningvaluesinsteadofmodifyingglobalstate,2)Ensureidempotencytohandlemultiplecallssafely,and3)Breakdowncomplexinitializationintosmaller,focusedfunctionstoenhancemodularityandm

Goisidealforbeginnersandsuitableforcloudandnetworkservicesduetoitssimplicity,efficiency,andconcurrencyfeatures.1)InstallGofromtheofficialwebsiteandverifywith'goversion'.2)Createandrunyourfirstprogramwith'gorunhello.go'.3)Exploreconcurrencyusinggorout

Developers should follow the following best practices: 1. Carefully manage goroutines to prevent resource leakage; 2. Use channels for synchronization, but avoid overuse; 3. Explicitly handle errors in concurrent programs; 4. Understand GOMAXPROCS to optimize performance. These practices are crucial for efficient and robust software development because they ensure effective management of resources, proper synchronization implementation, proper error handling, and performance optimization, thereby improving software efficiency and maintainability.


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

PhpStorm Mac version
The latest (2018.2.1) professional PHP integrated development tool

mPDF
mPDF is a PHP library that can generate PDF files from UTF-8 encoded HTML. The original author, Ian Back, wrote mPDF to output PDF files "on the fly" from his website and handle different languages. It is slower than original scripts like HTML2FPDF and produces larger files when using Unicode fonts, but supports CSS styles etc. and has a lot of enhancements. Supports almost all languages, including RTL (Arabic and Hebrew) and CJK (Chinese, Japanese and Korean). Supports nested block-level elements (such as P, DIV),

MinGW - Minimalist GNU for Windows
This project is in the process of being migrated to osdn.net/projects/mingw, you can continue to follow us there. MinGW: A native Windows port of the GNU Compiler Collection (GCC), freely distributable import libraries and header files for building native Windows applications; includes extensions to the MSVC runtime to support C99 functionality. All MinGW software can run on 64-bit Windows platforms.

MantisBT
Mantis is an easy-to-deploy web-based defect tracking tool designed to aid in product defect tracking. It requires PHP, MySQL and a web server. Check out our demo and hosting services.

EditPlus Chinese cracked version
Small size, syntax highlighting, does not support code prompt function
