search
HomeBackend DevelopmentGolangLearn about the unsafe package in Golang

In some low-level libraries, you often see the use of unsafe packages. This article will take you to understand the unsafe package in Golang, introduce the role of the unsafe package and how to use Pointer. I hope it will be helpful to you!

Learn about the unsafe package in Golang

The unsafe package provides some operations that can bypass go's type safety check, thereby directly operating memory addresses and doing some tricky operations. The running environment of the sample code is go version go1.18 darwin/amd64

Memory alignment

The unsafe package provides the Sizeof method to obtain the memory size occupied by variables. "Does not include the memory size of the variable pointed by the pointer", Alignof obtains the memory alignment coefficient, the specific memory alignment rules can be googled by yourself.

type demo1 struct {
   a bool  // 1
   b int32 // 4
   c int64 // 8
}

type demo2 struct {
   a bool  // 1
   c int64 // 8
   b int32 // 4
}

type demo3 struct { // 64 位操作系统, 字长 8
   a *demo1 // 8
   b *demo2 // 8
}

func MemAlign() {
   fmt.Println(unsafe.Sizeof(demo1{}), unsafe.Alignof(demo1{}), unsafe.Alignof(demo1{}.a), unsafe.Alignof(demo1{}.b), unsafe.Alignof(demo1{}.c)) // 16,8,1,4,8
   fmt.Println(unsafe.Sizeof(demo2{}), unsafe.Alignof(demo2{}), unsafe.Alignof(demo2{}.a), unsafe.Alignof(demo2{}.b), unsafe.Alignof(demo2{}.c)) // 24,8,1,4,8
   fmt.Println(unsafe.Sizeof(demo3{}))                                                                                                           // 16
}                                                                                                         // 16}复制代码

From the above case, you can see that demo1 and demo2 contain the same Attributes, only the order of the defined attributes is different, but the memory size of the variables is different. This is because memory alignment occurs.

When the computer is processing tasks, it will process data in units of specific word lengths "For example: 32-bit operating system, word length is 4; 64-bit operating system, word length is 8". Then, when reading data, the unit is also based on word length. For example: For 64-bit operating systems, the number of bytes read by the program at one time is a multiple of 8. The following is the layout of demo1 under non-memory alignment and memory alignment:

Non-memory alignment:

The variable c will be placed in different word lengths, and the CPU needs to read at the same time Read twice and process the results of both times at the same time to get the value of c. Although this method saves memory space, it will increase processing time.

Memory alignment:

Memory alignment adopts a scheme that can avoid the situation of the same non-memory alignment, but it will take up some extra space "space for time". You can google for specific memory alignment rules.

Learn about the unsafe package in Golang

Unsafe Pointer

You can declare a pointer type in go. The type here is safe pointer, that is, it is necessary to make it clear that the pointer points to Type, if the types do not match, an error will occur during compilation. As in the following example, the compiler will think that MyString and string are different types and cannot be assigned.

func main() {
   type MyString string
   s := "test"
   var ms MyString = s // Cannot use 's' (type string) as the type MyString
   fmt.Println(ms)
}

Is there a type that can point to variables of any type? You can use unsfe.Pointer, which can point to any type of variable. Through the declaration of Pointer, we can know that it is a pointer type, pointing to the address of the variable. The value corresponding to the specific address can be converted through uinptr. Pointer has the following four special operations:

  • Any type of pointer can be converted into a Pointer type
  • Pointer type variables can be converted into any type of pointer
  • uintptr type variables can be converted to Pointer type
  • Pointer type variables can be converted to uintprt type
type Pointer *ArbitraryType

// uintptr is an integer type that is large enough to hold the bit pattern of
// any pointer.
type uintptr uintptr

func main() {
   d := demo1{true, 1, 2}
   p := unsafe.Pointer(&d)                // 任意类型的指针可以转换为 Pointer 类型
   pa := (*demo1)(p)                      // Pointer 类型变量可以转换成 demo1 类型的指针
   up := uintptr(p)                       // Pointer 类型的变量可以转换成 uintprt 类型
   pu := unsafe.Pointer(up)               // uintptr 类型的变量可以转换成 Pointer 类型; 当 GC 时, d 的地址可能会发生变更, 因此, 这里的 up 可能会失效
   fmt.Println(d.a, pa.a, (*demo1)(pu).a) // true true true

}

Six ways to use Pointer

The official documentation provides six usage postures of Pointer.

  1. Convert *T1 to *T2 through Pointer

Pointer directly points to a piece of memory, so this piece of memory can be Convert the memory address to any type. It should be noted here that T1 and T2 need to have the same memory layout and there will be abnormal data.

func main() {
   type myStr string
   ms := []myStr{"1", "2"}
   //ss := ([]string)(ms) Cannot convert an expression of the type '[]myStr' to the type '[]string'
   ss := *(*[]string)(unsafe.Pointer(&ms)) // 将 pointer 指向的内存地址直接转换成 *[]string
   fmt.Println(ms, ss)
}

Learn about the unsafe package in Golang

What happens if the memory layout of T1 and T2 is different? In the example below, although demo1 and demo2 contain the same structure, they have different memory layouts due to memory alignment. When converting Pointer, 24 "sizeof" bytes will be read starting from the address of demo1, and the conversion will be performed according to the memory alignment rules of demo2. The first byte will be converted to a:true, and 8-16 bytes will be converted to c. :2, 16-24 bytes are beyond the range of demo1, but can still be read directly, and the unexpected value b:17368000 is obtained.

type demo1 struct {
   a bool  // 1
   b int32 // 4
   c int64 // 8
}

type demo2 struct {
   a bool  // 1
   c int64 // 8
   b int32 // 4
}

func main() {
   d := demo1{true, 1, 2}
   pa := (*demo2)(unsafe.Pointer(&d)) // Pointer 类型变量可以转换成 demo2 类型的指针
   fmt.Println(pa.a, pa.b, pa.c) // true, 17368000, 2, 
}

Learn about the unsafe package in Golang

  1. Convert Pointer type to uintptr type "Uinptr should not be converted to Pointer"

Pointer is a pointer type that can point to any variable. You can print the address of the variable pointed by Pointer by converting Pointer to uintptr. Additionally: uintptr should not be converted to Pointer. Take the following example: When GC occurs, the address of d may change, then up points to the wrong memory due to unsynchronized updates.

func main() {
   d := demo1{true, 1, 2}
   p := unsafe.Pointer(&d)
   up := uintptr(p)
   fmt.Printf("uintptr: %x, ptr: %p \n", up, &d) // uintptr: c00010c010, ptr: 0xc00010c010
   fmt.Println(*(*demo1)(unsafe.Pointer(up)))    // 不允许
}
  1. 通过算数计算将 Pointer 转换为 uinptr 再转换回 Pointer

当 Piointer 指向一个结构体时, 可以通过此方式获取到结构体内部特定属性的 Pointer。

func main() {
   d := demo1{true, 1, 2}
   // 等同于 unsafe.Pointer(&d.b); unsafe.Add(unsafe.Pointer(&d), unsafe.Offsetof(d.b))
   pb := unsafe.Pointer(uintptr(unsafe.Pointer(&d)) + unsafe.Offsetof(d.b))
   fmt.Println(pb)
}
  1. 当调用 syscall.Syscall 的时候, 可以讲 Pointer 转换为 uintptr

前面说过, 由于 GC 会导致变量的地址发生变更, 因此不可以直接处理 uintptr。但是, 在调用 syscall.Syscall 时候可以允许传递一个 uintptr, 这里可以简单理解为是编译器做了特殊处理, 来保证 uintptr 是安全的。

  • 调用方式:
  •   syscall.Syscall(SYS_READ, uintptr( fd ), uintptr(unsafe.Pointer(p)), uintptr(n))

下面这种方式是不允许的:

u := uintptr(unsafe.Pointer(p)) // 不应该保存到一个变量上 syscall.Syscall(SYS_READ, uintptr( fd ), u, uintptr(n))

  1. 可以将 reflect.Value.Pointer 或 reflect.Value.UnsafeAddr 的结果「uintptr」转换为 Pointer

在 reflect 包中的 Value.Pointer 和 Value.UnsafeAddr 直接返回了地址对应的值「uintptr」, 可以直接将其结果转为 Pointer

func main() {
   d := demo1{true, 1, 2}
   // 等同于 unsafe.Pointer(&d.b); unsafe.Add(unsafe.Pointer(&d), unsafe.Offsetof(d.b))
   pb := unsafe.Pointer(uintptr(unsafe.Pointer(&d)) + unsafe.Offsetof(d.b))
   // up := reflect.ValueOf(&d.b).Pointer(), pc := unsafe.Pointer(up); 不安全, 不应存储到变量中
   pc := unsafe.Pointer(reflect.ValueOf(&d.b).Pointer())
   fmt.Println(pb, pc)
}
  1. 可以将 reflect.SliceHeader 或者 reflect.StringHeader 的 Data 字段与 Pointer 相互转换

SliceHeader 和 StringHeader 其实是 slice 和 string 的内部实现, 里面都包含了一个字段 Data「uintptr」, 存储的是指向 []T 的地址, 这里之所以使用 uinptr 是为了不依赖 unsafe 包。

func main() {
   s := "a"
   hdr := (*reflect.StringHeader)(unsafe.Pointer(&s)) // *string to *StringHeader
   fmt.Println(*(*[1]byte)(unsafe.Pointer(hdr.Data))) // 底层存储的是 utf 编码后的 byte 数组
   arr := [1]byte{65}
   hdr.Data = uintptr(unsafe.Pointer(&arr))
   hdr.Len = len(arr)
   ss := *(*string)(unsafe.Pointer(hdr))
   fmt.Println(ss) // A
   arr[0] = 66
   fmt.Println(ss) //B
}

应用

string、byte 转换

在业务上, 经常遇到 string 和 []byte 的相互转换。我们知道, string 底层其实也是存储的一个 byte 数组, 可以通过 reflect 直接获取 string 指向的 byte 数组, 赋值给 byte 切片, 避免内存拷贝。

func StrToByte(str string) []byte {
   return []byte(str)
}

func StrToByteV2(str string) (b []byte) {
   bh := (*reflect.SliceHeader)(unsafe.Pointer(&b))
   sh := (*reflect.StringHeader)(unsafe.Pointer(&str))
   bh.Data = sh.Data
   bh.Cap = sh.Len
   bh.Len = sh.Len
   return b
}

// go test -bench .
func BenchmarkStrToArr(b *testing.B) {
   for i := 0; i < b.N; i++ {
      StrToByte(`{"f": "v"}`)
   }
}

func BenchmarkStrToArrV2(b *testing.B) {
   for i := 0; i < b.N; i++ {
      StrToByteV2(`{"f": "v"}`)
   }
}

//goos: darwin
//goarch: amd64
//pkg: github.com/demo/lsafe
//cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
//BenchmarkStrToArr-12            264733503                4.311 ns/op
//BenchmarkStrToArrV2-12          1000000000               0.2528 ns/op

通过观察 string 和 byte 的内存布局我们可以知道, 无法直接将 string 转为 []byte 「确实 cap 字段」, 但是可以直接将 []byte 转为 string

Learn about the unsafe package in Golang

func ByteToStr(b []byte) string {
   return string(b)
}

func ByteToStrV2(b []byte) string {
   return *(*string)(unsafe.Pointer(&b))
}

// go test -bench .
func BenchmarkArrToStr(b *testing.B) {
   for i := 0; i < b.N; i++ {
      ByteToStr([]byte{65})
   }
}

func BenchmarkArrToStrV2(b *testing.B) {
   for i := 0; i < b.N; i++ {
      ByteToStrV2([]byte{65})
   }
}

//goos: darwin
//goarch: amd64
//pkg: github.com/demo/lsafe
//cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
//BenchmarkArrToStr-12            536188455                2.180 ns/op
//BenchmarkArrToStrV2-12          1000000000               0.2526 ns/op

总结

本文介绍了如何使用 unsafe 包绕过类型检查, 直接操作内存。正如 go 作者对包的命名一样, 它是 unsafe 的, 随着 go 版本的迭代, 有些机制可能会发生变更。如无必要, 不应使用这个包。如果要使用 unsafe 包, 一定要理解清楚Pointer、uinptr、对齐系数等概念。

推荐学习:Golang教程

The above is the detailed content of Learn about the unsafe package in Golang. For more information, please follow other related articles on the PHP Chinese website!

Statement
This article is reproduced at:掘金社区. If there is any infringement, please contact admin@php.cn delete
Security Considerations When Developing with GoSecurity Considerations When Developing with GoApr 27, 2025 am 12:18 AM

Gooffersrobustfeaturesforsecurecoding,butdevelopersmustimplementsecuritybestpracticeseffectively.1)UseGo'scryptopackageforsecuredatahandling.2)Manageconcurrencywithsynchronizationprimitivestopreventraceconditions.3)SanitizeexternalinputstoavoidSQLinj

Understanding Go's error InterfaceUnderstanding Go's error InterfaceApr 27, 2025 am 12:16 AM

Go's error interface is defined as typeerrorinterface{Error()string}, allowing any type that implements the Error() method to be considered an error. The steps for use are as follows: 1. Basically check and log errors, such as iferr!=nil{log.Printf("Anerroroccurred:%v",err)return}. 2. Create a custom error type to provide more information, such as typeMyErrorstruct{MsgstringDetailstring}. 3. Use error wrappers (since Go1.13) to add context without losing the original error message,

Error Handling in Concurrent Go ProgramsError Handling in Concurrent Go ProgramsApr 27, 2025 am 12:13 AM

ToeffectivelyhandleerrorsinconcurrentGoprograms,usechannelstocommunicateerrors,implementerrorwatchers,considertimeouts,usebufferedchannels,andprovideclearerrormessages.1)Usechannelstopasserrorsfromgoroutinestothemainfunction.2)Implementanerrorwatcher

How do you implement interfaces in Go?How do you implement interfaces in Go?Apr 27, 2025 am 12:09 AM

In Go language, the implementation of the interface is performed implicitly. 1) Implicit implementation: As long as the type contains all methods defined by the interface, the interface will be automatically satisfied. 2) Empty interface: All types of interface{} types are implemented, and moderate use can avoid type safety problems. 3) Interface isolation: Design a small but focused interface to improve the maintainability and reusability of the code. 4) Test: The interface helps to unit test by mocking dependencies. 5) Error handling: The error can be handled uniformly through the interface.

Comparing Go Interfaces to Interfaces in Other Languages (e.g., Java, C#)Comparing Go Interfaces to Interfaces in Other Languages (e.g., Java, C#)Apr 27, 2025 am 12:06 AM

Go'sinterfacesareimplicitlyimplemented,unlikeJavaandC#whichrequireexplicitimplementation.1)InGo,anytypewiththerequiredmethodsautomaticallyimplementsaninterface,promotingsimplicityandflexibility.2)JavaandC#demandexplicitinterfacedeclarations,offeringc

init Functions and Side Effects: Balancing Initialization with Maintainabilityinit Functions and Side Effects: Balancing Initialization with MaintainabilityApr 26, 2025 am 12:23 AM

Toensureinitfunctionsareeffectiveandmaintainable:1)Minimizesideeffectsbyreturningvaluesinsteadofmodifyingglobalstate,2)Ensureidempotencytohandlemultiplecallssafely,and3)Breakdowncomplexinitializationintosmaller,focusedfunctionstoenhancemodularityandm

Getting Started with Go: A Beginner's GuideGetting Started with Go: A Beginner's GuideApr 26, 2025 am 12:21 AM

Goisidealforbeginnersandsuitableforcloudandnetworkservicesduetoitssimplicity,efficiency,andconcurrencyfeatures.1)InstallGofromtheofficialwebsiteandverifywith'goversion'.2)Createandrunyourfirstprogramwith'gorunhello.go'.3)Exploreconcurrencyusinggorout

Go Concurrency Patterns: Best Practices for DevelopersGo Concurrency Patterns: Best Practices for DevelopersApr 26, 2025 am 12:20 AM

Developers should follow the following best practices: 1. Carefully manage goroutines to prevent resource leakage; 2. Use channels for synchronization, but avoid overuse; 3. Explicitly handle errors in concurrent programs; 4. Understand GOMAXPROCS to optimize performance. These practices are crucial for efficient and robust software development because they ensure effective management of resources, proper synchronization implementation, proper error handling, and performance optimization, thereby improving software efficiency and maintainability.

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

PhpStorm Mac version

PhpStorm Mac version

The latest (2018.2.1) professional PHP integrated development tool

mPDF

mPDF

mPDF is a PHP library that can generate PDF files from UTF-8 encoded HTML. The original author, Ian Back, wrote mPDF to output PDF files "on the fly" from his website and handle different languages. It is slower than original scripts like HTML2FPDF and produces larger files when using Unicode fonts, but supports CSS styles etc. and has a lot of enhancements. Supports almost all languages, including RTL (Arabic and Hebrew) and CJK (Chinese, Japanese and Korean). Supports nested block-level elements (such as P, DIV),

MinGW - Minimalist GNU for Windows

MinGW - Minimalist GNU for Windows

This project is in the process of being migrated to osdn.net/projects/mingw, you can continue to follow us there. MinGW: A native Windows port of the GNU Compiler Collection (GCC), freely distributable import libraries and header files for building native Windows applications; includes extensions to the MSVC runtime to support C99 functionality. All MinGW software can run on 64-bit Windows platforms.

MantisBT

MantisBT

Mantis is an easy-to-deploy web-based defect tracking tool designed to aid in product defect tracking. It requires PHP, MySQL and a web server. Check out our demo and hosting services.

EditPlus Chinese cracked version

EditPlus Chinese cracked version

Small size, syntax highlighting, does not support code prompt function