[Basics] Python+Go - Let's find another way to improve computing performance

Hello friends, we all know that Python is a very productive language. I also like Python very much. I often use Python to help me complete the work with the highest efficiency. The biggest thing, but the performance of Python is an issue we have been criticizing, especially the big GIL lock. Sometimes it feels like eating a fly when I think about it.

Of course, most of our programs now are (IO) network-intensive programs, and Python is sufficient for it, but if we have existing projects or want to There are computationally intensive programming scenarios in the project to be developed. What should we do?

Some friends may have heard of Python C\C and use C\C to rewrite the computationally intensive parts of Python to improve performance.

Of course, this is a good solution, but we know that C\C has some learning costs. Is there a better solution?

##/2 Try to call Golang code in Python/

Later I was fortunate enough to come into contact with Golang. After using it for a while, the editor was thinking, it would be great if Python could call Go code. I really don’t want to learn C\C. After all, C\C There are still some barriers to pointers and freeing memory yourself, but Go is very convenient. It has automatic garbage collection, saves memory leaks, and has the advantages of inherently high concurrency.

After constantly consulting some information and stepping on some pitfalls, the hard work paid off and I finally found a suitable method, which I would like to share with you.

The most widely used Python interpreter at present is CPython. Python just has a module that can call C\C code. Go can also be compiled into a callable code similar to Python through some methods. C\C files.

##/3 Test environment/

System：windows

Python interpreter

：Python 3.7.6(64-bit)

Go compiler

:Go 1.14(64-bit)

/4 Performance comparison/

In order to better reflect it After optimization, we can roughly compare the difference between the two languages ​​in computationally intensive situations.

Test: Calculate the accumulation of one hundred million (100000000) to simulate a large number of calculations.

1) Python code

import time

def run(n):
    sum = 0
    for i in range(n):
        sum += i
    print(sum)


if __name__ == '__main__':
    startTime = time.time()
    run(100000000)
    endTime = time.time()
    print("耗时:", endTime - startTime)

can be seen Time consumption: about 10s, as shown in the figure below.

2) Go code

package main

import (
  "fmt"
  "time"
)

func run(n int) {
  sum := 0
  for i := 0; i < n; i++ {
    sum += i
  }
  fmt.Println(sum)
}
func main() {
  var startTime = time.Now()
  run(100000000)
  fmt.Println("耗时:", time.Since(startTime))
}

You can see that it takes about 200ms, as shown in the figure below.

3）测试结论

    我们可以看到，在计算方面，Python和Go是有很大的差距的，如果计算这一块能放在Go上就好了。

    别着急，马上开始。

/5 Go代码编译为Python可调用的.so文件/

1）Go代码

功能：接收传入的值进行累加，并且返回最终的累加值。

package main

import (
  "C" //C必须导入
)

//export run
func run(n int) int{
  /*
    必须要export 函数名
    //是注释的意思，相当于Python中的 #
    我也是第一次见注释还有作用，黑人三问好？？？
    固定写法
  */
  sum := 0
  for i := 0; i < n; i++ {
    sum += i
  }
    fmt.Println("我是Go代码,我跑完了,我的结果是:",sum)
  return sum
}

func main() {
  //main函数中什么都不要写，和包名main要对应
}

2）编译为.so文件供Python调用。

命令如下：

go build -buildmode=c-shared -o 输出的.so文件 go源文件

例如：

go build -buildmode=c-shared -o s1.so s1.go

会生成.h文件和.so文件，.so文件供Python调用，如下图所示：

3）Ptyhon调用so文件

将上述生成的.so文件复制到Python项目的同一级目录。

4）Python代码

依然是计算一个亿，关键部分由Go生成的.so执行。

from ctypes import *
import time


if __name__ == '__main__':
    startTime = time.time()

    s = CDLL("s1.so")  # 加载s1.so文件
    result = s.run(100000000)  # 调用Go生成的.so文件里面的run函数
    print("result:", result)

    endTime = time.time()
    print("耗时:", endTime - startTime)

可以看到耗时：0.11s左右，如下图所示。

5）为什么计算的耗时时间不一致，难道是计算错了？？？

    我们可以看到，虽然速度很快，但是Python在调用Go生成的.so文件之后，拿到的返回值竟然是错的，但是在Go中打印的确实对的，这是为什么呢？？？

    不要慌，问题不大！我们来计算一次稍微小一点的，上个100w的。

额，怎么还是错误。

6）我们再来计算更小一些的数，以10023为例，一个不零不整的数值。

This time you can see that the result is correct. But why is the 1 million result wrong? ? ?

We can know from the above that the calculation result of the .so file is correct. It may be that the conversion was wrong when python received it, but don’t worry, this chapter has a little bit It’s been a long time. This pitfall will definitely be solved in the next chapter, so stay tuned~

/6 Summary/

Maybe Python Go is not the best at improving key performance compared to Python C\C, but I think this method is the most worry-free. After all, the threshold of C\C is relatively High.

But having said that, the current performance may indeed be enough. After all, Python Go may be several times more efficient than Python C\C.

The above is the detailed content of [Basics] Python+Go - Let's find another way to improve computing performance. For more information, please follow other related articles on the PHP Chinese website!