In the above optimization, for every 500 users, some calculations will be performed and the results will be recorded in a disk file. I originally thought that by doing this, these results would be in the disk file and would no longer occupy memory; but in fact, the big pitfall of Python is that Python will not automatically clean up these memories. This is determined by its own implementation. There are many articles on the Internet explaining the specific reasons, so I won’t copy them here.
This article will post an experimental script of the author to illustrate that Python does have such a phenomenon of not releasing memory. In addition, a solution is also proposed, that is: del first, and then explicitly call gc.collect(). Script See below for specific effects.
Experimental environment one: Win 7, Python 2.7
from time import sleep, time import gc def mem(way=1): print time() for i in range(10000000): if way == 1: pass else: # way 2, 3 del i print time() if way == 1 or way == 2: pass else: # way 3 gc.collect() print time() if __name__ == "__main__": print "Test way 1: just pass" mem(way=1) sleep(20) print "Test way 2: just del" mem(way=2) sleep(20) print "Test way 3: del, and then gc.collect()" mem(way=3) sleep(20)
The running results are as follows:
Test way 1: just pass 1426688589.47 1426688590.25 1426688590.25 Test way 2: just del 1426688610.25 1426688611.05 1426688611.05 Test way 3: del, and then gc.collect() 1426688631.05 1426688631.85 1426688631.95
For way 1 and way 2, the results They are exactly the same. The peak memory consumption of the program is 326772KB. When sleeping for 20 seconds, the real-time memory consumption is 244820KB;
For way 3, the peak memory consumption of the program is the same as above, but the real-time memory consumption during sleep is only 6336KB.
Experimental environment two: Ubuntu 14.10, Python 2.7.3
Running results:
Test way 1: just pass 1426689577.46 1426689579.41 1426689579.41 Test way 2: just del 1426689599.43 1426689601.1 1426689601.1 Test way 3: del, and then gc.collect() 1426689621.12 1426689622.8 1426689623.11
ubuntu@my_machine:~$ ps -aux | grep test_mem Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html ubuntu 9122 10.0 6.0 270916 245564 pts/1 S+ 14:39 0:03 python test_mem.py ubuntu 9134 0.0 0.0 8104 924 pts/2 S+ 14:40 0:00 grep --color=auto test_mem ubuntu@my_machine:~$ ps -aux | grep test_mem Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html ubuntu 9122 10.0 6.0 270916 245564 pts/1 S+ 14:39 0:03 python test_mem.py ubuntu 9134 0.0 0.0 8104 924 pts/2 S+ 14:40 0:00 grep --color=auto test_mem ubuntu@my_machine:~$ ps -aux | grep test_mem Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html ubuntu 9122 11.6 0.1 30956 5608 pts/1 S+ 14:39 0:05 python test_mem.py
Conclusion:
The above description, when calling del, actually Python does not actually release the memory, but continues to put it in its memory pool; the memory is only released when gc.collect() is explicitly called.
Further:
In fact, go back to the script of the previous blog and let it introduce gc.collect(), and then write a monitoring script to monitor the memory consumption:
while ((1)); do ps -aux | sort -n -k5,6 | grep my_script; free; sleep 5; done
Result It was found that the memory will not be restored after each group of 500 users is executed, but will continue to be consumed until only about 70MB is left, and gc seems to work. In this environment, the machine uses Cloud instance, the total memory is 2G, and the available memory is about 1G. The common memory consumption of this script is 900M - 1G. In other words, for this script, gc does not take effect immediately, but when the available memory of the system drops from 1 - 1.2G to only about 70M, gc starts to take effect. This is indeed strange. I don’t know if it has something to do with the fact that the script uses gc.collect() in Thread, or if the function of gc is not controllable in the first place. The author has not done relevant experiments yet, and may continue to discuss it in the next blog.
However, it is certain that if gc.collect() is not used, the original script will exhaust the system memory and be killed. This is evident from the syslog.
The above is the detailed content of How to manually release Python memory. For more information, please follow other related articles on the PHP Chinese website!

本篇文章给大家带来了关于Python的相关知识,其中主要介绍了关于Seaborn的相关问题,包括了数据可视化处理的散点图、折线图、条形图等等内容,下面一起来看一下,希望对大家有帮助。

本篇文章给大家带来了关于Python的相关知识,其中主要介绍了关于进程池与进程锁的相关问题,包括进程池的创建模块,进程池函数等等内容,下面一起来看一下,希望对大家有帮助。

本篇文章给大家带来了关于Python的相关知识,其中主要介绍了关于简历筛选的相关问题,包括了定义 ReadDoc 类用以读取 word 文件以及定义 search_word 函数用以筛选的相关内容,下面一起来看一下,希望对大家有帮助。

VS Code的确是一款非常热门、有强大用户基础的一款开发工具。本文给大家介绍一下10款高效、好用的插件,能够让原本单薄的VS Code如虎添翼,开发效率顿时提升到一个新的阶段。

本篇文章给大家带来了关于Python的相关知识,其中主要介绍了关于数据类型之字符串、数字的相关问题,下面一起来看一下,希望对大家有帮助。

pythn的中文意思是巨蟒、蟒蛇。1989年圣诞节期间,Guido van Rossum在家闲的没事干,为了跟朋友庆祝圣诞节,决定发明一种全新的脚本语言。他很喜欢一个肥皂剧叫Monty Python,所以便把这门语言叫做python。

本篇文章给大家带来了关于Python的相关知识,其中主要介绍了关于numpy模块的相关问题,Numpy是Numerical Python extensions的缩写,字面意思是Python数值计算扩展,下面一起来看一下,希望对大家有帮助。


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

AI Hentai Generator
Generate AI Hentai for free.

Hot Article

Hot Tools

Zend Studio 13.0.1
Powerful PHP integrated development environment

Atom editor mac version download
The most popular open source editor

ZendStudio 13.5.1 Mac
Powerful PHP integrated development environment

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Dreamweaver Mac version
Visual web development tools
