Home  >  Article  >  Backend Development  >  Is python compressed files efficient?

Is python compressed files efficient?

anonymity
anonymityOriginal
2019-06-17 11:33:203004browse

python压缩文件的效率高吗?这里使用shell脚本和Python脚本同样压缩文件进行对比,通过时间和压缩比对比发现Python的压缩比可以,但是效率低耗时较多。

Is python compressed files efficient?

Shell使用系统tar

Python使用tarfile模块,Python不开多线程的情况两者做文件打包压缩对比。

Python代码:

#-*- coding: utf-8 -*- 
#!/usr/bin/evn python 
""" 
USAGE: 
        Bak_rsync_Python.py version 
""" 
import sys 
import time 
import os 
import shutil 
import subprocess 
import tarfile 
 
rundir=os.getcwd() 
src_dir='/data/rsync_center' 
bak_dir='/data/backup/game/test' 
today=time.strftime("%Y_%m_%d", time.localtime()) 
 
def main(version): 
        #set local var 
        md5="/sbin/md5" 
        bak_file="rsync_center_"+today+"_"+version+".tgz" #多个变量联合做变量 
        md5_file=rundir+"/md5/"+bak_file+".md5" 
        cmd="%s %s > %s" % (md5,bak_file,md5_file)     
        #print bak_file 
        #print md5_file 
 
        #tar src dir 
        os.chdir(src_dir) 
        tar=tarfile.open(bak_file,'w|gz') 
        tar.add(src_dir) 
        tar.close() 
 
        #tgz file md5 
        ret=subprocess.call(cmd,shell=True) #subprocess.call返回值是退出状态 
        if ret !=0:              
                print "md5 failed" 
                sys.exit(1) 
        #else: print ret 
 
        #move tgz to dst dir 
        shutil.move(bak_file,bak_dir) 
 
 
if __name__=='__main__': 
        try: 
                len(sys.argv)!=2 
        except: 
                print "Please give version" 
                print __doc__ 
        else: 
                print "start at",time.ctime() 
                version=sys.argv[1] 
                main(version) 
                print "end at",time.ctime()

shell脚本:

#!/bin/sh - 
# Bak_rsync_Shell.sh version 
 
shell_dir=`pwd` 
src_dir=/data/rsync_center 
bak_dir="/data/backup/game/test" 
log_file=${shell_dir}/rsync_center_bak.log 
Version=$1 
Today=`/bin/date +%Y_%m_%d` 
Fourteenday=`/bin/date -v -8d +%Y_%m_%d` 
bak_file="rsync_center_${Today}_${Version}.tgz" 
md5_file=${shell_dir}/md5/$bak_file.md5 
 
if [ $# -ne 1 ] 
        then    echo "Please give a version!" 
                break 
else 
        startdate=`date "+%Y-%m-%d %H:%M:%S"` 
        echo "rsync_center backup Start  at " $startdate 
        cd $src_dir 
        tar -zcf $bak_file ./ 
        /sbin/md5 $bak_file > $md5_file 
        mv $bak_file $bak_dir/ 
        enddate=`date "+%Y-%m-%d %H:%M:%S"` 
        echo "rsync_center backup Complete at" $enddate 
fi

运行对比:

#du -sh /data/rsync_center/
112M    /data/rsync_center/
#sh Bak_rsync_Shell.sh testShell
rsync_center backup Start  at  2012-03-15 14:53:37
tar: ./rsync_center_2012_03_15_testShell.tgz: Can't add archive to itself
rsync_center backup Complete at 2012-03-15 14:53:46

使用了9秒

#python Bak_rsync_Python.py testPython
start at Thu Mar 15 14:54:54 2012
end at Thu Mar 15 14:55:20 2012

使用了27秒

压缩大小差不多

#ll -h /data/backup/game/test/ | awk '{print$5,$9}'
49M rsync_center_2012_03_15_testPython.tgz
49M rsync_center_2012_03_15_testShell.tgz

python使用代码53行(取出注释也有40行),shell使用代码26行

python的逻辑更复杂,shell更简单一点。

 

可见python在单线程的情况下,对于系统基本文件处理,比如复制移动,打包压缩不如shell简单高效。逻辑上也不利于SA快速理解。所以,Python在基本的系统管理上,可能不如Shell.

The above is the detailed content of Is python compressed files efficient?. For more information, please follow other related articles on the PHP Chinese website!

Statement:
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn