


FastDFS is very suitable for storing a large number of small files. Unfortunately, it does not support custom file names. The file name is a file_id generated based on the storage location after successful storage. Many application scenarios have to use custom file names. Without modifying the source code, you can add a database to the storage client fdfs_client to store the mapping relationship between the custom file name and the file_id of fastdfs to indirectly implement the custom file. For name access and access, here we choose reids. By the way, Taobao also has a file storage system TFS similar to FastDFS. For custom file names, it uses mysql to store mapping relationships. I think mysql itself is a bottleneck under high concurrent access, so it is used in this solution. redis.
Preparation work:
fastdfs environment installation... slightly... (official: https://code.google.com/p/fastdfs/)
redis environment installation... slightly... (official: http://redis.io/)
is implemented in python, so you need to install the python client of fastdfs (download: https://fastdfs.googlecode.com/files/fdfs_client-py-1.2.6.tar.gz)
Python’s redis client, go to https://pypi.python.org/pypi/redis to download
# -*- coding: utf-8 -*- import setting from fdfs_client.client import * from fdfs_client.exceptions import * from fdfs_client.connection import * import redis import time import logging import random logging.basicConfig(format='[%(levelname)s]: %(message)s', level=logging.DEBUG) logger = logging.getLogger(__name__) logger.setLevel(logging.DEBUG) class RedisError(Exception): def __init__(self, value): self.value = value def __str__(self): return repr(self.value) class fastdfsClient(Fdfs_client): def __init__(self): self.tracker_pool = ConnectionPool(**setting.fdfs_tracker) self.timeout = setting.fdfs_tracker['timeout'] return None def __del__(self): try: self.pool.destroy() self.pool = None except: pass class fastdfs(object): def __init__(self): ''' conf_file:配置文件 ''' self.fdfs_client = fastdfsClient() self.fdfs_redis = [] for i in setting.fdfs_redis_dbs: self.fdfs_redis.append(redis.Redis(host=i[0], port=i[1], db=i[2])) def store_by_buffer(self,buf,filename=None,file_ext_name = None): ''' buffer存储文件 参数: filename:自定义文件名,如果不指定,将远程file_id作为文件名 file_ext_name:文件扩展名(可选),如果不指定,将根据自定义文件名智能判断 返回值: { 'group':组名, 'file_id':不含组名的文件ID, 'size':文件尺寸, 'upload_time':上传时间 } ''' if filename and random.choice(self.fdfs_redis).exists(filename): logger.info('File(%s) exists.'%filename) return random.choice(self.fdfs_redis).hgetall(filename) t1 = time.time() # try: ret_dict = self.fdfs_client.upload_by_buffer(buf,file_ext_name) # except Exception,e: # logger.error('Error occurred while uploading: %s'%e.message) # return None t2 = time.time() logger.info('Upload file(%s) by buffer, time consume: %fs' % (filename,(t2 - t1))) for key in ret_dict: logger.debug('[+] %s : %s' % (key, ret_dict[key])) stored_filename = ret_dict['Remote file_id'] stored_filename_without_group = stored_filename[stored_filename.index('/')+1:] if not filename: filename =stored_filename_without_group vmp = {'group':ret_dict['Group name'],'file_id':stored_filename_without_group,'size':ret_dict['Uploaded size'],'upload_time':int(time.time()*1000)} try: for i in self.fdfs_redis: if not i.hmset(filename,vmp): raise RedisError('Save Failure') logger.info('Store file(%s) by buffer successful' % filename) except Exception,e: logger.error('Save info to Redis failure. rollback...') try: ret_dict = self.fdfs_client.delete_file(stored_filename) except Exception,e: logger.error('Error occurred while deleting: %s'%e.message) return None return vmp def remove(self,filename): ''' 删除文件, filename是用户自定义文件名 return True|False ''' fileinfo = random.choice(self.fdfs_redis).hgetall(filename) stored_filename = '%s/%s'%(fileinfo['group'],fileinfo['file_id']) try: ret_dict = self.fdfs_client.delete_file(stored_filename) logger.info('Remove stored file successful') except Exception,e: logger.error('Error occurred while deleting: %s'%e.message) return False for i in self.fdfs_redis: if not i.delete(filename): logger.error('Remove fileinfo in redis failure') logger.info('%s removed.'%filename) return True def download(self,filename): ''' 下载文件 返回二进制 ''' finfo = self.getInfo(filename) if finfo: ret = self.fdfs_client.download_to_buffer('%s/%s'%(finfo['group'],finfo['file_id'])) return ret['Content'] else: logger.debug('%s is not exists'%filename) return None def list(self,pattern='*'): ''' 列出文件列表 ''' return random.choice(self.fdfs_redis).keys(pattern) def getInfo(self,filename): ''' 获得文件信息 return:{ 'group':组名, 'file_id':不含组名的文件ID, 'size':文件尺寸, 'upload_time':上传时间 } ''' return random.choice(self.fdfs_redis).hgetall(filename)
Configuration:
# -*- coding: utf-8 -*- #fastdfs tracker, multiple tracker supported fdfs_tracker = { 'host_tuple':('192.168.2.233','192.168.2.234'), 'port':22122, 'timeout':30, 'name':'Tracker Pool' } #fastdfs meta db, multiple redisdb supported fdfs_redis_dbs = ( ('192.168.2.233',6379,0), ('192.168.2.233',6379,1) )

Python's flexibility is reflected in multi-paradigm support and dynamic type systems, while ease of use comes from a simple syntax and rich standard library. 1. Flexibility: Supports object-oriented, functional and procedural programming, and dynamic type systems improve development efficiency. 2. Ease of use: The grammar is close to natural language, the standard library covers a wide range of functions, and simplifies the development process.

Python is highly favored for its simplicity and power, suitable for all needs from beginners to advanced developers. Its versatility is reflected in: 1) Easy to learn and use, simple syntax; 2) Rich libraries and frameworks, such as NumPy, Pandas, etc.; 3) Cross-platform support, which can be run on a variety of operating systems; 4) Suitable for scripting and automation tasks to improve work efficiency.

Yes, learn Python in two hours a day. 1. Develop a reasonable study plan, 2. Select the right learning resources, 3. Consolidate the knowledge learned through practice. These steps can help you master Python in a short time.

Python is suitable for rapid development and data processing, while C is suitable for high performance and underlying control. 1) Python is easy to use, with concise syntax, and is suitable for data science and web development. 2) C has high performance and accurate control, and is often used in gaming and system programming.

The time required to learn Python varies from person to person, mainly influenced by previous programming experience, learning motivation, learning resources and methods, and learning rhythm. Set realistic learning goals and learn best through practical projects.

Python excels in automation, scripting, and task management. 1) Automation: File backup is realized through standard libraries such as os and shutil. 2) Script writing: Use the psutil library to monitor system resources. 3) Task management: Use the schedule library to schedule tasks. Python's ease of use and rich library support makes it the preferred tool in these areas.

To maximize the efficiency of learning Python in a limited time, you can use Python's datetime, time, and schedule modules. 1. The datetime module is used to record and plan learning time. 2. The time module helps to set study and rest time. 3. The schedule module automatically arranges weekly learning tasks.

Python excels in gaming and GUI development. 1) Game development uses Pygame, providing drawing, audio and other functions, which are suitable for creating 2D games. 2) GUI development can choose Tkinter or PyQt. Tkinter is simple and easy to use, PyQt has rich functions and is suitable for professional development.


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

AI Hentai Generator
Generate AI Hentai for free.

Hot Article

Hot Tools

EditPlus Chinese cracked version
Small size, syntax highlighting, does not support code prompt function

WebStorm Mac version
Useful JavaScript development tools

Safe Exam Browser
Safe Exam Browser is a secure browser environment for taking online exams securely. This software turns any computer into a secure workstation. It controls access to any utility and prevents students from using unauthorized resources.

SublimeText3 English version
Recommended: Win version, supports code prompts!

Zend Studio 13.0.1
Powerful PHP integrated development environment