


Python simulation method of splitting large files and implementing multi-thread processing
This article mainly introduces the method of Python to implement simulated splitting of large files and multi-threaded processing, involving Python file reading, splitting and multi-threading related operation skills. Friends in need can refer to the examples of this article
Describes how Python implements simulation of splitting large files and multi-threaded processing. Share it with everyone for your reference, the details are as follows:
#!/usr/bin/env python #--*-- coding:utf-8 --*-- from random import randint from time import ctime from time import sleep import queue import threading class MyTask(object): """具体的任务类""" def __init__(self, name): self.name = name self._work_time = randint(1, 5) def work(self): print("Task %s is start : %s, sleep time= %d" % (self.name, ctime(), self._work_time)) sleep(self._work_time) print("Task %s is end : %s" % (self.name, ctime())) class MyThread(threading.Thread): """多线程的类""" def __init__(self, my_queue): self.my_queue = my_queue super(MyThread, self).__init__() def run(self): while True: if self.my_queue.qsize() > 0: self.my_queue.get().work() else: break def print_split_line(num=30): print("*" * num) if __name__ == "__main__": print_split_line() import my_read_file # 分割文件 sf = my_read_file.SplitFiles(r"F:\multiple_thread_read_file.txt", line_count=300) file_num = sf.split_file() queue_length = file_num my_queue = queue.LifoQueue(queue_length) threads = [] for i in range(queue_length): file_name = sf.get_part_file_name(i) mt = MyTask(file_name) my_queue.put_nowait(mt) for i in range(queue_length): mtd = MyThread(my_queue) threads.append(mtd) for i in range(queue_length): threads[i].start() for i in range(queue_length): threads[i].join() print_split_line()
The above is the detailed content of Python simulation method of splitting large files and implementing multi-thread processing. For more information, please follow other related articles on the PHP Chinese website!

To maximize the efficiency of learning Python in a limited time, you can use Python's datetime, time, and schedule modules. 1. The datetime module is used to record and plan learning time. 2. The time module helps to set study and rest time. 3. The schedule module automatically arranges weekly learning tasks.

Python excels in gaming and GUI development. 1) Game development uses Pygame, providing drawing, audio and other functions, which are suitable for creating 2D games. 2) GUI development can choose Tkinter or PyQt. Tkinter is simple and easy to use, PyQt has rich functions and is suitable for professional development.

Python is suitable for data science, web development and automation tasks, while C is suitable for system programming, game development and embedded systems. Python is known for its simplicity and powerful ecosystem, while C is known for its high performance and underlying control capabilities.

You can learn basic programming concepts and skills of Python within 2 hours. 1. Learn variables and data types, 2. Master control flow (conditional statements and loops), 3. Understand the definition and use of functions, 4. Quickly get started with Python programming through simple examples and code snippets.

Python is widely used in the fields of web development, data science, machine learning, automation and scripting. 1) In web development, Django and Flask frameworks simplify the development process. 2) In the fields of data science and machine learning, NumPy, Pandas, Scikit-learn and TensorFlow libraries provide strong support. 3) In terms of automation and scripting, Python is suitable for tasks such as automated testing and system management.

You can learn the basics of Python within two hours. 1. Learn variables and data types, 2. Master control structures such as if statements and loops, 3. Understand the definition and use of functions. These will help you start writing simple Python programs.

How to teach computer novice programming basics within 10 hours? If you only have 10 hours to teach computer novice some programming knowledge, what would you choose to teach...

How to avoid being detected when using FiddlerEverywhere for man-in-the-middle readings When you use FiddlerEverywhere...


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

AI Hentai Generator
Generate AI Hentai for free.

Hot Article

Hot Tools

Dreamweaver Mac version
Visual web development tools

SublimeText3 English version
Recommended: Win version, supports code prompts!

Notepad++7.3.1
Easy-to-use and free code editor

Atom editor mac version download
The most popular open source editor

SAP NetWeaver Server Adapter for Eclipse
Integrate Eclipse with SAP NetWeaver application server.