How to Read Extremely Large Text Files Using Python

Faced with super large text files, ordinary text editors are unable to do anything? Python may be the ideal solution for you. This article will demonstrate how to use Python to efficiently read and process such files.

Get file

The experiment requires an oversized text file. This tutorial uses the hg38.txt file downloaded by the UCSC genomic Bioinformatics website as an example. We will use Python's open() function to open the file and get the file object.

Read line by line and write new file

The following code snippet demonstrates how to read the hg38.txt file line by line and write the first 500 lines to the output.txt file:

with open('hg38.txt', 'r') as input_file, open('output.txt', 'w') as output_file:
    for lines in range(500):
        line = input_file.readline()
        output_file.write(line)

The

with statement ensures that the file is automatically closed and resources are released.

Browse the large file directly

The above method requires writing the file content to a new file. To browse large files more flexibly, you can use the following code to display 50 lines of content at a time:

with open('hg38.txt','r') as input_file:
    while(1):
        for lines in range(50):
            print(input_file.readline())
        user_input = input('输入STOP退出，否则按回车键继续 ')
        if user_input.upper() == 'STOP':
            break

This code allows you to browse large files directly in the terminal and enter "STOP" to exit.

Python's efficiency makes it easy to handle super large text files.

This article was updated by Monty Shokeen. Monty is a full stack developer who is passionate about writing tutorials and learning new JavaScript libraries.

The above is the detailed content of How to Read Extremely Large Text Files Using Python. For more information, please follow other related articles on the PHP Chinese website!