How to batch read txt files into DataFrame format in python

This time I will bring you pythonHow to batch read txt files into DataFrame format, python batch read txt files into DataFrame formatWhat are the precautions, here is the actual combat Let’s take a look at the case.

We sometimes process files in the same folder in batches, and hope to read them into a file to facilitate our calculation operations. For example, I have a series of txt files as shown below. How should I write them into a txt file and read them into DataFrame format?

First of all, we need to use the glob module. This python built-in module can be said to be very easy to use.

glob.glob('*.txt')

Get the following results:

##all.txt is the final result file I got. It can be seen that what is returned is a list containing txt file names. Of course, if there are only txt files under your folder, then you can use os.listdir() to get the same list

and then just read it Just pay attention to the encoding format of the txt file (you can use notepad to open Notepad to view) and the form of the separator. The complete code is as follows:

import os
import pandas
import codecs
import glob
import pandas as pd
os.getcwd()
os.chdir('D:\AAAASXQ\python study\data preprocessing')
def txtcombine()：
 
 files = glob.glob('*.txt')
 
 all = codecs.open('all.txt','a')
 
 for filename in flist: 
 print(filename) 
 fopen=codecs.open(filename,'r',encoding='utf-8') 
 lines=[] 
 lines=fopen.readlines() 
 fopen.close() 
 i=0 
 for line in lines: 
  for x in line: 
  all.write(x)
 #读取为DataFrame格式
 all1 = pd.read_csv('all.txt',sep=' ',encoding='GB2312')
 #保存为csv格式
 all1.to_csv('all.csv',encoding='GB2312')
 
if name == 'main': 
 txtcombine()

I believe you have mastered the method after reading the case in this article. More exciting Please pay attention to other related articles on php Chinese website!

Recommended reading:

How to convert a python string into a two-dimensional array

Detailed steps for using JS’s EventEmitter

The above is the detailed content of How to batch read txt files into DataFrame format in python. For more information, please follow other related articles on the PHP Chinese website!