How to Efficiently List Files from Recursive Subfolders in Python?

Recursive Subfolder Search to List Files: How to Refine the Process

When searching through subfolders and constructing a list of specific file types, it's essential to ensure that the subfolder variable points to the correct folder. The following code snippet demonstrates this issue:

<code class="python">for root, subFolder, files in os.walk(PATH):
    for item in files:
        if item.endswith(".txt"):
            fileNamePath = str(os.path.join(root, subFolder, item))</code>

Here, the subFolder variable contains a list of subfolders instead of the folder where the ITEM file resides. To rectify this, we can use the dirpath, represented by the root variable, as it holds the correct directory path. This modified code snippet addresses the issue:

<code class="python">import os
result = [os.path.join(dp, f) for dp, dn, filenames in os.walk(PATH) for f in filenames if os.path.splitext(f)[1] == '.txt']</code>

Another elegant approach is to utilize the glob module, which efficiently selects files based on their extensions:

<code class="python">from glob import glob
result = [y for x in os.walk(PATH) for y in glob(os.path.join(x[0], '*.txt'))]</code>

Python 3.4 and above offer a generator version of the glob-based solution:

<code class="python">from itertools import chain
result = (chain.from_iterable(glob(os.path.join(x[0], '*.txt')) for x in os.walk('.')))</code>

Lastly, for Python 3.4 , a modern approach using the pathlib module is:

<code class="python">from pathlib import Path
result = list(Path(".").rglob("*.[tT][xX][tT]"))</code>

The above is the detailed content of How to Efficiently List Files from Recursive Subfolders in Python?. For more information, please follow other related articles on the PHP Chinese website!