Problem. A directory inventory that needs to classify entries as regular files can start with os.listdir(), then ask os.path.isfile() about each returned name.[1]
Baseline. The baseline performs one path-based predicate per directory entry, so its scan is O(m) for m entries, with a name string and joined path handled in the loop.
Solution. os.scandir() yields DirEntry objects, whose is_file() method can use directory-entry type information when the operating system provides it.[1]
Measured result. On one Raspberry Pi run, both functions returned the same counts and scandir_isfile() took 7.55x to 8.02x less median time across directories containing 256, 1,024 and 4,096 regular files.
Reproduce the result
Complete code
from __future__ import annotationsimport jsonimport osimport platformimport sysimport tempfileimport timeitfrom pathlib import PathSIZES = (256, 1024, 4096)REPEAT = 7NUMBER = 1SUBDIRECTORY_DIVISOR = 8def listdir_isfile(root: str) -> int: return sum( 1 for name in os.listdir(root) if os.path.isfile(os.path.join(root, name)) )def scandir_isfile(root: str) -> int: count = 0 with os.scandir(root) as entries: for entry in entries: if entry.is_file(): count += 1 return countdef build_tree(root: Path, regular_files: int) -> None: for index in range(regular_files): (root / f"file-{index:05d}.dat").write_bytes(b"x") for index in range(regular_files // SUBDIRECTORY_DIVISOR): (root / f"dir-{index:05d}").mkdir()def median(values: list[float]) -> float: ordered = sorted(values) middle = len(ordered) // 2 return ordered[middle]def measure_case(regular_files: int) -> dict[str, object]: with tempfile.TemporaryDirectory(prefix="read0nly-scandir-") as temporary: root = Path(temporary) build_tree(root, regular_files) root_text = os.fspath(root) expected_entries = regular_files + regular_files // SUBDIRECTORY_DIVISOR baseline_count = listdir_isfile(root_text) solution_count = scandir_isfile(root_text) if baseline_count != regular_files or solution_count != regular_files: raise AssertionError("correctness_check_failed") listdir_isfile(root_text) scandir_isfile(root_text) baseline_samples = timeit.repeat( lambda: listdir_isfile(root_text), repeat=REPEAT, number=NUMBER, ) solution_samples = timeit.repeat( lambda: scandir_isfile(root_text), repeat=REPEAT, number=NUMBER, ) baseline_median_ms = median(baseline_samples) * 1000 solution_median_ms = median(solution_samples) * 1000 return { "regular_files": regular_files, "total_entries": expected_entries, "counts": {"listdir_isfile": baseline_count, "scandir_isfile": solution_count}, "listdir_isfile_ms": { "samples": [round(value * 1000, 6) for value in baseline_samples], "median": round(baseline_median_ms, 6), }, "scandir_isfile_ms": { "samples": [round(value * 1000, 6) for value in solution_samples], "median": round(solution_median_ms, 6), }, "baseline_over_solution": round(baseline_median_ms / solution_median_ms, 6), }def edge_checks() -> dict[str, list[int]]: with tempfile.TemporaryDirectory(prefix="read0nly-scandir-edge-") as temporary: root = Path(temporary) empty = [listdir_isfile(os.fspath(root)), scandir_isfile(os.fspath(root))] for index in range(3): (root / f"directory-{index}").mkdir() directories_only = [ listdir_isfile(os.fspath(root)), scandir_isfile(os.fspath(root)), ] return {"empty_directory": empty, "directories_only": directories_only}def main() -> None: cases = [measure_case(size) for size in SIZES] edges = edge_checks() all_counts_equal = all( case["counts"]["listdir_isfile"] == case["counts"]["scandir_isfile"] for case in cases ) and all(values == [0, 0] for values in edges.values()) result = { "python": sys.version.split()[0], "implementation": platform.python_implementation(), "platform": platform.platform(), "machine": platform.machine(), "input_generation": "fixed filenames, one-byte files, no random generator", "regular_file_sizes": list(SIZES), "subdirectories_per_case": [size // SUBDIRECTORY_DIVISOR for size in SIZES], "repeat": REPEAT, "number": NUMBER, "warmup_calls_per_function": 1, "cases": cases, "edge_checks": edges, "all_counts_equal": all_counts_equal, } print(json.dumps(result, indent=2, sort_keys=True))if __name__ == "__main__": main()
Command
python3 scandir_benchmark.py
Output
$ python3 scandir_benchmark.py{ "all_counts_equal": true, "cases": [ { "baseline_over_solution": 7.810427, "counts": { "listdir_isfile": 256, "scandir_isfile": 256 }, "listdir_isfile_ms": { "median": 3.198495, "samples": [ 3.557547, 3.194865, 3.129235, 3.198495, 3.14605, 3.716972, 3.335234 ] }, "regular_files": 256, "scandir_isfile_ms": { "median": 0.409516, "samples": [ 0.472219, 0.413238, 0.408923, 0.405904, 0.409516, 0.438793, 0.408552 ] }, "total_entries": 288 }, { "baseline_over_solution": 8.024132, "counts": { "listdir_isfile": 1024, "scandir_isfile": 1024 }, "listdir_isfile_ms": { "median": 13.219067, "samples": [ 13.219067, 13.22127, 13.119438, 13.191937, 13.131993, 13.917617, 13.553397 ] }, "regular_files": 1024, "scandir_isfile_ms": { "median": 1.647414, "samples": [ 1.739635, 1.660561, 1.68319, 1.647414, 1.644043, 1.638636, 1.642821 ] }, "total_entries": 1152 }, { "baseline_over_solution": 7.548357, "counts": { "listdir_isfile": 4096, "scandir_isfile": 4096 }, "listdir_isfile_ms": { "median": 56.862181, "samples": [ 57.333807, 57.466751, 56.098576, 56.52611, 56.668997, 57.978747, 56.862181 ] }, "regular_files": 4096, "scandir_isfile_ms": { "median": 7.533054, "samples": [ 7.539277, 7.599128, 7.528165, 7.507313, 7.533054, 7.604664, 7.490721 ] }, "total_entries": 4608 } ], "edge_checks": { "directories_only": [ 0, 0 ], "empty_directory": [ 0, 0 ] }, "implementation": "CPython", "input_generation": "fixed filenames, one-byte files, no random generator", "machine": "aarch64", "number": 1, "platform": "Linux-6.12.47+rpt-rpi-v8-aarch64-with-glibc2.41", "python": "3.13.5", "regular_file_sizes": [ 256, 1024, 4096 ], "repeat": 7, "subdirectories_per_case": [ 32, 128, 512 ], "warmup_calls_per_function": 1}exit=0
Environment
The run used CPython 3.13.5 on Linux-6.12.47+rpt-pi-v8-aarch64-with-glibc2.41, machine aarch64, with the Python standard library only. The benchmark creates each test tree inside tempfile.TemporaryDirectory(), whose context-manager form removes the directory when the block exits.[3]
Methodology
Each case creates the stated number of one-byte regular files and one subdirectory for every eight files. Filenames are fixed and there is no random generator or seed. The source checks both functions before timing, runs one warm-up call per function, then uses timeit.repeat() for seven samples with one call per sample.[2] Directory construction happens outside the timed functions. The reported ratio divides the baseline median by the scandir() median. The raw samples remain in the Output block.
The edge checks cover an empty directory and a directory containing only subdirectories. The two functions must return the same regular-file count before a timing is accepted. Python’s timeit temporarily disables garbage collection during timing by default.[2]
Limits of the experiment
This is one Raspberry Pi, one CPython build and one local temporary filesystem. The warm-up calls exclude a cold-cache comparison. The run does not count system calls, measure memory, create symlinks, traverse nested directories, use a network filesystem or include file opening and hashing. The timing therefore describes this shallow classification workload, not every directory scan.
The problem
The baseline can be written as a generator expression that counts each name when os.path.isfile(os.path.join(root, name)) returns true.
os.listdir() gives the names in a directory. The loop then joins each name to the directory path and applies a file test. The work is linear in the number of returned entries, including directories that will be rejected.
That complexity is already acceptable for a single small directory. The question is the repeated work inside the loop. The programme has a name, constructs a path from it, and asks the operating system about the path for every entry. The result is one count, so none of those names needs to survive the scan.
The solution
os.scandir() changes the object carried through the loop. Each item is a DirEntry, and entry.is_file() tests the entry directly. The documentation says that scandir() can improve code that also needs file type or attribute information because the returned objects expose information supplied by the operating system when available.[1]
The implementation uses with os.scandir(root) as entries, so the iterator is closed after the scan. The algorithm is still O(m): every directory entry is considered once. The improvement comes from the work performed for each entry, not from a different growth rate.
The benchmark deliberately keeps the claim narrow. It compares os.path.isfile(os.path.join(...)) with DirEntry.is_file() on the same generated tree. It does not claim that scandir() removes a fixed number of system calls on every filesystem. The source exposes a possible metadata advantage; the stopwatch records the complete Python call path used here.
What the measurement shows
| Regular files | Total entries | listdir() plus isfile() median | scandir() median | Baseline / solution |
|---|---|---|---|---|
| 256 | 288 | 3.198495 ms | 0.409516 ms | 7.810427x |
| 1,024 | 1,152 | 13.219067 ms | 1.647414 ms | 8.024132x |
| 4,096 | 4,608 | 56.862181 ms | 7.533054 ms | 7.548357x |
The correctness signal is simple. Every timed case produced the requested regular-file count in both implementations, and the empty and directory-only checks returned zero from both paths. The result is therefore a comparison of matching work, rather than a faster function that quietly counted a different set of entries.
The ratio stays in the same range as the directory grows. That is consistent with both functions visiting every entry, while the baseline does more path-oriented work per item in this run. It remains an observation about this host and filesystem. A different operating system, filesystem, Python build, directory shape or requested attribute could change the balance.
Practical applications
File inventories
A tool that prepares a list of regular files before hashing, archiving or indexing can use scandir() for the classification pass. The benchmark measures the classification only. Once the programme opens every file and reads its contents, I/O and hashing may dominate the total time, so the full pipeline needs its own measurement.
Log rotation checks
A shallow log directory often contains both files and subdirectories. DirEntry.is_file() can reject the directories while the programme applies a filename or age rule to the surviving entries. This article measures the type check, not deletion, timestamp filtering or compression; those operations can change the useful comparison.
Build and test discovery
A build helper can scan a known directory for regular source or test files before handing paths to another tool. scandir() suits that first classification step. Recursive discovery, ignore rules and later parsing are separate workloads, so a project should benchmark the complete discovery command rather than import this ratio unchanged.
Where it stops helping
The baseline remains a reasonable choice when the code needs only names, when directories are tiny, or when the next operation already has to perform a more expensive read. scandir() also does not turn a directory walk into a recursive traversal, and this run says nothing about network storage or symlink-heavy trees.
For this exact task, the practical rule is small: when a scan needs file-type tests as well as names, start with os.scandir() and measure the complete operation on the filesystem that matters.
Sources
[1] Python 3.13 os documentation: scandir, listdir and stat
