+ 3

os.walk vs os.scandir, which is faster?

Working on a duplicate file finding package https://pypi.org/project/dupliCat/ and wanted to speed things up a little bit. So according to this PEP https://peps.python.org/pep-0471/ scandir is faster than os.walk, I currently use python 3.10. I wrote a recursive function with os.scandir to traverse a directory path, benchmarked both the function and os.walk using the timeit module to see which one's faster but surprisingly, os.walk seems to be faster, idk whether it has been optimized to run faster in 3.10 or it's something else. What seems to be my main concern is that Idk how to benchmark the two functions in fetching all files in the directory. Should it be the time taken to get all the files as a list/tuple or the time taken for the functions to run. The recursive scantree function I wrote yields each file it comes across.

23rd Apr 2022, 12:11 AM
Divine Darkey
Divine Darkey - avatar
1 Odpowiedź
0
It's up to you. We haven't seen any code. Like it matters though because you will probably be the only one to use your own program. If you really wanna know, throw some imports in and time it. Either way, make a program that WORKS. That's what you should be working twords
23rd Apr 2022, 2:26 AM
Slick
Slick - avatar