擅长:python、mysql、java
<p><strong><a href="https://docs.python.org/3.7/library/os.html#os.walk" rel="nofollow noreferrer">os.walk</a>已经返回一个生成器,该生成器将动态生成一个3元组(dirpath、dirnames、filenames)值,因此您只需成批生成filenames数组的片段。</strong></p>
<hr/>
<p><strong>这是一个例子:</strong></p>
<pre><code>import os
# Your root directory path
rootdir = r"Root"
#Your batch size
batch_size = 32
def walk_dirs(directory, batch_size):
walk_dirs_generator = os.walk(directory)
for dirname, subdirectories, filenames in walk_dirs_generator:
for i in range(0, len(filenames), batch_size):
# slice the filenames list 0-31, 32-64 and so on
yield [os.path.join(dirname, filename) for filename in filenames[i:i+batch_size]]
# Finally iterate over the walk_dirs function which itself returns a generator
for file_name_batch in walk_dirs(rootdir, batch_size):
for file_name in file_name_batch:
# Do some processing on the batch now
print (file_name)
pass
</code></pre>