擅长:python、mysql、java
<p>我编写了一个简单的helper来实现这一点—您可以使用batch_size、查询的对象类和处理查询中元素的回调来调用它。在</p>
<p>(注意,我使用的是djangoappengine,因此使用的是django查询格式,但是您可以根据需要修改它。)</p>
<pre><code>def loop_over_objects_in_batches(batch_size, object_class, callback):
logging.info("Calling batched loop with batch_size: %d, object_class: %s, callback: %s" % (batch_size, object_class, callback))
num_els = object_class.objects.all().count()
num_loops = num_els / batch_size
remainder = num_els - num_loops * batch_size
offset = 0
while offset < num_loops * batch_size:
logging.info("Processing batch (%d:%d)" % (offset, offset+batch_size))
query = object_class.objects.all()[offset:offset + batch_size]
for q in query:
callback(q)
offset = offset + batch_size
if remainder:
logging.info("Processing remainder batch (%d:-)" % offset)
query = object_class.objects.all()[offset:]
for q in query:
callback(q)
</code></pre>