假设我有一个大的,100兆字节的字典,我想把它做成磁盘书架。我正在使用pypar来利用MPI生成主列表的清理位。最好的办法是什么?示例:
# much earlier
masterDict = shelve.open( 'aShelveFile' )
# .. . . . .
# then we work out which parts of masterDict to keep
# and we put into aCleanDict
# then use mpi pypar to send the cleaned bits to the master rank
if pypar.rank() == 0:
tempdict = {}
for p in range(1,pypar.size()):
tempdict.append(pypar.receive(p))
for l1 in tempdict:
for l2 in l1:
realDict.append(l2)
for p in range(1,pypar.size()):
pypar.send(realDict,p)
# now realDict has all the cleaned bits
# which we send to the other hosts
else:
pypar.send(aCleanDict, 0 )
aCleanDict = pypar.receive( 0 )
# now to replace masterDict with aCleanDict
# note: cannot send shelve dictonaries using pypar
# insert stackover flow code here.
在这里,您可以搁置一个简单的字典,并通过键
myDict
使其易于访问:请注意,字典的内容必须是可腌制的,如任何要搁置的东西。在
如果要复制shelve中字典的结构,即没有
^{pr2}$myDict
键,而是将字典的键直接作为shelve的键,可以使用shelve的update
方法:shelve
接口与dict
接口有很大的重叠。在相关问题 更多 >
编程相关推荐