将4d列表转换为Pandas数据帧的优化方法

Question

我正在尝试将一个四维列表转换成一个数据帧。我有一个解决方案，它使用三重嵌套的for循环来实现这一点，但它是高度不优化的-我觉得必须有一个更快的解决方案。我使用的代码如下：

import pandas as pd

master_df = pd.DataFrame(columns=('a1', 'a2', 'intersection', 'similarity'))

for i in master_list[0:2]:
    for x in i:
        for y in x:        
            t = [y[0], y[1], repr(y[2]), y[3]]
            master_df.loc[-1] = t
            master_df.index = master_df.index + 1
            master_df = master_df.sort_index()

这是我一直试图插入到数据帧中的master_list片段。你知道吗

master_list = [[[['residential property 42 holywell hill st. albans east of england al1 1bx',
'gnd flr 38 holywell hill st albans herts al1 1bx',
{'1bx', 'al1', 'albans', 'hill', 'holywell'},
0.5809767086589066],
['residential property 42 holywell hill st. albans east of england al1 1bx',
'62 holywell hill st albans herts al1 1bx',
{'1bx', 'al1', 'albans', 'hill', 'holywell'},
0.62250400597525191]]],
[[['aitchisons 2 holywell hill st. albans east of england al1 1bz',
'22 holywell hill st albans herts al1 1bz',
{'1bz', 'al1', 'albans', 'hill', 'holywell'},
0.64696827426453596],
['aitchisons 2 holywell hill st. albans east of england al1 1bz',
'24 holywell hill st albans herts al1 1bz',
{'1bz', 'al1', 'albans', 'hill', 'holywell'},
0.64660269146725069],
['aitchisons 2 holywell hill st. albans east of england al1 1bz',
'26 holywell hill st albans herts al1 1bz',
{'1bz', 'al1', 'albans', 'hill', 'holywell'},
0.64617599950794757],
['aitchisons 2 holywell hill st. albans east of england al1 1bz',
'20 holywell hill st albans herts al1 1bz',
{'1bz', 'al1', 'albans', 'hill', 'holywell'},
0.64798547824947428]]]]

有没有人有什么建议，把这个4d列表转换成熊猫数据帧在一个更。。。Python的方式？你知道吗

山姆

Answer 1

这里有一个解决方案：

• Flattenmaster_list
• 使用repr作为字典（我不认为你真的需要这个…）
• Reshape值必须有4列

代码：

def flatten(container):
    for i in container:
        if isinstance(i, (list,tuple)):
            for j in flatten(i):
                yield j
        else:
            yield i

def fix_dict(x):
    return repr(x) if isinstance(x, dict) else x

all_values = list(flatten(master_list))
all_values = [fix_dict(val) for val in all_values]

master_df = pd.DataFrame(np.reshape(all_values, (-1, 4)), columns = ['a1', 'a2', 'intersection', 'similarity'])

它给出了预期的输出。你知道吗