<p><strong>源数据源:</strong></p>
<pre><code>In [172]: df
Out[172]:
id attributes attr2
0 255RSSSTCHL-QLTDGLZD-BLK {"color":"Black","hardware":"Goldtone"} {"aaa":"aaa", "bbb":"bbb"}
1 C3ACCRDNFLP-QLTDS-S-BLK {"size":"Small","color":"Black"} {"ccc":"ccc"}
</code></pre>
<p><strong>解决方案1:</strong></p>
^{pr2}$
<p><strong>解决方案2:</strong>感谢<a href="https://stackoverflow.com/questions/43769733/constructing-pandas-dataframe-from-csv-that-has-columns-of-dictionaries/43769792?noredirect=1#comment74581521_43769792">@DYZ for the hint</a>:</p>
<pre><code>import json
attr_cols = ['attributes','attr2']
def f(df, attr_col):
return df.join(df.pop(attr_col) \
.apply(lambda x: pd.Series(json.loads(x))))
for col in attr_cols:
df = f(df, col)
</code></pre>
<p><strong>结果:</strong></p>
<pre><code>In [175]: df
Out[175]:
id color hardware size aaa bbb ccc
0 255RSSSTCHL-QLTDGLZD-BLK Black Goldtone NaN aaa bbb NaN
1 C3ACCRDNFLP-QLTDS-S-BLK Black NaN Small NaN NaN ccc
</code></pre>
<p><000行:<2000行:</p>
<pre><code>In [198]: df = pd.concat([df] * 10**4, ignore_index=True)
In [199]: df.shape
Out[199]: (20000, 3)
In [201]: %paste
def f_ast(df, attr_col):
return df.join(df.pop(attr_col) \
.apply(lambda x: pd.Series(ast.literal_eval(x))))
def f_json(df, attr_col):
return df.join(df.pop(attr_col) \
.apply(lambda x: pd.Series(json.loads(x))))
## End pasted text
In [202]: %%timeit
...: for col in attr_cols:
...: f_ast(df.copy(), col)
...:
1 loop, best of 3: 33.1 s per loop
In [203]:
In [203]: %%timeit
...: for col in attr_cols:
...: f_json(df.copy(), col)
...:
1 loop, best of 3: 30 s per loop
In [204]: df.shape
Out[204]: (20000, 3)
</code></pre>