如何在pandas中进行关键字映射

Question

我有关键词

India
Japan
United States
Germany
China

下面是示例数据帧

id    Address 
1     Chome-2-8 Shibakoen, Minato, Tokyo 105-0011, Japan
2     Arcisstraße 21, 80333 München, Germany
3     Liberty Street, Manhattan, New York, United States
4     30 Shuangqing Rd, Haidian Qu, Beijing Shi, China
5     Vaishnavi Summit,80feet Road,3rd Block,Bangalore, Karnataka, India

我的目标是

id    Address                                                          India Japan United States  Germany China    
1     Chome-2-8 Shibakoen, Minato, Tokyo 105-0011, Japan              0     1     0              0       0                  
2     Arcisstraße 21, 80333 München, Germany                          0     0     0              1       0
3     Liberty Street, Manhattan, New York, USA                        0     0     1              0       0
4     30 Shuangqing Rd, Haidian Qu, Beijing Shi, China                0     0     0              0       1
5     Vaishnavi Summit,80feet Road,Bangalore, Karnataka, India        1     0     0              0       0

基本思想是创建关键字检测器，我想使用str.contain和word2vec，但我不能得到逻辑

Answer 1

利用pd.get_dummies()：

countries = df.Address.str.extract('(India|Japan|United States|Germany|China)', expand = False)
dummies = pd.get_dummies(countries)
pd.concat([df,dummies],axis = 1)

另外，最直接的方法是将国家列在一个列表中，并使用for循环

countries = ['India','Japan','United States','Germany','China']
for c in countries:
    df[c] = df.Address.str.contains(c) * 1

但如果你有大量的数据和国家，这可能会很慢。你知道吗

Answer 2

In [58]: df = df.join(df.Address.str.extract(r'.*,(.*)', expand=False).str.get_dummies())

In [59]: df
Out[59]:
   id                                            Address   China   Germany   India   Japan   United States
0   1  Chome-2-8 Shibakoen, Minato, Tokyo 105-0011, J...       0         0       0       1               0
1   2             Arcisstra?e 21, 80333 Munchen, Germany       0         1       0       0               0
2   3  Liberty Street, Manhattan, New York, United St...       0         0       0       0               1
3   4   30 Shuangqing Rd, Haidian Qu, Beijing Shi, China       1         0       0       0               0
4   5  Vaishnavi Summit,80feet Road,3rd Block,Bangalo...       0         0       1       0               0

注意：如果country不在Address列的最后一个位置，或者country name包含,，则此方法不起作用

Answer 3

from numpy.core.defchararray import find

kw = 'India|Japan|United States|Germany|China'.split('|')
a = df.Address.values.astype(str)[:, None]

df.join(
    pd.DataFrame(
        find(a, kw) >= 0,
        df.index, kw,
        dtype=int
    )
)

   id                        Address  India  Japan  United States  Germany  China
0   1  Chome-2-8 Shibakoen, Minat...      0      1              0        0      0
1   2  Arcisstraße 21, 80333 Münc...      0      0              0        1      0
2   3  Liberty Street, Manhattan,...      0      0              1        0      0
3   4  30 Shuangqing Rd, Haidian ...      0      0              0        0      1
4   5  Vaishnavi Summit,80feet Ro...      1      0              0        0      0