删除空行和数据帧中数据不匹配的行

Question

我有如下数据帧：

id        name      passenger       date
5abcdefg  Ani         True         2019-01-01
          # empty rows
Makassar  3          2019-12-01    TRUE
5bcdefgh  Budi       False         2019-11-10
5cdefghi  Rani       True          2019-11-11

我想删除空行和与其他架构不匹配的行，然后输出如下：

    id    name      passenger       date
5abcdefg  Ani         True         2019-01-01
5bcdefgh  Budi       False         2019-11-10
5cdefghi  Rani       True          2019-11-11

我试过这样，但没有成功：

if df['id'].empty:
    df['id'].replace('',np.nan, inplace=True)
    df.dropna(subset=['id'], inplace=True)
if not df.empty:
   # doing something
   return True
else:
   return False

对于信息，id字段总是以5开头

你知道我该怎么办吗？你知道吗

Answer 1

您可以将Boolean Indexing与str.startswith一起使用。你知道吗

这也会处理“id”列中的空值

df[df['id'].str.startswith('5', na=False)]

Answer 2

您可以执行此操作以仅获取符合您的架构的匹配：

df[(df.passenger.str.contains(r'(?:True|False)')) & (df.name.str.contains(r'(?:[a-zA-Z ]+)')) & (df.date.str.contains(r'(?:\d+-\d+-\d+)'))] 

输出：

         id  name passenger        date
0  5abcdefg   Ani      True  2019-01-01
2  5bcdefgh  Budi     False  2019-11-10
3  5cdefghi  Rani      True  2019-11-11

Answer 3

>>> import pandas as pd
>>> import numpy as np
>>> 
>>> df = pd.DataFrame(np.random.randn(10, 2), columns=list('AB'))
>>> df['Tenant'] = np.random.choice(['Babar', 'Rataxes', ''], 10)
>>> print df

          A         B   Tenant
0 -0.588412 -1.179306    Babar
1 -0.008562  0.725239         
2  0.282146  0.421721  Rataxes
3  0.627611 -0.661126    Babar
4  0.805304 -0.834214         
5 -0.514568  1.890647    Babar
6 -1.188436  0.294792  Rataxes
7  1.471766 -0.267807    Babar
8 -1.730745  1.358165  Rataxes
9  0.066946  0.375640         

Now we replace any empty strings in the Tenants column with np.nan objects, like so:

>>> df['Tenant'].replace('', np.nan, inplace=True)
>>> print df

          A         B   Tenant
0 -0.588412 -1.179306    Babar
1 -0.008562  0.725239      NaN
2  0.282146  0.421721  Rataxes
3  0.627611 -0.661126    Babar
4  0.805304 -0.834214      NaN
5 -0.514568  1.890647    Babar
6 -1.188436  0.294792  Rataxes
7  1.471766 -0.267807    Babar
8 -1.730745  1.358165  Rataxes
9  0.066946  0.375640      NaN
Now we can drop the null values:

>>> df.dropna(subset=['Tenant'], inplace=True)
>>> print df





          A         B   Tenant
0 -0.588412 -1.179306    Babar
2  0.282146  0.421721  Rataxes
3  0.627611 -0.661126    Babar
5 -0.514568  1.890647    Babar
6 -1.188436  0.294792  Rataxes
7  1.471766 -0.267807    Babar
8 -1.730745  1.358165  Rataxes