删除具有相同日期时间的行

Question

我怎么能在同一分钟内只划船？秒值无关紧要。似乎可以使用df.drop(index=2)之类的方法删除行，但是数据太多，需要逐个删除。你知道吗

import json
import math
from pandas.io.json import json_normalize
import pandas as pd
a=open(r'C:\work\kenkyuu\FITBIT\MyFitbitData (4)\AswadMdnor\user-site-export\heart_rate-2019-11- 
17.json')
b=json.load(a)
df = json_normalize(b)
df = df.rename(columns={'value.bpm':'bpm','value.confidence':'confidence'})
print(df)

    dateTime           bpm           confidence
11/17/19 02:28:05  113           0
11/17/19 02:28:17   70           0
11/17/19 02:28:31   70           0
11/17/19 02:28:42   70           0
11/17/19 02:29:29   70           0
11/17/19 02:29:46   70           0
11/17/19 02:30:43   70           0
11/17/19 02:32:13   70           0
11/17/19 02:49:39   70           0

我希望这个结果：

dateTime           bpm           confidence
11/17/19 02:28:05  113           0
11/17/19 02:29:29   70           0
11/17/19 02:30:43   70           0
11/17/19 02:32:13   70           0
11/17/19 02:49:39   70           0

以下是作为字典的数据，可用于重新创建数据帧：

{'dateTime': {0: '11/17/19 02:28:05', 1: '11/17/19 02:28:17', 2: '11/17/19 02:28:31', 3: '11/17/19 02:28:42', 4: '11/17/19 02:29:29', 5: '11/17/19 02:29:46', 6: '11/17/19 02:30:43', 7: '11/17/19 02:32:13', 8: '11/17/19 02:49:39', 9: '11/17/19 02:49:49', 10: '11/17/19 02:49:54', 11: '11/17/19 02:49:59', 12: '11/17/19 02:50:04', 13: '11/17/19 02:50:09', 14: '11/17/19 02:50:14', 15: '11/17/19 02:50:24', 16: '11/17/19 02:50:29', 17: '11/17/19 02:50:34', 18: '11/17/19 02:50:39', 19: '11/17/19 02:50:44', 20: '11/17/19 02:50:49', 21: '11/17/19 02:51:04', 22: '11/17/19 02:51:09', 23: '11/17/19 03:04:05', 24: '11/17/19 03:04:33', 25: '11/17/19 11:14:27', 26: '11/17/19 11:14:42', 27: '11/17/19 11:14:52', 28: '11/17/19 11:15:01', 29: '11/17/19 11:15:06', 30: '11/17/19 11:15:21'}, 'bpm': {0: 113, 1: 70, 2: 70, 3: 70, 4: 70, 5: 70, 6: 70, 7: 70, 8: 70, 9: 67, 10: 62, 11: 57, 12: 58, 13: 60, 14: 60, 15: 62, 16: 63, 17: 65, 18: 66, 19: 67, 20: 65, 21: 66, 22: 67, 23: 69, 24: 70, 25: 70, 26: 70, 27: 70, 28: 70, 29: 70, 30: 70}, 'confidence': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0, 8: 0, 9: 1, 10: 1, 11: 2, 12: 2, 13: 2, 14: 1, 15: 1, 16: 1, 17: 1, 18: 1, 19: 1, 20: 1, 21: 1, 22: 1, 23: 0, 24: 0, 25: 0, 26: 0, 27: 1, 28: 1, 29: 0, 30: 1}}

Answer 1

在这里，我们通过忽略秒来删除重复项，并获取它的索引值，以获得具有秒的原始时间，如下所示。你知道吗

>>> df.iloc[df['dateTime'].astype(str).str[:-2].drop_duplicates(keep='first').index,:]

---

输出：

            dateTime  bpm  confidence
0  11/17/19 02:28:05  113           0
4  11/17/19 02:29:29   70           0
6  11/17/19 02:30:43   70           0
7  11/17/19 02:32:13   70           0
8  11/17/19 02:49:39   70           0

Answer 2

我将舍入秒，然后检查重复项，然后子集或删除舍入日期时间的重复项

df[~df['dateTime'].dt.round('min').duplicated()]

Answer 3

我相信这个解决方案是最惯用的，尽管我会继续寻找。你知道吗

import pandas as pd

df = pd.read_csv('../resources/fitbit_time_data.csv', dtype={'bpm': np.int64, 'confidence': np.int64}, parse_dates=['date_time'], names=['date_time', 'bpm', 'confidence'], skiprows=[0])

df = df.resample(rule='min', on='date_time').first().dropna().reset_index(drop=True)

结果：

            date_time    bpm  confidence
0 2019-11-17 02:28:05  113.0         0.0
1 2019-11-17 02:29:29   70.0         0.0
2 2019-11-17 02:30:43   70.0         0.0
3 2019-11-17 02:32:13   70.0         0.0
4 2019-11-17 02:49:39   70.0         0.0

---

import pandas as pd

df = pd.read_csv('../resources/fitbit_time_data.csv', dtype={'bpm': np.int64, 'confidence': np.int64}, parse_dates=['date_time'], names=['date_time', 'bpm', 'confidence'], skiprows=[0])

df['minute'] = df.set_index('date_time').index.minute

df = df.loc[df['minute'].shift() != df['minute']]

数据帧结果：

            date_time  bpm  confidence  minute
0 2019-11-17 02:28:05  113           0      28
4 2019-11-17 02:29:29   70           0      29
6 2019-11-17 02:30:43   70           0      30
7 2019-11-17 02:32:13   70           0      32
8 2019-11-17 02:49:39   70           0      49

---