是否有一个函数来查找日期时间之间的差异？

Question

我有多个数据帧，它们可以有相同的时间戳（也是+-1秒），其中有毫秒。因此，当它们都在新的数据帧中时，我想过滤掉它们之间相差超过1秒的行

有没有类似于dftogether['unique'] = np.ediff1d(dftogether['DateTime']的函数可以处理时间戳？你知道吗

我目前的解决方案是可行的，但我正在寻找一种合适的方法。 假设我有3个数据帧，df1，df2和df3。对于每个数据帧，我执行以下操作：

df1['DateTime'] = df1['DateTime'].apply(lambda 
x: x.strftime('%Y%d%m%H%M%S'))
df1['DateTime']= df1['DateTime'].astype(np.int64)

它把我的DateTime变成了int，所以我可以这样做：

dftogether= pd.concat(z, sort=True)
dftogether= dftogether.sort_values('DateTime')
dftogether['unique'] = np.ediff1d(dftogether['DateTime'], to_begin=20181211150613411) <1
dftogether= dftogether[dftogether.unique == False]

然后我把int转换回datetime

 dftogether['DateTime'] = dftogether['DateTime'].apply(lambda x: pd.to_datetime(str(x), format='%Y%d%m%H%M%S'))

我不知道如何为时间戳创建示例数据，所以我将只复制粘贴部分数据帧。你知道吗

df1型

737    2018-12-18 12:37:19.717
738    2018-12-18 12:37:21.936
739    2018-12-18 12:37:22.841
740    2018-12-18 12:37:23.144
877    2018-12-18 12:40:53.268
878    2018-12-18 12:40:56.597
879    2018-12-18 12:40:56.899
880    2018-12-18 12:40:57.300
968    2018-12-18 12:43:31.411
969    2018-12-18 12:43:36.150
970    2018-12-18 12:43:36.452

df2型

691    2018-12-18 12:35:23.612
692    2018-12-18 12:35:25.627
788    2018-12-18 12:38:33.248
789    2018-12-18 12:38:33.553
790    2018-12-18 12:38:34.759
866    2018-12-18 12:40:29.487
867    2018-12-18 12:40:31.199
868    2018-12-18 12:40:32.206

df3型

699    2018-12-18 12:35:42.452
701    2018-12-18 12:35:45.081
727    2018-12-18 12:36:47.466
730    2018-12-18 12:36:51.796
741    2018-12-18 12:37:23.448
881    2018-12-18 12:40:57.603
910    2018-12-18 12:42:02.904
971    2018-12-18 12:43:37.361

我希望dftogether看起来像这样，但是使用时间戳而不是int

   Unique  DateTime
 737    False  20181812123719
 738    False  20181812123721
 739    False  20181812123722
 741    False  20181812123723
 742     True  20181812123723
 740     True  20181812123723
 785    False  20181812123830
 786    False  20181812123831
 787    False  20181812123832
 787     True  20181812123832
 788    False  20181812123833

所以我可以把那些放在Unique == True

 785    False 2018-12-18 12:38:30
 786    False 2018-12-18 12:38:31
 787    False 2018-12-18 12:38:32
 788    False 2018-12-18 12:38:33
 790    False 2018-12-18 12:38:34
 812    False 2018-12-18 12:39:10
 813    False 2018-12-18 12:39:11

还有一点：我在哪里可以表达我对这项新计划的看法？我可以问一个问题吗？在我看来，这真的很糟糕，它一直在向上滚动，输入/复制代码现在真的很混乱，所有的例子真的让人分心。写这个问题花了我30多分钟

Answer 1

我将您的df1和df2加入到df中，并创建了如下日期列表：

df = pd.concat([df1,df2]).sort_values('DateTime').reset_index(drop=True)

date_list = [datetime.strptime(i, '%Y-%m-%d %H:%M:%S.%f') for i in df.DateTime.tolist()]

然后我用1行代码得到所需的输出：

df[[x>1 for x in [0]+[(j-i).total_seconds() for i,j in zip(date_list, date_list[1:])]]]

要了解其工作原理，请首先检查以下输出：

[x>1 for x in [0]+[(j-i).total_seconds() for i,j in zip(date_list, date_list[1:])]]

希望这有帮助。干杯。你知道吗

Answer 2

我这样做了，你的初始列是a和b-这是你需要的吗？你知道吗

from datetime import timedelta
df = pd.concat([df1, df2, df3])
df = df.sort_values('b')
df['s'] = df['b'].shift()
df['d'] = df['b'] - df['s'] 
df['f'] = df['d'] < timedelta(0, 1)
print(df)

结果：

      a                       b                       s               d      f
0   691 2018-12-18 12:35:23.612                     NaT             NaT  False
1   692 2018-12-18 12:35:25.627 2018-12-18 12:35:23.612 00:00:02.015000  False
0   699 2018-12-18 12:35:42.452 2018-12-18 12:35:25.627 00:00:16.825000  False
1   701 2018-12-18 12:35:45.081 2018-12-18 12:35:42.452 00:00:02.629000  False
2   727 2018-12-18 12:36:47.466 2018-12-18 12:35:45.081 00:01:02.385000  False
3   730 2018-12-18 12:36:51.796 2018-12-18 12:36:47.466 00:00:04.330000  False
0   737 2018-12-18 12:37:19.717 2018-12-18 12:36:51.796 00:00:27.921000  False
1   738 2018-12-18 12:37:21.936 2018-12-18 12:37:19.717 00:00:02.219000  False
2   739 2018-12-18 12:37:22.841 2018-12-18 12:37:21.936 00:00:00.905000   True
3   740 2018-12-18 12:37:23.144 2018-12-18 12:37:22.841 00:00:00.303000   True
4   741 2018-12-18 12:37:23.448 2018-12-18 12:37:23.144 00:00:00.304000   True
2   788 2018-12-18 12:38:33.248 2018-12-18 12:37:23.448 00:01:09.800000  False
3   789 2018-12-18 12:38:33.553 2018-12-18 12:38:33.248 00:00:00.305000   True
4   790 2018-12-18 12:38:34.759 2018-12-18 12:38:33.553 00:00:01.206000  False
5   866 2018-12-18 12:40:29.487 2018-12-18 12:38:34.759 00:01:54.728000  False
6   867 2018-12-18 12:40:31.199 2018-12-18 12:40:29.487 00:00:01.712000  False
7   868 2018-12-18 12:40:32.206 2018-12-18 12:40:31.199 00:00:01.007000  False
4   877 2018-12-18 12:40:53.268 2018-12-18 12:40:32.206 00:00:21.062000  False
5   878 2018-12-18 12:40:56.597 2018-12-18 12:40:53.268 00:00:03.329000  False
6   879 2018-12-18 12:40:56.899 2018-12-18 12:40:56.597 00:00:00.302000   True
7   880 2018-12-18 12:40:57.300 2018-12-18 12:40:56.899 00:00:00.401000   True
5   881 2018-12-18 12:40:57.603 2018-12-18 12:40:57.300 00:00:00.303000   True
6   910 2018-12-18 12:42:02.904 2018-12-18 12:40:57.603 00:01:05.301000  False
8   968 2018-12-18 12:43:31.411 2018-12-18 12:42:02.904 00:01:28.507000  False
9   969 2018-12-18 12:43:36.150 2018-12-18 12:43:31.411 00:00:04.739000  False
10  970 2018-12-18 12:43:36.452 2018-12-18 12:43:36.150 00:00:00.302000   True
7   971 2018-12-18 12:43:37.361 2018-12-18 12:43:36.452 00:00:00.909000   True