擅长:python、mysql、java
<p>您可以使用<code>unique</code>关键字轻松地完成它</p>
<p>代码如下</p>
<pre><code>import pandas as pd
data = pd.read_csv('d:\\test.txt', sep="/n", header=None)
df = pd.DataFrame(data[0].unique())
with open('d:\\testnew.txt', 'a') as f:
f.write(df.to_string(header = False, index = False)))
</code></pre>
<p>结果:要读取的测试文件包含数据</p>
<p><a href="https://i.stack.imgur.com/o77Vd.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/o77Vd.png" alt="enter image description here"/></a></p>
<p>结果是它删除了重复的行</p>
<p><a href="https://i.stack.imgur.com/FGp0S.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/FGp0S.png" alt="enter image description here"/></a></p>