如何从字符串生成唯一的数据

Question

我有这样的数据。字符串用逗号分隔。你知道吗

"India1,India2,myIndia     "
"Where,Here,Here   "
"Here,Where,India,uyete"
"AFD,TTT"

我要做的是把它们放在一列（一列在另一列下面），这样就变成这样了

India1
India2
myIndia
Where
Here
Here
Here
Where
India
uyete
AFD
TTT

然后我保留了导致这一切的独特的东西

India1
India2
myIndia
Where
Here
India
uyete
AFD
TTT

所以我有一个.txt格式的第一个数据，我试着用numpy来实现这个

这是我的密码

#!/usr/bin/python
import numpy as np

# give a name to my data 
file_name = 'path to my data/test.txt'
# set my output 
with open ( 'output.txt' , 'w' ) as out:
    # read all the lines
    for n , line in enumerate ( open ( file_name ).readlines ( ) ):
        # split each stirg from another one by a comma
        item1 = file_name.split ( ',' )
    myList = ','.join ( map ( str , item1 ) )
    item2 = np.unique ( myList , return_inverse=True )
    # save the data into out
    out.write ( item2 )

我得到了TypeError: expected a character buffer object

我搜索了一下，发现了几个类似的帖子 TypeError: expected a character buffer object - while trying to save integer to textfile

如果我加上out.seek ( 0 )，我仍然得到同样的错误

但是由于TypeError: expected a character buffer object的原因，将它改为out.write ( str(item2 ))我没有得到任何错误，但是输出显示了这一点

(array(['/path to the file/test.txt'], dtype='|S29'), array([0]))

下面是我尝试使用的解决方案

import csv

data = []
def remove_quotes(file):
    for line in file:
        yield line.strip ( '"\n' )
with open ( 'test.txt' ) as f:
    reader = csv.reader ( remove_quotes ( f ) )
    for row in reader:
        data.extend ( row )

没有错误，但也不会生成data

Answer 1

为什么要用numpy？？？我不确定你是否想使用同一个文件作为输入和输出

#!/usr/bin/env python


# give a name to my data 
inputData = """India1,India2,myIndia
Where,Here,Here   
Here,Where,India,uyete
AFD,TTT"""

# if you want to read the data from a file
#inputData = open(fileName, 'r').readlines()

outputData = ""
tempData = list()
for line in inputData.split("\n"):
    lineStripped = line.strip()
    lineSplit = lineStripped.split(',')
    lineElementsStripped = [element.strip() for element in lineSplit]
    tempData.extend( lineElementsStripped )
tempData = set(tempData)
outputData = "\n".join(tempData)
print("\nInputdata: \n%s" % inputData)
print("\nOutputdata: \n%s" % outputData)

Answer 2

stack.txt包含以下内容：

"India1,India2,myIndia"
"Where,Here,Here"
"Here,Where,India,uyete"
"AFD,TTT"

给你：

from collections import OrderedDict

with open("stack.txt", "r") as f:
    # read your data in from the gist site and strip off any new-line characters
    data = [eval(line.strip()) for line in f.readlines()]
    # get individual words into a list
    individual_elements = [word for row in data for word in row.split(",")]
    # remove duplicates and preserve order
    uniques = OrderedDict.fromkeys(individual_elements)   
    # convert from OrderedDict object to plain list
    final = [word for word in uniques]

print(final)

由此得出：

['India1', 'India2', 'myIndia', 'Where', 'Here', 'India', 'uyete', 'AFD', 'TTT']

编辑：要获得所需的输出，只需按所需格式打印列表：

print("\n".join(final))

从输出的角度来看，这相当于：

for x in final:
    print(x)

由此得出：

India1
India2
myIndia
Where
Here
India
uyete
AFD
TTT

Answer 3

听起来你可能有一个csv文件。你不需要numpy的，包括电池是所有你需要的。你知道吗

 import csv

 data = []
 with open('test.txt') as f:
     reader = csv.reader(f)
     for row in reader:
         data.extend(row)

您可以.extend列表，而不是.append列表。基本上就像说

for thing in row:
    data.append(thing)

不过，这仍然会留下复制品。如果您不关心顺序，可以将其设为set，并调用.update()，而不是extend：

 data = set()
 with open('test.txt') as f:
     reader = csv.reader(f)
     for row in reader:
         data.extend(row)

现在一切都是独一无二的。但如果你关心秩序，你就得把事情过滤掉：

unique_data = []
for thing in data:
    if thing not in unique_data:
        unique_data.append(thing)

如果test.txt文件包含以下文本：

"India1,India2,myIndia     "
"Where,Here,Here   "
"Here,Where,India,uyete"
"AFD,TTT"

而不是

India1,India2,myIndia     
Where,Here,Here   
Here,Where,India,uyete
AFD,TTT

那你就没有csv了。您可以修复生成csv的内容，也可以手动删除引号或即时修复。你知道吗

def remove_quotes(file):
    for line in file:
        yield line.strip('"\n')

reader = csv.reader(remove_quotes(f))