如何在两个CSV文件之间查找相似的行值并考虑计数重复值

Question

我有2个csv文件，格式如下：

csv A

Tweet1,pos
Tweet2,neg
Tweet2,neg

csv B

Tweet2,neg
Tweet2,neg
Tweet2,pos

我想找出罗氏之间的相似度

我试过这个，但这个有区别

def compare( fileA, fileB ):
    a_file = open(fileA, 'r')
    a_data = a_file.read()
    a_file.close()

    b_file = open(fileB, 'r')
    b_data = b_file.read()
    b_file.close()

    # compare the contents
    a_set = set(a_data.split(','))
    b_set = set(b_data.split(','))

    return list(a_set.intersection(b_set))

print compare('f.csv', 'full-corpus.csv')

输出应为1

Answer 1

要做到这一点，只需从集合中导入类计数器，然后将每个文件作为列表打开。你知道吗

import csv
from collections import Counter
a_list = []
with open('1.csv', 'Ur') as a_file:
    for line in csv.reader(a_file):
        a_list.append(line[0]+' '+line[1])
print a_list

b_list = []
with open('2.csv', 'Ur') as b_file:
    for line in csv.reader(b_file):
        b_list.append(line[0]+' '+line[1])
print b_list
counterA = Counter(a_list)
counterB = Counter(b_list)
counterSum = counterB & counterA
print counterA
print counterB
print counterB & counterA
print sum(counterSum.values())

Answer 2

你可以试试return len(a_set & b_set)。&是查找所有集合中存在的元素的运算符，len将是所有集合中存在的元素数