需要python中数据帧多列短语中的字符串匹配帮助吗

Question

需要帮助匹配以下数据中的短语，我需要匹配TextA和TextB中的短语

下面的代码没有帮到我如何解决这个问题我有100个代码要匹配

#对混乱的短语进行排序

def sorts(string_value):
    sorted_string = sorted(string_value.split())
    sorted_string = ' '.join(sorted_string)
    return sorted_string

#删除字符串中的标点符号

punc = '''!()-[]{};:'"\,<>./?@#$%^&*_~'''

def punt(test_str):
    for ele in test_str:
        if ele in punc:
            test_str = test_str.replace(ele, "")
    return(test_str)

#匹配字符串

def lets_match(x):

    for text1 in TextA:
        for text2 in TextB:
            try:
                if sorts(punt(x[text1.casefold()])) == sorts(punt(x[text2.casefold()])):
                    return True
            except:
                continue
    return False
df['result'] = df.apply(lets_match,axis =1)

即使在实现了字符串排序、删除标点符号和区分大小写之后，我仍然认为这些字符串不匹配。我在这里错过了一些东西，可以帮助我实现它吗

Answer 1

我认为如果没有字符串距离的概念，你就无法做到这一点，你能做的就是使用，例如record linkage

我将不详细介绍，但我将向您展示此案例的用法示例

import pandas as pd 
import recordlinkage as rl 
from recordlinkage.preprocessing import clean

# creating first dataframe
df_text_a = pd.DataFrame({
    "Text A":[
        "AKIL KUMAR SINGH",
        "OUSMANI DJIBO",
        "PETER HRYB",
        "CNOC LIMITED",
        "POLY NOVA INDUSTRIES LTD",
        "SAM GAWED JR",
        "ADAN GENERAL LLC",
        "CHINA MOBLE LIMITED",
        "CASTAR CO., LTD.",
        "MURAN",
        "OLD SAROOP FOR CAR SEAT COVERS",
        "CNP HEALTHCARE, LLC",
        "GLORY PACK LTD",
        "AUNCO VENTURES",
        "INTERNATIONAL COMPANY",
        "SAMEERA HEAT AND ENERGY FUND"]
                }
            )

# creating second dataframe
df_text_b = pd.DataFrame({
        "Text B":[
            "Singh, Akil Kumar",
            "DJIBO, Ousmani Illiassou",
            "HRYB, Peter",
            "CNOOC LIMITED",
            "POLYNOVA INDUSTRIES LTD. ",
            "GAWED, SAM",
            "ADAN GENERAL TRADING FZE",
            "CHINA MOBILE LIMITED",
            "CASTAR GROUP CO., LTD.",
            "MURMAN    ",
            "Old Saroop for Car Seat Covers",
            "CNP HEATHCARE, LLC",
            "GLORY PACK LTD.",
            "AUNCO VENTURE",
            "INTL COMPANY",
            "SAMEERA HEAT AND ENERGY PROPERTY FUND"
            ]
        }
)

# preprocessing in very important on results, you have to find which fit well on yuor problem.
cleaned_a = pd.DataFrame(clean(df_text_a["Text A"], lowercase=True))
cleaned_b = pd.DataFrame(clean(df_text_b["Text B"], lowercase=True))

# creating an indexing which will be used for comprison, you have various type of indexing, watch documentation.

indexer = rl.Index()
indexer.full()
# generating all passible pairs
pairs = indexer.index(cleaned_a, cleaned_b)

# starting evaluation phase
compare = rl.Compare(n_jobs=-1)  
compare.string("Text A", "Text B", method='jarowinkler', label = 'text')
matches = compare.compute(pairs, cleaned_a, cleaned_b)

matches现在是一个多索引数据帧，接下来要做的是按第一个索引查找第二个索引上的所有最大值。因此，您将获得所需的结果

结果可以在距离、索引和/或预处理方面得到改进

Answer 2

实际上，您可以使用difflib来匹配两个文本，下面是您可以尝试的：

from difflib import SequenceMatcher

def similar(a, b):
    a=str(a).lower()
    b=str(b).lower()
    return SequenceMatcher(None, a, b).ratio()

def lets_match(d):
    print(d[0],"  - ",d[1])
    result=similar(d[0],d[1])
    print(result)
    if result>0.6:
        return True
    else:
        return False
    
df["result"]=df.apply(lets_match,axis =1)

您可以使用if result>0.6值进行播放

有关difflib的更多信息，请访问here。还有其他类似textdistance的序列匹配器，但我发现它很容易，所以我尝试了这个

Answer 3

使用模糊匹配库有什么问题吗？鉴于上述数据相对相似，该实现非常简单，运行良好。我在没有预处理的情况下执行了以下操作

    import pandas as pd
    """ Install the libs below via terminal:

            $pip install fuzzywuzzy
            $pip install python-Levenshtein
    """

    from fuzzywuzzy import fuzz
    from fuzzywuzzy import process


    #creating the data frames
        text_a = ['AKIL KUMAR SINGH','OUSMANI DJIBO','PETER HRYB','CNOC LIMITED','POLY NOVA INDUSTRIES LTD','SAM GAWED JR','ADAN GENERAL LLC','CHINA MOBLE LIMITED','CASTAR CO., LTD.','MURAN','OLD SAROOP FOR CAR SEAT COVERS','CNP HEALTHCARE, LLC','GLORY PACK LTD','AUNCO VENTURES','INTERNATIONAL COMPANY','SAMEERA HEAT AND ENERGY FUND']
        text_b = ['Singh, Akil Kumar','DJIBO, Ousmani Illiassou','HRYB, Peter','CNOOC LIMITED','POLYNOVA INDUSTRIES LTD.','GAWED, SAM','ADAN GENERAL TRADING FZE','CHINA MOBILE LIMITED','CASTAR GROUP CO., LTD.','MURMAN','Old Saroop for Car Seat Covers','CNP HEATHCARE, LLC','GLORY PACK LTD.','AUNCO VENTURE','INTL COMPANY','SAMEERA HEAT AND ENERGY PROPERTY FUND']
        df_text_a = pd.DataFrame(text_a, columns=['text_a'])
        df_text_b = pd.DataFrame(text_b, columns=['text_b'])
        
        def lets_match(txt: str, chklist: list) -> str: 
            return process.extractOne(txt, chklist, scorer=fuzz.token_set_ratio)


    #match Text_A against Text_B
        result_txt_ab = df_text_a.apply(lambda x: lets_match(str(x), text_b), axis=1, result_type='expand')
        result_txt_ab.rename(columns={0:'Return Match', 1:'Match Value'}, inplace=True)
        df_text_a[result_txt_ab.columns]=result_txt_ab
        df_text_a

---

    text_a  Return Match    Match Value
    0   AKIL KUMAR SINGH    Singh, Akil Kumar   100
    1   OUSMANI DJIBO   DJIBO, Ousmani Illiassou    72
    2   PETER HRYB  HRYB, Peter 100
    3   CNOC LIMITED    CNOOC LIMITED   70
    4   POLY NOVA INDUSTRIES LTD    POLYNOVA INDUSTRIES LTD.    76
    5   SAM GAWED JR    GAWED, SAM  100
    6   ADAN GENERAL LLC    ADAN GENERAL TRADING FZE    67
    7   CHINA MOBLE LIMITED CHINA MOBILE LIMITED    79
    8   CASTAR CO., LTD.    CASTAR GROUP CO., LTD.  81
    9   MURAN   SAMEERA HEAT AND ENERGY PROPERTY FUND   41
    10  OLD SAROOP FOR CAR SEAT COVERS  Old Saroop for Car Seat Covers  100
    11  CNP HEALTHCARE, LLC CNP HEATHCARE, LLC  58
    12  GLORY PACK LTD  GLORY PACK LTD. 100
    13  AUNCO VENTURES  AUNCO VENTURE   56
    14  INTERNATIONAL COMPANY   INTL COMPANY    74
    15  SAMEERA HEAT AND ENERGY FUND    SAMEERA HEAT AND ENERGY PROPERTY FUND   86

---

    #match Text_B against Text_A
    result_txt_ba= df_text_b.apply(lambda x: lets_match(str(x), text_a), axis=1, result_type='expand')
    result_txt_ba.rename(columns={0:'Return Match', 1:'Match Value'}, inplace=True)
    df_text_b[result_txt_ba.columns]=result_txt_ba
    df_text_b

---

text_b  Return Match    Match Value
0   Singh, Akil Kumar   AKIL KUMAR SINGH    100
1   DJIBO, Ousmani Illiassou    OUSMANI DJIBO   100
2   HRYB, Peter PETER HRYB  100
3   CNOOC LIMITED   CNOC LIMITED    74
4   POLYNOVA INDUSTRIES LTD.    POLY NOVA INDUSTRIES LTD    74
5   GAWED, SAM  SAM GAWED JR    86
6   ADAN GENERAL TRADING FZE    ADAN GENERAL LLC    86
7   CHINA MOBILE LIMITED    CHINA MOBLE LIMITED 81
8   CASTAR GROUP CO., LTD.  CASTAR CO., LTD.    100
9   MURMAN  ADAN GENERAL LLC    33
10  Old Saroop for Car Seat Covers  OLD SAROOP FOR CAR SEAT COVERS  100
11  CNP HEATHCARE, LLC  CNP HEALTHCARE, LLC 56
12  GLORY PACK LTD. GLORY PACK LTD  100
13  AUNCO VENTURE   AUNCO VENTURES  53
14  INTL COMPANY    INTERNATIONAL COMPANY   50
15  SAMEERA HEAT AND ENERGY PROPERTY FUND   SAMEERA HEAT AND ENERGY FUND    100