如何微调regex语法来解析棘手的脏文本？

# read in review volume .txt file import pandas as pd import numpy as np import re file = '/Users/sinykin/Dropbox/US_LIT_PRODUCTION_DATA/REVIEWS_DATA/BOOK_REVIEWS_INDEX_TEXTS/1965_1984_Vol_5_M-P.txt' with open(file) as f: content = f.readlines() content = [x.strip() for x in content] content = " ".join(content) # Get all authors pattern = r"[A-Z\-]{2,}[\,]+\s[A-Za-z\s\,\.]+\s[\-\*\•\.\■ ]{1}" authors = re.findall(pattern, content) # Now replace all found authors with XXX_XXX if re.search(pattern, content): r = re.compile(pattern) content2 = r.sub(r'XXX_XXX', content) # Now get all the content for each author content3 = content2.split('XXX_XXX') bib = content3[1:] # Now separate reviews from titles pattern2 = r"\s+(?:([A-Z][a-z][a-z]((?!\s+-|\s+Choice\s*-).)*?\w)(?:\s+- \s*|\s+(?=Choice\s*-)|\s*$))" bib2 = "".join(bib) titles = re.findall(pattern2, bib2) print (titles[:1000])

2条回答

网友

1楼 · 编辑于 2024-09-29 19:18:59

Question: ... refine my regex to capture just the titles?

在这些模式上拆分content2以获得书籍列表：

books = []
for c in re.split('(XXX_XXX|MA, | MA, | MA )', content2):
    if c and not c in ['MA, ', ' MA, ', ' MA ', 'XXX_XXX']:
        books.append(c.strip())

循环books以提取title：

reObj = re.compile(r'(.+?)(( [A-Z\&]{1,4})?| Choice| Cur ?R) ?- ?[’v][l]?(\d{1,2}|O )')
for book in books:
    match = reObj.match(book)
    if match:
        title = match.groups()[0]
        print('{}'.format(title))
    else:
        print('FAIL:{}'.format(book))

Output:

Chi-Hua - Huan Chiu Hsin Ying
Ching-Hsien - Pei Niang Niang Ti Ku Shih
Hsin-Teh - Chinese Women In The Great Leap Forward
The Overall Survey Of The Ocean’s Shores 1433
Ying-Yai Sheng-Lan
Commercial Development And Urban Change In Sung China 960-1279
The Environment
Urban Development In Modern China
Cook Chinese
Don’t Lick The Chopsticks
... (omitted for brevity)

测试Python:3.4.2-回复：2.2.1

网友
2楼 · 编辑于 2024-09-29 19:18:59

您可以通过添加一些特定的表达式，在最后一个正向展望的基础上继续构建，以筛选出不需要的尾随字符。例如，扩展
(?:\s+-\s*|\s+(?=Choice\s*-)|\s*$))
进入
(?:\s+-\s*|\s+(?=Choice\s*-)|\s*$|\s+[A-Z]{2,3}|\s+Cur R))
将消除Cur R和代码结尾（AB，JAS，等等）：
('The Environment', 'n')
('Urban Development In Modern China', 'n')
('Cook Chinese', 's')
('Nancy Chih \xe2\x80\xa2 Don\xe2\x80\x99t Lick The Chopsticks', 'k')
('Mrs. Ma\xe2\x80\x99s Japanese Cooking', 'n')

相关问题更多 >

编程相关推荐

热门问题

热门文章