如何从大型文本文件中提取模式

Question

我有一个巨大的文本文件，它由A-Z的随机字母组成，我想提取一些字符。棘手的部分是给定以下输入：

AFVAJFLDVAJPQDVAJDSNJKVAJGHD

对于模式VAJ，我想提取每个匹配项，直到字符串结束。我想要以下输出：

[ "VAJFLDVAJPQDVAJDSNJKVAJGHD", "VAJPQDVAJDSNJKVAJGHD", "VAJDSNJKVAJGHD", "VAJGHD" ]

Answer 1

您可以使用^{}来查找索引，即您的模式出现的位置。然后可以相应地对字符串进行切片。实现可以如下所示：

def find(inp, what):
  matches = []
  while what in inp:
    idx = inp.find(what)
    matches.append(inp[idx:])
    # remove the previous pattern from the string
    inp = inp[idx+len(what):]

  return matches

您可以将它与find("AFVAJFLDVAJPQDVAJDSNJKVAJGHD", "VAJ")一起使用。你知道吗

Answer 2

这将调用具有子组匹配的正则表达式。 （https://docs.python.org/3.5/library/re.html#match-objects）

我的测试文件data.txt：

QWEEEFVAJFLDVAJPQDVAJDSNJKVAJGHD
AFVAJFLDVAJPQDVAJDSNJKHFGHERQWFS
ONLY_TWO_VAJsOOVAJ123VAQQWERTY
START_VAJs_with_more_VAJ123VAJ_space_between
AAPVAJRCGVAJJKYVAJJJJJJJJVAJOOOO
AAPVAJRCGVAJJKYVAJJJJJJJJQQQOOOOO

Python代码：

import re

pattern = "VAJ"

re_str = pattern + "..." + "(" + pattern + "..." +"(" +  pattern + "(.*)))"
regex = re.compile(re_str)

regex_extra = re.compile(pattern + ".*")

for line in open("data.txt"):
    line = line.strip()
    match = regex.search(line)
    if match:
        result = list()
        result.append(match.group(0))   # entire regex match
        result.append(match.group(1))   # outer regex parenthesis'ed group
        result.append(match.group(2))   # middle regex parenthesis'ed group

        # Most inner regex parenthesis'ed group contains rest of the line.
        # Use this to find extra pattern.
        #
        the_rest = match.group(3)
        match_extra = regex_extra.search(the_rest)
        if match_extra:   # If one more <pattern> in the rest of the line
            result.append(match_extra.group(0))   # add it to the result list

        # Output         
        print(result)

输出：

['VAJFLDVAJPQDVAJDSNJKVAJGHD', 'VAJPQDVAJDSNJKVAJGHD', 'VAJDSNJKVAJGHD', 'VAJGHD']
['VAJFLDVAJPQDVAJDSNJKHFGHERQWFS', 'VAJPQDVAJDSNJKHFGHERQWFS', 'VAJDSNJKHFGHERQWFS']
['VAJRCGVAJJKYVAJJJJJJJJVAJOOOO', 'VAJJKYVAJJJJJJJJVAJOOOO', 'VAJJJJJJJJVAJOOOO', 'VAJOOOO']
['VAJRCGVAJJKYVAJJJJJJJJQQQOOOOO', 'VAJJKYVAJJJJJJJJQQQOOOOO', 'VAJJJJJJJJQQQOOOOO']

文件的庞大性不是这段代码的问题，只要最长的一行在内存中放几次就可以了。你知道吗