从具有特定模式的txt文件创建Pandas数据帧

Question

我需要基于基于以下结构的文本文件创建Pandas数据帧：

Alabama[edit]
Auburn (Auburn University)[1]
Florence (University of North Alabama)
Jacksonville (Jacksonville State University)[2]
Livingston (University of West Alabama)[2]
Montevallo (University of Montevallo)[2]
Troy (Troy University)[2]
Tuscaloosa (University of Alabama, Stillman College, Shelton State)[3][4]
Tuskegee (Tuskegee University)[5]
Alaska[edit]
Fairbanks (University of Alaska Fairbanks)[2]
Arizona[edit]
Flagstaff (Northern Arizona University)[6]
Tempe (Arizona State University)
Tucson (University of Arizona)
Arkansas[edit]

带“[编辑]”的行是状态，[编号]的行是区域。我需要拆分以下内容，然后为每个区域名重复状态名。

Index          State          Region Name
0              Alabama        Aurburn...
1              Alabama        Florence...
2              Alabama        Jacksonville...
...
9              Alaska         Fairbanks...
10             Alaska         Arizona...
11             Alaska         Flagstaff...

熊猫数据框

我不知道如何将基于“[编辑]”和“[数字]”或“（字符）”的文本文件拆分到相应的列中，并为每个区域名称重复状态名称。请大家给我一个起点来完成以下工作。

Answer 1

您可以首先使用参数name来DataFrame创建DataFrame列Region Name，分隔符是不在值中的值（如;）：

df = pd.read_csv('filename.txt', sep=";", names=['Region Name'])

然后^{}新列State带有^{}行，其中文本[edit]和^{}所有值从(到列Region Name。

df.insert(0, 'State', df['Region Name'].str.extract('(.*)\[edit\]', expand=False).ffill())
df['Region Name'] = df['Region Name'].str.replace(r' \(.+$', '')

最后删除由^{}创建文本[edit]的行，掩码由^{}创建：

df = df[~df['Region Name'].str.contains('\[edit\]')].reset_index(drop=True)
print (df)
      State   Region Name
0   Alabama        Auburn
1   Alabama      Florence
2   Alabama  Jacksonville
3   Alabama    Livingston
4   Alabama    Montevallo
5   Alabama          Troy
6   Alabama    Tuscaloosa
7   Alabama      Tuskegee
8    Alaska     Fairbanks
9   Arizona     Flagstaff
10  Arizona         Tempe
11  Arizona        Tucson

如果需要，所有价值观解决方案更容易：

df = pd.read_csv('filename.txt', sep=";", names=['Region Name'])
df.insert(0, 'State', df['Region Name'].str.extract('(.*)\[edit\]', expand=False).ffill())
df = df[~df['Region Name'].str.contains('\[edit\]')].reset_index(drop=True)
print (df)
      State                                        Region Name
0   Alabama                      Auburn (Auburn University)[1]
1   Alabama             Florence (University of North Alabama)
2   Alabama    Jacksonville (Jacksonville State University)[2]
3   Alabama         Livingston (University of West Alabama)[2]
4   Alabama           Montevallo (University of Montevallo)[2]
5   Alabama                          Troy (Troy University)[2]
6   Alabama  Tuscaloosa (University of Alabama, Stillman Co...
7   Alabama                  Tuskegee (Tuskegee University)[5]
8    Alaska      Fairbanks (University of Alaska Fairbanks)[2]
9   Arizona         Flagstaff (Northern Arizona University)[6]
10  Arizona                   Tempe (Arizona State University)
11  Arizona                     Tucson (University of Arizona)

Answer 2

可以先将文件解析为元组：

import pandas as pd
from collections import namedtuple

Item = namedtuple('Item', 'state area')
items = []

with open('unis.txt') as f: 
    for line in f:
        l = line.rstrip('\n') 
        if l.endswith('[edit]'):
            state = l.rstrip('[edit]')
        else:            
            i = l.index(' (')
            area = l[:i]
            items.append(Item(state, area))

df = pd.DataFrame.from_records(items, columns=['State', 'Area'])

print df

输出：

      State          Area
0   Alabama        Auburn
1   Alabama      Florence
2   Alabama  Jacksonville
3   Alabama    Livingston
4   Alabama    Montevallo
5   Alabama          Troy
6   Alabama    Tuscaloosa
7   Alabama      Tuskegee
8    Alaska     Fairbanks
9   Arizona     Flagstaff
10  Arizona         Tempe
11  Arizona        Tucson

Answer 3

假设您有以下DF：

In [73]: df
Out[73]:
                                                 text
0                                       Alabama[edit]
1                       Auburn (Auburn University)[1]
2              Florence (University of North Alabama)
3     Jacksonville (Jacksonville State University)[2]
4          Livingston (University of West Alabama)[2]
5            Montevallo (University of Montevallo)[2]
6                           Troy (Troy University)[2]
7   Tuscaloosa (University of Alabama, Stillman Co...
8                   Tuskegee (Tuskegee University)[5]
9                                        Alaska[edit]
10      Fairbanks (University of Alaska Fairbanks)[2]
11                                      Arizona[edit]
12         Flagstaff (Northern Arizona University)[6]
13                   Tempe (Arizona State University)
14                     Tucson (University of Arizona)
15                                     Arkansas[edit]

您可以使用Series.str.extract()方法：

In [117]: df['State'] = df.loc[df.text.str.contains('[edit]', regex=False), 'text'].str.extract(r'(.*?)\[edit\]', expand=False)

In [118]: df['Region Name'] = df.loc[df.State.isnull(), 'text'].str.extract(r'(.*?)\s*[\(\[]+.*[\n]*', expand=False)

In [120]: df.State = df.State.ffill()

In [121]: df
Out[121]:
                                                 text     State   Region Name
0                                       Alabama[edit]   Alabama           NaN
1                       Auburn (Auburn University)[1]   Alabama        Auburn
2              Florence (University of North Alabama)   Alabama      Florence
3     Jacksonville (Jacksonville State University)[2]   Alabama  Jacksonville
4          Livingston (University of West Alabama)[2]   Alabama    Livingston
5            Montevallo (University of Montevallo)[2]   Alabama    Montevallo
6                           Troy (Troy University)[2]   Alabama          Troy
7   Tuscaloosa (University of Alabama, Stillman Co...   Alabama    Tuscaloosa
8                   Tuskegee (Tuskegee University)[5]   Alabama      Tuskegee
9                                        Alaska[edit]    Alaska           NaN
10      Fairbanks (University of Alaska Fairbanks)[2]    Alaska     Fairbanks
11                                      Arizona[edit]   Arizona           NaN
12         Flagstaff (Northern Arizona University)[6]   Arizona     Flagstaff
13                   Tempe (Arizona State University)   Arizona         Tempe
14                     Tucson (University of Arizona)   Arizona        Tucson
15                                     Arkansas[edit]  Arkansas           NaN

In [122]: df = df.dropna()

In [123]: df
Out[123]:
                                                 text    State   Region Name
1                       Auburn (Auburn University)[1]  Alabama        Auburn
2              Florence (University of North Alabama)  Alabama      Florence
3     Jacksonville (Jacksonville State University)[2]  Alabama  Jacksonville
4          Livingston (University of West Alabama)[2]  Alabama    Livingston
5            Montevallo (University of Montevallo)[2]  Alabama    Montevallo
6                           Troy (Troy University)[2]  Alabama          Troy
7   Tuscaloosa (University of Alabama, Stillman Co...  Alabama    Tuscaloosa
8                   Tuskegee (Tuskegee University)[5]  Alabama      Tuskegee
10      Fairbanks (University of Alaska Fairbanks)[2]   Alaska     Fairbanks
12         Flagstaff (Northern Arizona University)[6]  Arizona     Flagstaff
13                   Tempe (Arizona State University)  Arizona         Tempe
14                     Tucson (University of Arizona)  Arizona        Tucson