有没有可能用样式标记将HTML表读入pandas？

Question

我试图使用pandasread_html函数来阅读位于here的“众议院正式名单”。在

使用

df_list = pd.read_html('http://clerk.house.gov/member_info/olmbr.aspx',header=0,encoding = "UTF-8")
house = df_list[0]

我确实得到了一个很好的数据框，上面有代表的姓名、州和地区。标题正确，编码也正确。到现在为止，一直都还不错。在

然而，问题在于党。聚会没有专栏。相反，该方用字体（罗马或斜体）表示。查看HTML源代码，下面是一个民主党人的条目：

^{pr2}$

这里有一个共和党人的条目：

<tr><td>Anderholt, Robert B.</td><td>AL</td><td>4th</td></tr>

共和党人的名字周围缺少<em></em>标签。在

如何检索这些信息？可以用pandas完成吗？还是需要一些更复杂的HTML解析器？如果是，那是哪一个？在

Answer 1

我认为您需要创建解析器：

import requests
from bs4 import BeautifulSoup

url = "http://clerk.house.gov/member_info/olmbr.aspx"
res = requests.get(url)
soup = BeautifulSoup(res.text,'html5lib')
table = soup.find_all('table')[0] 
#print (table)

data = []
#remove first header 
rows = table.find_all('tr')[1:]
for row in rows:
    cols = row.find_all('td')
    #get all children tags of first td
    childrens = cols[0].findChildren()
    #extracet all tags joined by ,
    a = ', '.join([x.name for x in childrens]) if len(childrens) > 0 else ''

    cols = [ele.text.strip() for ele in cols]
    #add tag value for each row
    cols.append(a)
    data.append(cols)

---

^{pr2}$

---

也可以为所有可能的标记创建带有1和0的列：

import requests
from bs4 import BeautifulSoup

url = "http://clerk.house.gov/member_info/olmbr.aspx"
res = requests.get(url)
soup = BeautifulSoup(res.text,'html5lib')
table = soup.find_all('table')[0] 
#print (table)

data = []
rows = table.find_all('tr')[1:]
for row in rows:
    cols = row.find_all('td')
    childrens = cols[0].findChildren()
    a = '|'.join([x.name for x in childrens]) if len(childrens) > 0 else ''
    cols = [ele.text.strip() for ele in cols]
    cols.append(a)
    data.append(cols)

cols = ['Representative', 'State', 'District', 'Tag']
df = pd.DataFrame(data, columns=cols)
df = df.join(df.pop('Tag').str.get_dummies())
print (df.head())
        Representative State District  em  strong
0   Abraham, Ralph Lee    LA      5th   0       0
1       Adams, Alma S.    NC     12th   1       0
2  Aderholt, Robert B.    AL      4th   0       0
3        Aguilar, Pete    CA     31st   1       0
4       Allen, Rick W.    GA     12th   0       0