我正在用Pandas队和美女队刮足球花名册，但它刮不到两列

Question

这是我的密码：

from bs4 import BeautifulSoup as bs
import requests
import pandas as pd

r = requests.get('https://rolltide.com/roster.aspx?roster=226&path=football', headers = {'User-Agent':'Mozilla/5.0'})
soup = bs(r.content, 'lxml')
results = {}

for num, p in enumerate(soup.select('.sidearm-roster-player')):
    results[num] = {'position': p.select_one('.sidearm-roster-player-position >span:first-child').text.strip()
           ,'Height': p.select_one('.sidearm-roster-player-height').text
           ,'Weight': p.select_one('.sidearm-roster-player-weight').text
           ,'Number': p.select_one('.sidearm-roster-player-jersey-number').text
           ,'Name': p.select_one('.sidearm-roster-player-name a').text
           ,'Year': p.select_one('.sidearm-roster-player-academic-year').text
           ,'Hometown': p.select_one('.sidearm-roster-player-hometown').text
           ,'Highschool': p.select_one('.sidearm-roster-player-highschool').text
          }
df = pd.DataFrame(results.values(), columns = ['Number','Name','Position','Height','Year','Hometown','Highschool'])
df.to_excel(r'desktop\Alabama.xlsx', index=False)

除了“数字”和“位置”我不知道为什么。知道怎么回事吗

Answer 1

您可以使用pandas内置的read\u html函数：

df = pd.read_html('https://rolltide.com/sports/football/roster/2019')[2] #one you're after is the 3rd table being scraped

结果：

         #            Full Name   Pos.   Ht.  Wt. Academic Year                       Hometown / High School
0        1            Ben Davis     LB   6-4  243         R-Jr.                          Gordo, Ala. / Gordo
1        2      Keilan Robinson     RB   5-9  190           Fr.                Washington, D.C. / St. John's
2        2   Patrick Surtain II     DB   6-2  203           So.         Plantation, Fla. / American Heritage
3        3        Daniel Wright     DB   6-1  190         R-So.        Fort Lauderdale, Fla. / Boyd Anderson
4        4    Christopher Allen     LB   6-4  250         R-So.       Baton Rouge, La. / Southern Lab School
5        4          Jerry Jeudy     WR   6-1  192           Jr.      Deerfield Beach, Fla. / Deerfield Beach
6        5       Shyheim Carter     DB   6-0  191           Sr.                     Kentwood, La. / Kentwood
7        5    Taulia Tagovailoa     QB  5-11  208           Fr.                Ewa Beach, Hawai'i / Thompson
8        6        DeVonta Smith     WR   6-1  175           Jr.                           Amite, La. / Amite
9        7       Braxton Barker     QB   6-1  202         R-Fr.                Birmingham, Ala. / Spain Park
10       7         Trevon Diggs     DB   6-2  207           Sr.            Gaithersburg, Md. / Avalon School
11       8     Christian Harris     LB   6-2  244           Fr.            Baton Rouge, La. / University Lab
12       8     John Metchie III     WR   6-0  195           Fr.    Brampton, Canada / St. James School (Md.)
13       9        Jordan Battle     DB   6-1  201           Fr.   Fort Lauderdale, Fla. / St. Thomas Aquinas
14       9      Xavier Williams     WR   6-1  195         R-Fr.     Hollywood, Fla. / Chaminade-Madonna Prep
15      10            Mac Jones     QB   6-2  205         R-So.       Jacksonville, Fla. / The Bolles School
16      11        Scooby Carter     DB   6-0  186           Fr.          Mansfield, Texas / Mansfield Legacy
17      11      Henry Ruggs III     WR   6-0  190           Jr.                       Montgomery, Ala. / Lee
18      12        Skyler DeLong      P   6-4  188           So.                Fort Mill, S.C. / Nation Ford
19      12   Chadarius Townsend  RB/WR   6-0  194         R-So.                        Tanner, Ala. / Tanner
20      13       Tua Tagovailoa     QB   6-1  218           Jr.               Ewa Beach, Hawai'i / St. Louis
21      14       Tyrell Shavers     WR   6-6  205         R-So.               Lewisville, Texas / Lewisville
22      14      Brandon Turnage     DB   6-1  185           Fr.                    Oxford, Miss. / Lafayette
23      15      Xavier McKinney     DB   6-1  200           Jr.                       Roswell, Ga. / Roswell
24      15           Paul Tyson     QB   6-5  220           Fr.         Trussville, Ala. / Hewitt-Trussville
25      16        Jayden George     QB   6-3  192           Fr.          Indianapolis, Ind. / Warren Central
26      16        Will Reichard     PK   6-1  180           Fr.                        Hoover, Ala. / Hoover
27      17        Jaylen Waddle     WR  5-10  182           So.                   Houston, Texas / Episcopal
28      18         Slade Bolden     WR  5-11  191         R-Fr.               West Monroe, La. / West Monroe
29      19  Jahleel Billingsley     TE   6-4  228           Fr.             Chicago, Ill. / Phillips Academy
..     ...                  ...    ...   ...  ...           ...                                          ...
94      76        Scott Lashley     OL   6-7  307         R-Jr.               West Point, Miss. / West Point
95      77          Matt Womack     OL   6-7  325         R-Sr.           Hernando, Miss. / Magnolia Heights
96      78          Amari Kight     OL   6-7  302           Fr.                   Alabaster, Ala. / Thompson
97      80       Michael Parker     TE   6-6  216         R-Fr.     Huntsville, Ala. / Westminster Christian
98      81         Cameron Latu     TE   6-5  247         R-Fr.               Salt Lake City, Utah / Olympus
99      82         Richard Hunt     TE   6-7  235           Fr.        Memphis, Tenn. / Briarcrest Christian
100     83          John Parker     WR   6-0  190           Sr.     Huntsville, Ala. / Westminster Christian
101     84        Joshua Lanier     WR  5-11  160           Sr.        Tuscaloosa, Ala. / Tuscaloosa Academy
102  84/79          Chris Owens     OL   6-3  315         R-Jr.                     Arlington, Texas / Lamar
103     85       Drew Kobayashi     WR   6-2  200         R-Jr.                Honolulu, Hawai'i / St. Louis
104  85/60     Kendall Randolph  TE/OL   6-4  296         R-So.                    Madison, Ala. / Bob Jones
105     86         Connor Adams     DB   6-1  194           Sr.                   Sugar Land, Texas / Austin
106     86   Quindarius Watkins     TE   6-4  230           Jr.       Fort Stewart, Ga. / Bradwell Institute
107     87    Miller Forristall     TE   6-5  242         R-Jr.             Cartersville, Ga. / Cartersville
108     88       Major Tennison     TE   6-5  248         R-So.                       Flint, Texas / Bullard
109     89        Grant Krieger     WR   6-2  192           Fr.              Pittsburgh, Pa. / Pine-Richland
110     89          LaBryan Ray     DL   6-5  292           Jr.                Madison, Ala. / James Clemens
111     90     Stephon Wynn Jr.     DL   6-4  311         R-Fr.                 Anderson, S.C. / IMG Academy
112     91        Tevita Musika     DL   6-1  338           Sr.   Milpitas, Calif. / Milpitas/San Mateo J.C.
113     92       Justin Eboigbe     DL   6-5  294           Fr.               Forest Park, Ga. / Forest Park
114     93      Landon Bothwell     DL  5-11  220           So.                      Oneonta, Ala. / Oneonta
115     93         Tripp Slyman   PK/P   6-1  180         R-Fr.                  Huntsville, Ala. / Randolph
116     94              DJ Dale     DL   6-3  308           Fr.           Birmingham, Ala. / Clay-Chalkville
117     95          Jack Martin      P   6-0  206           Fr.                 Mobile, Ala. / McGill-Toolen
118     95      Ishmael Sopsher     DL   6-4  334           Fr.                           Amite, La. / Amite
119     96        Taylor Wilson     DL   6-0  232           Sr.         Huntington Beach, Calif. / Mater Dei
120     97       Joseph Bulovas     PK   6-0  203         R-So.                 Mandeville, La. / Mandeville
121     98         Mike Bernier      P   6-2  219         R-Sr.  Madison, Ala. / Bob Jones/ Eastern Illinois
122     99        Raekwon Davis     DL   6-7  312           Sr.                   Meridian, Miss. / Meridian
123     99            Ty Perine   PK/P   6-1  190           Fr.                Prattville, Ala. / Prattville