这是我发现的一个脚本,用于使用grad cafe网站搜集不同项目的研究生招生结果。然而,当我运行它来查找“政治学”的结果时,它指出我有以下错误
Traceback (most recent call last):
File "C:/Users/lakna/OneDrive/Desktop/Spring 2018/Statistical Programming/Final Project/gradcafe_scraping.py", line 57, in <module>
data = get_data()
File "C:/Users/lakna/OneDrive/Desktop/Spring 2018/Statistical Programming/Final Project/gradcafe_scraping.py", line 52, in get_data
pages = get_pages()
File "C:/Users/lakna/OneDrive/Desktop/Spring 2018/Statistical Programming/Final Project/gradcafe_scraping.py", line 45, in get_pages
n = find_n_pages()
File "C:/Users/lakna/OneDrive/Desktop/Spring 2018/Statistical Programming/Final Project/gradcafe_scraping.py", line 41, in find_n_pages
reg = re.search('over\s([\d]*)\spages',html)
File "C:\Users\lakna\AppData\Local\Programs\Python\Python36-32\lib\re.py", line 182, in search
return _compile(pattern, flags).search(string)
TypeError: expected string or bytes-like object
我该如何着手解决这个问题?下面是我使用的代码
def get_page(i=0, keyword="Political Science"):
time.sleep(10)
if i==0:
#To change subjects, you want to change the keyword to say biostatistics,
#test it by searching on the site to make sure you get what you want.
url = "http://thegradcafe.com/survey/index.php?q="+keyword+"*&t=a&o=&pp=250"
else:
url="http://thegradcafe.com/survey/index.php?q="+keyword+"*&t=a&pp=250&o=&p="+str(i)
response = urlopen(url)
html = response.read().decode('utf-8')
return
def find_n_pages():
html = get_page()
reg = re.search('over\s([\d]*)\spages',html)
return int(reg.groups()[0])
def get_pages():
n = find_n_pages()
print ("Getting",n,"pages.")
pages = [get_page(i) for i in range(1,n+1)]
return pages
def get_data():
data=[]
pages = get_pages()
for page in pages:
data=get_data_from_page(page,data)
return data
你的
get_page
函数没有返回html,它没有返回任何内容。你知道吗应为:
因此出现错误:
相关问题 更多 >
编程相关推荐