我正在抓取一个网站来获取公司和产品的详细信息。 它有一个div标记,其中有li标记,我想得到div标记中的所有li标记。 我使用的是python3.5.1和beauthoulsoup
我的代码:
from bs4 import BeautifulSoup
import urllib.request
import re
r = urllib.request.urlopen('http://i.cantonfair.org.cn/en/ExpExhibitorList.aspx?k=glassware')
soup = BeautifulSoup(r, "html.parser")
links = soup.find_all("a", href=re.compile(r"expexhibitorlist\.aspx\?categoryno=[0-9]+"))
linksfromcategories = ([link["href"] for link in links])
string = "http://i.cantonfair.org.cn/en/"
linksfromcategories = [string + x for x in linksfromcategories]
for link in linksfromcategories:
response = urllib.request.urlopen(link)
soup2 = BeautifulSoup(response, "html.parser")
links2 = soup2.find_all("a", href=re.compile(r"\ExpExhibitorList\.aspx\?categoryno=[0-9]+"))
linksfromsubcategories = ([link["href"] for link in links2])
linksfromsubcategories = [string + x for x in linksfromsubcategories]
for link in linksfromsubcategories:
response = urllib.request.urlopen(link)
soup3 = BeautifulSoup(response, "html.parser")
links3 = soup3.find_all("a", href=re.compile(r"\ExpExhibitorList\.aspx\?categoryno=[0-9]+"))
linksfromsubcategories2 = ([link["href"] for link in links3])
linksfromsubcategories2 = [string + x for x in linksfromsubcategories2]
for link in linksfromsubcategories2:
response2 = urllib.request.urlopen(link)
soup4 = BeautifulSoup(response2, "html.parser")
companylink = soup4.find_all("a", href=re.compile(r"\expCompany\.aspx\?corpid=[0-9]+"))
companylink = ([link["href"] for link in companylink])
companylink = [string + x for x in companylink]
for link in companylink:
response3 = urllib.request.urlopen(link)
soup5 = BeautifulSoup(response3, "html.parser")
companydetail = soup5.find_all("div", id="contact")
for element in companydetail:
companyname = element.a[0].get_text()
print (companyname)
companyaddress = element.a[1].get_text()
print (companyaddress)And I am getting error
我犯了错误
^{pr2}$
companydetail
是ResultSet
。也就是说,它是一个包含许多元素的iterable对象(比如list
或set
)。发生此错误是因为您试图对此ResultSet
对象调用.find_all()
。您应该像这样遍历这个对象,并对ResultSet
中的元素调用find_all()
:或者使用列表理解获取
^{pr2}$companydetail
中所有li
的列表:相关问题 更多 >
编程相关推荐