Python循环用于web报废

Question

我试图从a website中提取数据。你知道吗

我写的代码是

import csv

import requests 

from bs4 import BeautifulSoup

page = requests.get("http://www.realcommercial.com.au/sold/property-offices-
  retail-showrooms+bulky+goods-land+development-hotel+leisure+medical+consulting-other-in-wa/list-1?includePropertiesWithin=includesurrounding&activeSort=list-date&autoSuggest=true")

soup = BeautifulSoup(page.content, 'html.parser')

Address_1 = soup.find('p', attrs ={'class' :'details-panel__address'})

Address = Address.text.strip()

我得到的结果是

 'GF 255 Adelaide TerracePerth, WA 6000'

这只是一个列表的一行地址。你知道吗

当我使用soup.find_all时，得到的结果如下：

 p class="details-panel__address" data-reactid="90"><span class="details-
 panel__address-text text-truncate" data-reactid="91">GF 255 Adelaide 
 Terrace</span><span class="details-panel__address-text text-truncate" data-
 reactid="92">Perth, WA 6000</span></p

  p class="details-panel__address" data-reactid="122"><span class="details-
 panel__address-text text-truncate" data-reactid="123">369-371 Oxford 
 Street</span><span class="details-panel__address-text text-truncate" data-
 reactid="124">Mount Hawthorn, WA 6016</span></p>,

  p class="details-panel__address" data-reactid="148"><span class="details-
 panel__address-text text-truncate" data-reactid="149">2 Lloyd Street</span>
 <span class="details-panel__address-text text-truncate" data-
 reactid="150">Midland, WA 6056</span></p>,

  p class="details-panel__address" data-reactid="172"><span class="details-
 panel__address-text text-truncate" data-reactid="173">Bluenote Building, 16/162 
 Colin Street</span><span class="details-panel__address-text text-truncate" 
 data-reactid="174">West Perth, WA 6005</span></p>,

  p class="details-panel__address" data-reactid="196"><span class="details-
 panel__address-text text-truncate" data-reactid="197">Bluenote Building, 10/162 
 Colin Street</span><span class="details-panel__address-text text-truncate" 
 data-reactid="198">West Perth, WA 6005</span></p>

请建议我应该怎么做，以提取有关地址，物业类型，销售日期，销售价值，面积，机构名称，代理人的姓名和电话号码的所有物品在这个页面上的信息。另外，我不知道如何使用loop打开特定页面上的每个列表并从中获取信息。你知道吗

Answer 1

soup.find_all返回元素。到获取您必须遍历元素列表才能提取具有text属性的文本的文本。你知道吗

import requests 

from bs4 import BeautifulSoup

page = requests.get("""http://www.realcommercial.com.au/sold/property-offices-
  retail-showrooms+bulky+goods-land+development-hotel+leisure+medical+consulting-other-in-wa/list-1?includePropertiesWithin=includesurrounding&activeSort=list-date&autoSuggest=true""")

soup = BeautifulSoup(page.content, 'html.parser')

Address_1 = soup.find_all('p', attrs ={'class' :'details-panel__address'})
address_list = [ address.text.strip() for address in Address_1]
print(address_list)
links = soup.find_all('a', attrs ={'class' :'details-panel'})
hrefs = [link['href'] for link in links]
print(hrefs)
# Now iterate through the list of urls and extract the required data