我正在尝试使用Pythons请求捕获一个唯一的url
源网站是https://www.realestate.com.au/property/1-10-grosvenor-rd-terrigal-nsw-2260
目标Url是http://www.realestate.com.au/sold/property-unit-nsw-terrigal-124570934
当我试着
(Unique_ID,) = (x.text_content() for x in tree.xpath('//a[@class="property-
value__link--muted rui-button-brand property-value__btn-listing"]'))
CSV返回查看列表
除非我弄错了,否则我已经做了正确的类搜索,因为href不够唯一?我应该做些不同的事情来捕获URL而不是文本吗
如果需要,请填写下面的完整代码
提前谢谢
import requests
import csv
import datetime
import pandas as pd
import csv
from lxml import html
df = pd.read_excel("C:\Python27\Projects\REA_UNIQUE_ID\\UN.xlsx", sheetname="UN")
dnc = df['Property']
dnc_list = list(dnc)
url_base = "https://www.realestate.com.au/property/"
URL_LIST = []
for nd in dnc_list:
nd = nd.strip()
nd = nd.lower()
nd = nd.replace(" ", "-")
URL_LIST.append(url_base + nd)
text2search = '''The information provided'''
with open('Auctions.csv', 'wb') as csv_file:
writer = csv.writer(csv_file)
for index, url in enumerate(URL_LIST):
page = requests.get(url)
print '\r' 'Scraping URL ' + str(index+1) + ' of ' + str(len(URL_LIST)),
if text2search in page.text:
tree = html.fromstring(page.content)
(title,) = (x.text_content() for x in tree.xpath('//title'))
(Unique_ID,) = (x.text_content() for x in tree.xpath('//a[@class="property-value__link--muted rui-button-brand property- value__btn-listing"]'))
#(sold,) = (x.text_content().strip() for x in tree.xpath('//p[@class="property-value__agent"]'))
writer.writerow([title, Unique_ID])
text_content()
只允许获取文本。试着像下面那样刮@href
相关问题 更多 >
编程相关推荐