无法从请求中获取特定的href

2024-10-06 14:31:11 发布

您现在位置:Python中文网/ 问答频道 /正文

我正在尝试使用Pythons请求捕获一个唯一的url

源网站是https://www.realestate.com.au/property/1-10-grosvenor-rd-terrigal-nsw-2260

目标Url是http://www.realestate.com.au/sold/property-unit-nsw-terrigal-124570934

当我试着

(Unique_ID,) = (x.text_content() for x in tree.xpath('//a[@class="property-
value__link--muted rui-button-brand property-value__btn-listing"]'))

CSV返回查看列表

除非我弄错了,否则我已经做了正确的类搜索,因为href不够唯一?我应该做些不同的事情来捕获URL而不是文本吗

如果需要,请填写下面的完整代码

提前谢谢

import requests
import csv
import datetime
import pandas as pd
import csv
from lxml import html

df = pd.read_excel("C:\Python27\Projects\REA_UNIQUE_ID\\UN.xlsx",             sheetname="UN")
dnc = df['Property']
dnc_list = list(dnc)
url_base = "https://www.realestate.com.au/property/"
URL_LIST = []

for nd in dnc_list:
    nd = nd.strip()
    nd = nd.lower()
    nd = nd.replace(" ", "-")
    URL_LIST.append(url_base + nd)

text2search = '''The information provided'''

with open('Auctions.csv', 'wb') as csv_file:
    writer = csv.writer(csv_file)

    for index, url in enumerate(URL_LIST):
        page = requests.get(url)
        print '\r' 'Scraping URL ' + str(index+1) +   ' of  ' + str(len(URL_LIST)),

        if text2search in page.text:
            tree = html.fromstring(page.content)
            (title,) = (x.text_content() for x in tree.xpath('//title'))
            (Unique_ID,) = (x.text_content() for x in    tree.xpath('//a[@class="property-value__link--muted rui-button-brand property-    value__btn-listing"]'))
            #(sold,) = (x.text_content().strip() for x in     tree.xpath('//p[@class="property-value__agent"]'))
            writer.writerow([title, Unique_ID])

Tags: csvtextinimportidtreeurlfor