使用Selenium在多个元素上循环

Question

我正在努力浏览以下网站：

finsight.com/product/us/abs/ee.

特别是，对于每一行，我都试图提取类型（AUTO或CBMS）、公司名称，并下载链接。下面是每行1的源代码摘录。然而，当我运行循环时，我只得到第一行的名称和链接（在本例中是AUTo CarMax AUTo Owner Trust 2018-2）。你知道吗

到目前为止，我有以下代码：

import selenium
import time
import requests
from selenium import webdriver


url = "https://finsight.com/product/us/abs/ee"
driver = webdriver.Chrome()

driver.get(url)

time.sleep(1)

company_row = driver.find_elements_by_xpath("//div[@class='ee-item                 portlet box ng-scope']")

for row in company_row:
    RD_element = row.find_element_by_xpath("//a[@class='related-document ng-scope']")
    company_name = row.find_element_by_xpath("//span[contains(@class,'filing-left filing-issuer ng-binding')]")
    company_type = row.find_element_by_xpath("//span[contains(@class,'filing-left filing-sector ng-binding')]")

    RD_link = RD_element.get_attribute('href')

    print (company_name.text)
    print (company_type.text)
    print (RD_link)

我的代码输出如下：

DevTools listening on ws://127.0.0.1:12060/devtools/browser/c5d13168-0976-41c7-937c-ff2bd4cd99fe
CarMax Auto Owner Trust 2018-2
AUTO
https://finsight.com/api/download-csv?file_id=15395
CarMax Auto Owner Trust 2018-2
AUTO
https://finsight.com/api/download-csv?file_id=15395
CarMax Auto Owner Trust 2018-2
AUTO
https://finsight.com/api/download-csv?file_id=15395
CarMax Auto Owner Trust 2018-2
AUTO
https://finsight.com/api/download-csv?file_id=15395
CarMax Auto Owner Trust 2018-2

Answer 1

以下是您案例的工作代码：

from selenium import webdriver
from selenium.webdriver.support import ui
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By


driver = webdriver.Chrome()
driver.get("https://finsight.com/product/us/abs/ee")

company_rows = ui.WebDriverWait(driver, 10).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, ".ee-item.portlet.box.ng-scope")))

for row in company_rows:

    RD_element = row.find_element_by_css_selector(".related-document.ng-scope")
    RD_link = RD_element.get_attribute("href")
    company_name = row.find_element_by_css_selector(".filing-left.filing-issuer.ng-binding")
    company_type = row.find_element_by_css_selector(".filing-left.filing-sector.ng-binding")

    print(company_name.text)
    print(company_type.text)
    print(RD_link)

输出：

CarMax Auto Owner Trust 2018-2
AUTO
https://finsight.com/api/download-csv?file_id=15399
CarMax Auto Owner Trust 2018-2
AUTO
https://finsight.com/api/download-csv?file_id=15395
BENCHMARK 2018-B3 COMMERCIAL MORTGAGE TRUST
CMBS
https://finsight.com/api/download-csv?file_id=15325
BANK 2018-BNK11
CMBS
https://finsight.com/api/download-csv?file_id=15209
Santander Drive Auto Receivables Trust 2018-2
AUTO
https://www.sec.gov/Archives/edgar/data/1383094/000095013118000509/sdart182ex103_0404-1831.xml
Hyundai Auto Receivables Trust 2018-A
AUTO
https://www.sec.gov/Archives/edgar/data/1260125/000114420418019209/tv490265_ex103.xml
BMW Vehicle Owner Trust 2018-A
AUTO
https://www.sec.gov/Archives/edgar/data/1725617/000092963818000431/exhibit103.xml
BMW Vehicle Lease Trust 2017-2
AUTO
https://www.sec.gov/Archives/edgar/data/1716665/000092963818000430/exhibit103.xml
BMWLT 2017-1
AUTO
https://www.sec.gov/Archives/edgar/data/1694920/000092963818000429/exhibit103.xml
GM Financial Consumer Automobile Receivables Trust 2018-2
AUTO
https://www.sec.gov/Archives/edgar/data/1347185/000134718518000012/exh103loanv2.xml
BANK 2017-BNK8
CMBS
https://finsight.com/api/download-csv?file_id=15091
Morgan Stanley Capital I Trust 2017-H1
CMBS
https://finsight.com/api/download-csv?file_id=15087
Morgan Stanley Capital I Trust 2017-HR2
CMBS
https://finsight.com/api/download-csv?file_id=15083
BANK 2017-BNK5
CMBS
https://finsight.com/api/download-csv?file_id=15079
Morgan Stanley Bank of America Merrill Lynch Trust 2017-C33
CMBS
https://finsight.com/api/download-csv?file_id=15075
BANK 2018-BNK10
CMBS
https://finsight.com/api/download-csv?file_id=15059
Wells Fargo Commercial Mortgage Trust 2017-RC1
CMBS
https://finsight.com/api/download-csv?file_id=15055
Wells Fargo Commercial Mortgage Trust 2017-RB1
CMBS
https://finsight.com/api/download-csv?file_id=15051
Wells Fargo Commercial Mortgage Trust 2017-C42
CMBS
https://finsight.com/api/download-csv?file_id=15047
Wells Fargo Commercial Mortgage Trust 2017-C41
CMBS
https://finsight.com/api/download-csv?file_id=15043

PS：这里我使用CSS选择器而不是XPath。你知道吗