为多个URL运行python不会得到任何输出

Question

我试图运行这个网站的代码工程刮板当我只使用一个网址，但当我添加多它给我没有输出。我需要它来运行通过不同的网址和刮的信息。你知道吗

> Blockquote
>`import requests
>import csv
>from bs4 import BeautifulSoup
>from html.parser import HTMLParser
>from time import sleep
from random import randint
<import urllib.request

r=requests.get('https://www.qiagen.com/us/products/a-z-list/#&&s=Ascending&pg=55&q=&l=')
c=r.content
s=BeautifulSoup(c,"html.parser")


product_urls = ['https://www.qiagen.com/us/shop/pcr/primer-sets/miscript-precursor-assays/#orderinginformation', 
'https://www.qiagen.com/us/shop/pcr/primer-sets/miscript-primer-assay-plate/#orderinginformation', 
'https://www.qiagen.com/us/shop/pcr/primer-sets/miscript-primer-assays/#orderinginformation', 
'https://www.qiagen.com/us/shop/genes-and-pathways/technology-portals/browse-qpcr/mirna-gene-expression/mirna-isolation/miscript-single-cell-qpcr-kit/#orderinginformation']

for url in product_urls:
    page = urllib.request.urlopen(url)
    s = BeautifulSoup(page,"html.parser")

getall = s.find_all("div",{"class":"gene_globe_segment_0_OrderingInfoPane"})
getall

for i in getall:
    product_name = (i.find('div',{'class':'title'}).text.strip())
    product_discription = (i.find('div',{'class': 'copy'}).text.strip())
    product_number = (i.find('td',{'class': 'textLeft paddingTopLess'}).text.strip())
    cat_number = (i.find('td',{'class': 'textRight paddingTopLess'}).text.strip())
    product_price = (i.find('td',{'class': 'textRight paddingTopLess priceSingle'}).text.strip())

for i in getall:
    print(i.find('div',{'class':'title'}).text.strip()) #product name
    print(i.find('div',{'class': 'copy'}).text.strip()) #product discription
    print(i.find('td',{'class': 'textLeft paddingTopLess'}).text.strip()) #product number
    print(i.find('td',{'class': 'textRight paddingTopLess'}).text.strip()) #cat number
    print(i.find('td',{'class': 'textRight paddingTopLess priceSingle'}).text.strip()) #product price

    print(' ')`<

Answer 1

你的剧本有几个问题。在主容器中定义了错误的class名称。您的脚本缩进错误。最后，你需要调整你的选择器，使之能够处理不同的站点。我已经把你们的印刷品减少到两件，以便给你们一个透明的演示。我试着帮你收拾一下。我在下面粘贴的修改后的脚本是有效的。你知道吗

给你：

import requests
from bs4 import BeautifulSoup

product_urls = [
'https://www.qiagen.com/us/shop/pcr/primer-sets/miscript-precursor-assays/#orderinginformation', 
'https://www.qiagen.com/us/shop/pcr/primer-sets/miscript-primer-assay-plate/#orderinginformation', 
'https://www.qiagen.com/us/shop/pcr/primer-sets/miscript-primer-assays/#orderinginformation', 
]

for URL in product_urls:
    page = requests.get(URL)
    soup = BeautifulSoup(page.text,"lxml")

    for item in soup.select(".content"):
        product_name = item.select_one('.title').text.strip()
        product_discription = item.select_one('.copy').text.strip()
        print("Name: {}\n\nDescription: {}\n\n".format(product_name,product_discription))