刮数数据网站在每一个单一日期的美国

Question

我想从2002年的今天开始搜刮一个网站。每个都有自己的一组数字。我打开了检查开关，看起来是这样的：

<tr>
    <td class="chu17 need_blank">42</td>
    <td class="chu17 need_blank">46</td>
    <td class="chu17 need_blank">46</td>
    <td class="chu17 need_blank">46</td>
    <td class="chu17 need_blank">49</td>
    <td class="chu17 need_blank">49</td>
    <td class="chu17 need_blank">61</td>
    <td class="chu17 need_blank">62</td>
    <td class="chu17 need_blank">69</td>
</tr>

我想在</td>之前获取这些数字，并将其打印到.csv文件中。下面是我正在编写的python代码：

import datetime


urls = []

sdate = datetime.date(2002, 1, 5)
edate = datetime.date.today()

delta = edate - sdate

for i in range(delta.days + 1):
    day = sdate + datetime.timedelta(days=i)
    day = day.strftime("%d-%m-%Y")
    urls.append('https://website.php?date=' + day)

with open('output.csv', 'w') as data: 
    file = csv.writer(data)

代码将取代从2002年5月1日网站网址上的每一天。我希望它是：在每一天的程序将收集（刮）的HTML代码上的数字如上所述，一旦它完成了收集它将转到第二天，并做收集第二天的数字集，以此类推

怎么做

Answer 1

您只需要使用BeautifulSoup来解析和获取数据

from bs4 import BeautifulSoup

data = """
    <tr>
        <td class="chu17 need_blank">42</td>
        <td class="chu17 need_blank">46</td>
        <td class="chu17 need_blank">46</td>
        <td class="chu17 need_blank">46</td>
        <td class="chu17 need_blank">49</td>
        <td class="chu17 need_blank">49</td>
        <td class="chu17 need_blank">61</td>
        <td class="chu17 need_blank">62</td>
        <td class="chu17 need_blank">69</td>
    </tr>"""
soup = BeautifulSoup(data, 'html.parser')
numbers = soup.find_all('td', {'class': 'chu17 need_blank'})
for number in numbers:
    print(number.text)