如何获取具有相同类名和属性的特定项

Question

如何获得具有相同类名和属性的特定项

我要买这3件东西

April 14, 2013

580

Fort Pierce, FL

<dl class="pairsJustified">
<dt>Joined:</dt>
<dd>Apr 14, 2013</dd>
</dl>
<dl class="pairsJustified">
<dt>Messages:</dt>
<dd><a href="search/member?user_id=13302" class="concealed" 
rel="nofollow">580</a></dd>
</dl>

<dl class="pairsJustified">
<dt>Location:</dt>
<dd>
<a href="misc/location-info?location=Fort+Pierce%2C+FL" target="_blank" 
rel="nofollow noreferrer" itemprop="address" class="concealed">Fort 
Pierce, FL</a>

你知道吗 你知道吗

Answer 1

这是一个很好的起点：

In [18]: for a in response.css('.extraUserInfo'):
    ...:     print(a.css('*::text').extract())
    ...:     print('\n\n\n')
    ...:     
['\n', '\n', '\n', '\n']  # < this (and other outputs like this) is because there is an extra `extraUserInfo` class block above the desired info block if the user has a user group picture/avatar below their username




['\n', '\n', 'Joined:', '\n', 'Mar 24, 2013', '\n', '\n', '\n', 'Messages:', '\n', '6,747', '\n', '\n']




['\n', '\n', '\n', '\n']




['\n', '\n', 'Joined:', '\n', 'Mar 24, 2013', '\n', '\n', '\n', 'Messages:', '\n', '6,747', '\n', '\n']




['\n', '\n', 'Joined:', '\n', 'Apr 14, 2013', '\n', '\n', '\n', 'Messages:', '\n', '580', '\n', '\n', '\n', 'Location:', '\n', '\n', 'Fort Pierce, FL', '\n', '\n', '\n']




['\n', '\n', 'Joined:', '\n', 'Oct 20, 2012', '\n', '\n', '\n', 'Messages:', '\n', '2,476', '\n', '\n', '\n', 'Location:', '\n', '\n', 'Philadelphia, PA', '\n', '\n', '\n']




['\n', '\n', 'Joined:', '\n', 'Dec 11, 2012', '\n', '\n', '\n', 'Messages:', '\n', '2,938', '\n', '\n', '\n', 'Location:', '\n', '\n', 'Colorado', '\n', '\n', '\n']




['\n', '\n', 'Joined:', '\n', 'Sep 30, 2016', '\n', '\n', '\n', 'Messages:', '\n', '833', '\n', '\n', '\n', 'Location:', '\n', '\n', 'Indiana', '\n', '\n', '\n']


...

有很多方法可以做到这一点。稍微摆弄一下就会得到你喜欢的格式。上面的方法只是一个很好的起点，因为有很多行只有换行符列表作为输出，这是因为（看起来）用户信息块在用户有用户组图像的地方（比如亚利桑那州的特斯拉），然后extraUserInfo类也被用来对html块进行分组。会有更好的方法来分组

基本上，response.css（'.extraUserInfo'）将使用类extraUserInfo聚合所有块，这些块似乎是保存您要查找的用户信息的块。 从那里用::text伪选择器提取所有底层文本并解析数组

如果您仔细查看html结构，这样提取它的方式会减少以后的处理工作，那么肯定有更好的方法来实现这一点，但这应该会让您走上正确的轨道。CSS选择器或xpath文档应该很有帮助

Answer 2

使用它们位于<dd>标记下，使用.find_all()：

from bs4 import BeautifulSoup

test = '''<dl class="pairsJustified">
<dt>Joined:</dt>
<dd>Apr 14, 2013</dd>
</dl>
<dl class="pairsJustified">
<dt>Messages:</dt>
<dd><a href="search/member?user_id=13302" class="concealed" 
rel="nofollow">580</a></dd>
</dl>

<dl class="pairsJustified">
<dt>Location:</dt>
<dd>
<a href="misc/location-info?location=Fort+Pierce%2C+FL" target="_blank" 
rel="nofollow noreferrer" itemprop="address" class="concealed">Fort Pierce, FL</a>'''

soup = BeautifulSoup(test, 'html.parser')
data = soup.find_all("dd")
for d in data:
    print(d.text.strip())

输出：

Apr 14, 2013
580
Fort Pierce, FL