如何从HTML标题中获取带引号的字符串？

Question

给定这个HTML代码段，我如何使用python包请求或xlml来查找href=之后引用的字符串？你知道吗

<dl>
    <dt><a href="oq-phys.htm">
        <b>Physics and Astronomy</b></a>
    <dt><a href="oq-math.htm">
        <b>Mathematics</b></a>
    <dt><a href="oq-life.htm">
        <b>Life Sciences</b></a>
    <dt><a href="oq-tech.htm">
        <b>Technology</b></a>
    <dt><a href="oq-geo.htm">
        <b>Earth and Environmental Science</b></a>
</dl>

Answer 1

对于上面的示例，假设我们有包含上述代码段的html\u字符串。你知道吗

import requests
import lxml.etree as LH
html_string =  LH.fromstring(requests.get('http://openquestions.com').text)

for quoted_link in html_string.xpath('//a'): print(quoted_link.attrib['href'], quoted_link.text_content())

Answer 2

有很多方法可以剥这只猫的皮。下面是一个requests/lxml解决方案，它不包含（显式）for循环：

import requests
from lxml.html import fromstring

req = requests.get('http://www.openquestions.com')
resp = fromstring(req.content)
hrefs = resp.xpath('//dt/a/@href') 
print(hrefs)

编辑

我为什么这样写：

• 我更喜欢XPath而不是CSS选择器
• 很快

基准：

import requests,bs4
from lxml.html import fromstring
import timeit

req = requests.get('http://www.openquestions.com').content

def myfunc() :
    resp = fromstring(req)
    hrefs = resp.xpath('//dl/dt/a/@href')

print("Time for lxml: ", timeit.timeit(myfunc, number=100))

##############################################################

resp2 = requests.get('http://www.openquestions.com').content

def func2() :
    soup = bs4.BeautifulSoup(resp2, 'html.parser')
    hrefs = [a['href'] for a in soup.select('dl dt a')]

print("Time for beautiful soup:", timeit.timeit(func2, number=100))

输出：

('Time for lxml: ', 0.09621267095780464)
('Time for beautiful soup:', 0.8594218329542824)

Answer 3

to find the quoted strings after href=

短requests+beautifulsoup溶液：

import requests, bs4

soup = bs4.BeautifulSoup(requests.get('http://.openquestions.com').content, 'html.parser')
hrefs = [a['href'] for a in soup.select('dl dt a')]
print(hrefs)

输出：

['oq-phys.htm', 'oq-math.htm', 'oq-life.htm', 'oq-tech.htm', 'oq-geo.htm', 'oq-map.htm', 'oq-about.htm', 'oq-howto.htm', 'oqc/oqc-home.htm', 'oq-indx.htm', 'oq-news.htm', 'oq-best.htm', 'oq-gloss.htm', 'oq-quote.htm', 'oq-new.htm']