使用Scrapy在下一个同级标记中获取信息的Xpath

Question

我正试着接触Scrapy，现在我试着从一个词源网站上提取信息：http://www.etymonline.com 现在，我只想知道单词和它们的原始描述。这是一个常见的HTML代码块在etymonline中的显示方式：

<dt>
  <a href="/index.php?term=address&allowed_in_frame=0">address (n.)</a>
  <a href="http://dictionary.reference.com/search?q=address" class="dictionary" title="Look up address at Dictionary.com">
    <img src="graphics/dictionary.gif" width="16" height="16" alt="Look up address at Dictionary.com" title="Look up address at Dictionary.com"/>
  </a>
</dt>
<dd>
  1530s, "dutiful or courteous approach," from <a href="/index.php?term=address&allowed_in_frame=0" class="crossreference">address</a> (v.) and from French <span class="foreign">adresse</span>. Sense of "formal speech" is from 1751. Sense of "superscription of a letter" is from 1712 and led to the meaning "place of residence" (1888).
</dd>

单词包含在<dt>标记和下一个同级标记<dd>中的描述中。 要在像http://www.etymonline.com/index.php?l=a&p=9&allowed_in_frame=0这样的页面上获取单词列表，可以编写word = sel.xpath('//dl/dt/a/text()').extract()。

然后我尝试循环遍历这个单词列表，并使用这行代码info = selInfo.xpath("//dl/dt[a='"+word[i]+"']/following-sibling::dd")提取相关信息。但似乎没用。有什么想法吗？

Answer 1

使用以下同级的解决方案。

class SingleSpider(scrapy.Spider):
    name = "etym"
    allowed_domains = ["etymonline.com"]
    start_urls = [
        "http://www.etymonline.com/index.php?l=d&allowed_in_frame=0"]

    def parse(self, response):


        for nodes in response.xpath('//dl'):
            for i in nodes.xpath('dt'):
                print i.xpath('a/text()').extract()   
                print i.xpath('following-sibling::dd[1]/text()').extract()    

基本上：

• 你一个接一个地得到Dt元素
• 打印链接中包含的文本
• 移到下一个同级并打印包含的文本
• 列表项

这里是输出的摘录：

[u'daiquiri (n.)'] [u'type of alcoholic drink, 1920 (first recorded in F. Scott Fitzgerald), from ', u', name of a district or village in eastern Cuba.']

[u'dairy (n.)'] [u'late 13c., "building for making butter and cheese; dairy farm," formed with Anglo-French ', u' affixed to Middle English ', u' (in ', u' "dairymaid"), from Old English ', u' "kneader of bread, housekeeper, female servant" (see ', u' (n.1)). The purely native word was ', u'.']

[u'dais (n.)'] [u'mid-13c., from Anglo-French ', u', Old French ', u' "table, platform," from Latin ', u' "disk-shaped object," also, by medieval times, "table," from Greek ', u' "quoit, disk, dish" (see ', u' (n.)). Died out in English c.1600, preserved in Scotland, revived 19c. by antiquarians.']