从国家漏洞数据库中抓取数据：不知道点击按钮（Mechanize+Python）

Question

我正试图从国家脆弱性数据库中获取一些数据(http://web.nvd.nist.gov). 我想做的是输入一个搜索词，它会给我带来前20个结果，然后把这些数据刮掉。然后我想点击“next20”，直到我遍历所有结果。在

我可以成功地提交搜索条件，但点击“下一个20”根本不起作用。在

我使用的工具是Python+Mechanize

这是我的代码：

# Browser
b = mechanize.Browser()

# The URL to this service
URL = 'http://web.nvd.nist.gov/view/vuln/search'
Search = ['Linux', 'Mac OS X', 'Windows']

def searchDB():
    SearchCounter=0
    for i in Search:
        # Load the page
        read = b.open(URL)
        # Select the form
        b.select_form(nr=0)
        # Fill out the search form
        b['vulnSearchForm:text'] = Search[int(SearchCounter)] 
        b.submit('vulnSearchForm:j_id120')
        result=b.response().read()
        file=open(Search[SearchCounter]+".txt","w")
        file.write(result)

        '''Here is where the problem is. vulnResultsForm:j_id116 is value of the "next 20 button'''
        b.select_form(nr = 0)
        b.form.click('vulnResultsForm:j_id116')
        result=b.response().read()

if __name__ == '__main__':
    searchDB()

Answer 1

从b.form.click的docstring：

Return request that would result from clicking on a control.

The request object is a urllib2.Request instance, which you can pass to urllib2.urlopen (or ClientCookie.urlopen).

所以：

request = b.form.click('vulnResultsForm:j_id116')
b.open(request)
result = b.response().read()