我的网页抓取工具有问题
我已经用python做了一个功能,使用Elsevier开发门户上可用的SCOPUS搜索API获取文章详细信息。然而,在我运行该程序后,它给出了与Elsevier portal不同的结果集
埃尔塞韦门户网站https://www.elsevier.com/ SCOPUS API-https:// api.elsevier.com/content/search/scopus?query=all(gene)&;apiKey=7f59af901d2d86f78a 1fd60c1bf9426a
代码:
url='1〕https://api.elsevier.com/content/search/scopus?query=“+query+”&;apiKey='+scp_api
# response object
response = requests.get(url, headers=headers, timeout=30)
soup = BeautifulSoup(response.content, 'lxml')
# convert resonse into josn
obj = json.loads(soup.text)
######## Find required attributes in the response object
for item in obj['search-results']['entry']:
try:
if "prism:Issn" and "prism:issn" not in obj:
issn = item['prism:eIssn']
else:
issn = item['prism:issn']
resp_obj = {"entities": {"Search Engine": "Elsevier SCOPUS Search Engine",
"items": [
{"DOI": item['prism:doi'],
"Title": item['dc:title'],
"URLs": item['prism:url'],
"Authors": item['dc:creator'],
"Publication Name": item['prism:publicationName'],
"ISSN": issn,
"Cited count": item['citedby-count'],
"Affiliation": item['affiliation'][0]['affilname'],
"Type": item['subtypeDescription'],
"Published date": item['prism:coverDate'],
"Abstract": item['prism:publicationName']
}
]}}
# append dict object data
data.append(resp_obj)
except Exception as e: # raise e
pass
# print('error scopus:', e)
任何理由或帮助都将不胜感激
谢谢
目前没有回答
相关问题 更多 >
编程相关推荐