擅长:python、mysql、java
<p>既然@chitown88已经建议包含<code>User-Agent</code>,我想补充一点,您可能会使用<code>internal API</code>,这是:
<code>https://www.sciencedirect.com/search/api?qs=hydrogen&show=25&sortBy=date&years=2018&navigation=true</code></p>
<p>这会快得多(当然,如果你的目标是获得文章的<code>URL</code>),然后你可以做一些类似的事情</p>
<pre><code>...
r = requests.get('https://www.sciencedirect.com/search/api?qs=hydrogen&show=25&sortBy=date&years=2018&navigation=true')
data = r.json()
for result in data['searchResults']:
print(result['pdf']['getAccessLink']
...
</code></pre>