获取错误:“证书验证失败:无法获取本地颁发者证书”尝试在PyCharm中webscrape PDF

2024-10-04 09:17:50 发布

您现在位置:Python中文网/ 问答频道 /正文

以下是我一直试图修复的代码:

import requests
from urllib.parse import urljoin
from bs4 import BeautifulSoup
url = "https://www.pbs.gov.pk/node/3391/?name=001"

#If there is no such folder, the script will create one automatically
folder_location = r'infoFA'
if not os.path.exists(folder_location):os.mkdir(folder_location)
response = requests.get(url)
soup= BeautifulSoup(response.text, "html.parser")     
for link in soup.select("a[href$='.pdf']"):
    #Name the pdf files using the last portion of each link which are unique in this case
    filename = os.path.join(folder_location,link['href'].split('/')[-1])
    with open(filename, 'wb') as f:
        f.write(requests.get(urljoin(url,link['href'])).content)

由此,我在PyCharm中得到以下错误:

Traceback (most recent call last):
  File "/Users/justinlesko/PycharmProjects/pythonProject8/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/Users/justinlesko/PycharmProjects/pythonProject8/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
  File "/Users/justinlesko/PycharmProjects/pythonProject8/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
    conn.connect()
  File "/Users/justinlesko/PycharmProjects/pythonProject8/venv/lib/python3.9/site-packages/urllib3/connection.py", line 411, in connect
    self.sock = ssl_wrap_socket(
  File "/Users/justinlesko/PycharmProjects/pythonProject8/venv/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "/Users/justinlesko/PycharmProjects/pythonProject8/venv/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1040, in _create
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/justinlesko/PycharmProjects/pythonProject8/venv/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/Users/justinlesko/PycharmProjects/pythonProject8/venv/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/Users/justinlesko/PycharmProjects/pythonProject8/venv/lib/python3.9/site-packages/urllib3/util/retry.py", line 574, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.pbs.gov.pk', port=443): Max retries exceeded with url: /node/3391/?name=001 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/justinlesko/PycharmProjects/pythonProject8/main.py", line 10, in <module>
    response = requests.get(url)
  File "/Users/justinlesko/PycharmProjects/pythonProject8/venv/lib/python3.9/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/justinlesko/PycharmProjects/pythonProject8/venv/lib/python3.9/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/justinlesko/PycharmProjects/pythonProject8/venv/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/justinlesko/PycharmProjects/pythonProject8/venv/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/Users/justinlesko/PycharmProjects/pythonProject8/venv/lib/python3.9/site-packages/requests/adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='www.pbs.gov.pk', port=443): Max retries exceeded with url: /node/3391/?name=001 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)')))

Process finished with exit code 1

我已尝试以下方法来修复此ssl错误: -在requests.get(url)中将verify设置为“False” -安装和重新安装python 3.9文件夹中的证书 -在首选项中将未知证书设置为“信任”


Tags: inpyurlsslvenvlibpackagesline