Cloud Vision API客户端引发操作系统错误“打开的文件太多”

Question

我在使用Python通过Cloud Vision API客户端运行标签检测时遇到了一个错误：“打开的文件太多”。
在本文之前，当我在GitHub上问这个问题时，维护人员给了我一个建议，这个问题是一般的python问题，而不是API。
听了这个建议，我还不明白为什么Python抛出了“太多打开的文件”。
我做了日志记录，结果显示urllib3引发了这样的错误，尽管我没有显式地导入该包。
我错了什么？请帮帮我。
我的环境是

• Ubuntu 16.04.3 LTS（GNU/Linux 4.4.0-112-generic x86_64）
• Python 3.5.2版
• 谷歌云视觉（0.31.1）

错误日志：

[2018-05-25 20:18:46,573] {label_detection.py:60} DEBUG - success open decile_data/image/src/00000814.jpg
[2018-05-25 20:18:46,573] {label_detection.py:62} DEBUG - success convert image to types.Image
[2018-05-25 20:18:46,657] {requests.py:117} DEBUG - Making request: POST https://accounts.google.com/o/oauth2/token
[2018-05-25 20:18:46,657] {connectionpool.py:824} DEBUG - Starting new HTTPS connection (1): accounts.google.com
[2018-05-25 20:18:46,775] {connectionpool.py:396} DEBUG - https://accounts.google.com:443 "POST /o/oauth2/token HTTP/1.1" 200 None
[2018-05-25 20:18:47,803] {label_detection.py:60} DEBUG - success open decile_data/image/src/00000815.jpg
[2018-05-25 20:18:47,803] {label_detection.py:62} DEBUG - success convert image to types.Image
[2018-05-25 20:18:47,896] {requests.py:117} DEBUG - Making request: POST https://accounts.google.com/o/oauth2/token
[2018-05-25 20:18:47,896] {connectionpool.py:824} DEBUG - Starting new HTTPS connection (1): accounts.google.com
[2018-05-25 20:18:47,902] {_plugin_wrapping.py:81} ERROR - AuthMetadataPluginCallback "<google.auth.transport.grpc.AuthMetadataPlugin object at 0x7fcd94eb7dd8>" raised exception!
Traceback (most recent call last):
  File "/home/ishiyama/tensorflow/lib/python3.5/site-packages/urllib3/util/ssl_.py", line 313, in ssl_wrap_socket
OSError: [Errno 24] Too many open files

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ishiyama/tensorflow/lib/python3.5/site-packages/urllib3/connectionpool.py", line 601, in urlopen
  File "/home/ishiyama/tensorflow/lib/python3.5/site-packages/urllib3/connectionpool.py", line 346, in _make_request
  File "/home/ishiyama/tensorflow/lib/python3.5/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn
  File "/home/ishiyama/tensorflow/lib/python3.5/site-packages/urllib3/connection.py", line 326, in connect
  File "/home/ishiyama/tensorflow/lib/python3.5/site-packages/urllib3/util/ssl_.py", line 315, in ssl_wrap_socket
urllib3.exceptions.SSLError: [Errno 24] Too many open files

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ishiyama/tensorflow/lib/python3.5/site-packages/requests/adapters.py", line 440, in send
  File "/home/ishiyama/tensorflow/lib/python3.5/site-packages/urllib3/connectionpool.py", line 639, in urlopen
  File "/home/ishiyama/tensorflow/lib/python3.5/site-packages/urllib3/util/retry.py", line 388, in increment
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='accounts.google.com', port=443): Max retries exceeded with url: /o/oauth2/token (Caused by SSLError(OSError(24, 'Too many open files'),))

上述错误导出的脚本如下：

^{pr2}$

Answer 1

cloudvisionapi的content limit每个请求有16个图像，请求quota limit每分钟有600个请求。因此，超出限制可能会导致错误。在

Answer 2

这不是谷歌的限制。我猜，你已经达到了进程允许的最大打开文件数。您可以在进程运行时检查所有打开的文件。使用类似“lsof”的命令来查看进程的所有打开的文件。我猜你会看到很多ipv4，ipv6连接被打开。如果是，继续阅读。在

您在这里为每个映像打开客户端，这意味着为每个映像打开一个安全的已验证连接。使线客户机全局化。在

接受“客户=vision.ImageAnnotatorClient从该函数中退出。使客户机全球化。将使用一个打开的连接。这应该能解决你的问题。在

Answer 3

如果要处理的文件太多，只需增加每个进程可以打开的最大文件数。在

您可以通过：

ulimit -n your_number

例如：

^{pr2}$