aws3boto下载一些Jenkins作业的超时时间,而不是其他作业

2024-09-27 02:26:15 发布

您现在位置:Python中文网/ 问答频道 /正文

问题:我有一个python脚本,它是通过多配置jenkins作业运行的。作业下载脚本并向其传递参数。它产生10次,因为有10组数据是并行加载的。在某些情况下(并非总是2个或3个作业)会出现s3超时错误。我不知道为什么,因为它不总是相同的工作,也不是所有的时间。在

错误:

Downloading: /var/lib/jenkins/workspace/deploy-es-data/label/swarm/wos/wos_1/json_data/wos/20150707/wos_1/WOK5_20150410045810_WOS_0124.json.gz
Downloading: /var/lib/jenkins/workspace/deploy-es-data/label/swarm/wos/wos_1/json_data/wos/20150707/wos_1/WOK5_20150410045810_WOS_0125.json.gz
Downloading: /var/lib/jenkins/workspace/deploy-es-data/label/swarm/wos/wos_1/json_data/wos/20150707/wos_1/WOK5_20150410045810_WOS_0126.json.gz
Downloading: /var/lib/jenkins/workspace/deploy-es-data/label/swarm/wos/wos_1/json_data/wos/20150707/wos_1/WOK5_20150410045810_WOS_0127.json.gz
Downloading: /var/lib/jenkins/workspace/deploy-es-data/label/swarm/wos/wos_1/json_data/wos/20150707/wos_1/WOK5_20150410045810_WOS_0128.json.gz
Downloading: /var/lib/jenkins/workspace/deploy-es-data/label/swarm/wos/wos_1/json_data/wos/20150707/wos_1/WOK5_20150410045810_WOS_0129.json.gz
Downloading: /var/lib/jenkins/workspace/deploy-es-data/label/swarm/wos/wos_1/json_data/wos/20150707/wos_1/WOK5_20150410045810_WOS_0130.json.gz
Downloading: /var/lib/jenkins/workspace/deploy-es-data/label/swarm/wos/wos_1/json_data/wos/20150707/wos_1/WOK5_20150410045810_WOS_0131.json.gz
Downloading: /var/lib/jenkins/workspace/deploy-es-data/label/swarm/wos/wos_1/json_data/wos/20150707/wos_1/WOK5_20150410045810_WOS_0132.json.gz
Downloading: /var/lib/jenkins/workspace/deploy-es-data/label/swarm/wos/wos_1/json_data/wos/20150707/wos_1/WOK5_20150410045810_WOS_0133.json.gz
Downloading: /var/lib/jenkins/workspace/deploy-es-data/label/swarm/wos/wos_1/json_data/wos/20150707/wos_1/WOK5_20150410045810_WOS_0134.json.gz
Downloading: /var/lib/jenkins/workspace/deploy-es-data/label/swarm/wos/wos_1/json_data/wos/20150707/wos_1/WOK5_20150410045810_WOS_0135.json.gz
> Downloading:
> /var/lib/jenkins/workspace/deploy-es-data/label/swarm/wos/wos_1/json_data/wos/20150707/wos_1/WOK5_20150410045810_WOS_0135.json.gz
> Traceback (most recent call last):   File
> "/var/lib/jenkins/workspace/deploy-es-data/label/swarm/wos/wos_1/es_single_load.py",
> line 1094, in <module>
>     directory = download_s3(wos_data, 'tr-ips-ses-data',access_key, secret_key)   File
> "/var/lib/jenkins/workspace/deploy-es-data/label/swarm/wos/wos_1/es_single_load.py",
> line 31, in download_s3
>     l.get_contents_to_filename(d)   File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 1712, in
> get_contents_to_filename
>     response_headers=response_headers)   File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 1650, in
> get_contents_to_file
>     response_headers=response_headers)   File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 1482, in
> get_file
>     query_args=None)   File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 1535, in
> _get_file_internal
>     for bytes in self:   File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 386, in next
>     data = self.resp.read(self.BufferSize)   File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 413, in
> read
>     return http_client.HTTPResponse.read(self, amt)   File "/usr/lib64/python2.7/httplib.py", line 573, in read
>     s = self.fp.read(amt)   File "/usr/lib64/python2.7/socket.py", line 380, in read
>     data = self._sock.recv(left)   File "/usr/lib64/python2.7/ssl.py", line 714, in recv
>     return self.read(buflen)   File "/usr/lib64/python2.7/ssl.py", line 608, in read
>     v = self._sslobj.read(len or 1024) ssl.SSLError: ('The read operation timed out',) Build step 'Execute shell' marked build as
> failure Finished: FAILURE

Python:

^{pr2}$

Tags: jsondataesvarlibworkspacelabeldeploy

热门问题