python ssl eof违反了协议wantwriteerror和zeroreturnerror

Question

我正在使用gevent为游泳池运行许多芹菜任务（20000个）（也包括猴子修补所有）。这些任务中的每一项都会通过第三方服务（如adwords）来获取数据。

由于潜在的SSL错误，我一直有任务失败。下面是一些异常的堆栈跟踪（没有特定的顺序，这些是单独任务的失败）。我偶尔也会遇到WantWriteError和ZeroReturnError，但EOF错误似乎最多。

这些错误发生在使用不同的客户端库时，如googleads（用于soap通信的suds库）以及请求和elasticsearch。我猜其中一些库使用urllib3，而另一些库使用urllib2等。

关于EOF问题和强制TLSv1有很多信息，但我似乎找不到有效的解决方案。

我不确定我是否一次运行了太多的请求，如果有什么东西阻塞了或者是什么；任何帮助都将不胜感激，我正在为这个而烦恼。

Traceback (most recent call last):
  ...
  File "/srv/reporting/src/reporting/stats/adwords/client.py", line 58, in _awql_report
    downloader = self._get_client(client_id).GetReportDownloader(version=self.REPORT_DOWNLOADER_VERSION)
  File "/usr/local/lib/python2.7/dist-packages/googleads/adwords.py", line 283, in GetReportDownloader
    return ReportDownloader(self, version, server)
  File "/usr/local/lib/python2.7/dist-packages/googleads/adwords.py", line 400, in __init__
    proxy=proxy_option, cache=self._adwords_client.cache).wsdl.schema
  File "/usr/local/lib/python2.7/dist-packages/suds/client.py", line 115, in __init__
    self.wsdl = reader.open(url)
  File "/usr/local/lib/python2.7/dist-packages/suds/reader.py", line 150, in open
    d = self.fn(url, self.options)
  File "/usr/local/lib/python2.7/dist-packages/suds/wsdl.py", line 136, in __init__
    d = reader.open(url)
  File "/usr/local/lib/python2.7/dist-packages/suds/reader.py", line 74, in open
    d = self.download(url)
  File "/usr/local/lib/python2.7/dist-packages/suds/reader.py", line 92, in download
    fp = self.options.transport.open(Request(url))
  File "/usr/local/lib/python2.7/dist-packages/suds/transport/https.py", line 62, in open
    return HttpTransport.open(self, request)
  File "/usr/local/lib/python2.7/dist-packages/suds/transport/http.py", line 67, in open
    return self.u2open(u2request)
  File "/usr/local/lib/python2.7/dist-packages/suds/transport/http.py", line 132, in u2open
    return url.open(u2request, timeout=tm)
  File "/usr/lib/python2.7/urllib2.py", line 400, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 418, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1216, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1178, in do_open
    raise URLError(err)
URLError: <urlopen error [Errno 8] _ssl.c:504: EOF occurred in violation of protocol>



Traceback (most recent call last):
  ...
  File "/srv/reporting/src/reporting/stats/analytics/client.py", line 57, in get_access_token
    response = requests.post('https://accounts.google.com/o/oauth2/token', data)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 88, in post
    return request('post', url, data=data, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 456, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 559, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 382, in send
    raise SSLError(e, request=request)
SSLError: [Errno bad handshake] (-1, 'Unexpected EOF')




Traceback (most recent call last):
  ...
    self.es.index(index=self.INDICE, doc_type=self.ROOT_CLASS.__name__, body=self.export(obj), id=obj.id)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/utils.py", line 68, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/__init__.py", line 213, in index
    _make_path(index, doc_type, id), params=params, body=body)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/transport.py", line 284, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_requests.py", line 44, in perform_request
    response = self.session.request(method, url, data=body, timeout=timeout or self.timeout)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 456, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 559, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 327, in send
    timeout=timeout
  File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py", line 493, in urlopen
    body=body, headers=headers)
  File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py", line 319, in _make_request
    httplib_response = conn.getresponse(buffering=True)
  File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 407, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 365, in _read_status
    line = self.fp.readline()
  File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 273, in readline
    data = self._sock.recv(self._rbufsize)
  File "/usr/local/lib/python2.7/dist-packages/OpenSSL/SSL.py", line 995, in recv
    self._raise_ssl_error(self._ssl, result)
  File "/usr/local/lib/python2.7/dist-packages/OpenSSL/SSL.py", line 851, in _raise_ssl_error
    raise ZeroReturnError()
ZeroReturnError

Answer 1

所以让我们按每个回溯块来分解这个。第一段结尾是：

  File "/usr/lib/python2.7/urllib2.py", line 1178, in do_open
    raise URLError(err)
URLError: <urlopen error [Errno 8] _ssl.c:504: EOF occurred in violation of protocol>

这是来自urllib2。它接收EOF的事实让我认为服务器在您等待“线程”再次从套接字读取时关闭了连接。您可能希望使用更多的time.sleep（0）来屈服于gevent。

第二个回溯来自请求：

  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 382, in send
    raise SSLError(e, request=request)
SSLError: [Errno bad handshake] (-1, 'Unexpected EOF')

[Errno坏握手]会让我倾向于认为这是一个建立连接的问题，可能是由意外的EOF引起的。那是使用gevent造成的吗？我不确定。

最后的回溯肯定也来自请求，但它也来自PyOpenSSL，不会被urllib3或请求捕获。

  File "/usr/local/lib/python2.7/dist-packages/OpenSSL/SSL.py", line 851, in _raise_ssl_error
    raise ZeroReturnError()
ZeroReturnError

我做了一些搜索，发现“根据pyOpenSSL文档，ZeroReturnError意味着SSL连接已完全关闭。”这告诉我，服务器再次关闭了连接，因为你花了很长时间才从套接字读取任何内容。

简而言之，我认为您需要更频繁地显式地让步，以确保不会出现这些套接字问题。不过这只是个猜测，所以还是少说一点吧。