get调用中的aiohttp ClientSession错误：看到的头字节太多；返回检测到的溢出

Question

🐞 描述错误 在AIOHTTPClientSession中发送调用时，返回错误too many header bytes seen; overflow detected

💡 复制 没有精确的代码可以复制，因为它使用专有的源代码。守则：

import asyncio
import aiohttp

loop = asyncio.get_event_loop()

url = "https://url.com"
bodies = [{"body": 1}, {"body": 2}, {"body": 3}]

async def request(body, session):
    async with session.post(url, json=body) as resp:
        return await resp.json()
    #return await es_client.search(index=es_index, doc_type=es_doc_type, body=query)

async def msearch(bodies):
    async with aiohttp.ClientSession() as session:
        tasks = [asyncio.ensure_future(request(body, session)) for body in bodies]
        return await asyncio.gather(*tasks)

results = loop.run_until_complete(msearch(bodies))

💡 预期行为 这段代码应该返回一个json响应

它适用于大多数调用，但如果结果？/header？/。。。？电话号码越来越大，这条信息被抛出了吗

📋 日志/回溯

Traceback (most recent call last):
  File "/Users/stijngeuens/.virtualenvs/findme/lib/python3.7/site-packages/aiohttp/client_reqrep.py", line 847, in start
    message, payload = await self._protocol.read()  # type: ignore  # noqa
  File "/Users/stijngeuens/.virtualenvs/findme/lib/python3.7/site-packages/aiohttp/streams.py", line 591, in read
    await self._waiter
  File "/Users/stijngeuens/.virtualenvs/findme/lib/python3.7/site-packages/aiohttp/client_proto.py", line 201, in data_received
    messages, upgraded, tail = self._parser.feed_data(data)
  File "aiohttp/_http_parser.pyx", line 523, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadHttpMessage: 400, message='too many header bytes seen; overflow detected'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/Users/stijngeuens/.pyenv/versions/3.7.3/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3326, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-25-8dd725416572>", line 11, in <module>
    results = loop.run_until_complete(msearch(bodies))
  File "/Users/stijngeuens/.pyenv/versions/3.7.3/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "<ipython-input-25-8dd725416572>", line 9, in msearch
    return await asyncio.gather(*tasks)
  File "<ipython-input-25-8dd725416572>", line 2, in request
    async with session.post(url, json=body) as resp:
  File "/Users/stijngeuens/.virtualenvs/findme/lib/python3.7/site-packages/aiohttp/client.py", line 1012, in __aenter__
    self._resp = await self._coro
  File "/Users/stijngeuens/.virtualenvs/findme/lib/python3.7/site-packages/aiohttp/client.py", line 504, in _request
    await resp.start(conn)
  File "/Users/stijngeuens/.virtualenvs/findme/lib/python3.7/site-packages/aiohttp/client_reqrep.py", line 852, in start
    message=exc.message, headers=exc.headers) from exc
aiohttp.client_exceptions.ClientResponseError: 400, message='too many header bytes seen; overflow detected', url=URL('https://url.com')

📋 您的Python版本

Python 3.7.3 (default, Oct 18 2019, 10:37:04) 

📋 您的aiohttp/yarl/multidict发行版版本

aiohttp==3.6.2
yarl==1.3.0
multidict==4.5.2

📋 附加上下文 在Mac OS上运行。Pycharm和Docker中都有错误 呼叫将转到本地ES实例进行开发 使用客户端库（ClientSession）

Answer 1

aiohttp.client_exceptions.ClientResponseError: 400, message='too many header bytes seen; overflow detected', url=URL('https://url.com')

这意味着服务器正在用400 Bad Request错误进行应答，而服务器抱怨头的格式不正确

根据消息，很可能服务器的编程方式需要一定数量的头。当前请求通过以下标题发送：

{
    "Host": "example.com",
    "Accept": "*/*",
    "Accept-Encoding": "gzip, deflate",
    "User-Agent": "Python/3.8 aiohttp/3.7.4.post0",
    "Content-Length": "11",
    "Content-Type": "application/json",
}

您可以通过以下方式进行检查：

async def request(body, session):
    async with session.post(url, json=body) as resp:
        print(dict(resp.request_info.headers))
        return await resp.json()

我会检查服务器的文档，看看他们的post端点到底期望什么。如果您正在对某个端点进行反向工程，我建议也复制原始请求发送到服务器的头。这将有望解决您的问题

我无法深入研究这个问题，因为我假设url.com只是一个“临时”域，您将其替换为原始域（如果出于隐私目的，这是可以理解的）