Aiohttp: Possibility to disable Server header

[sp-1234]

Long story short

By default, aiohttp server adds Server header. It should be possible to disable this, for security reasons.

Expected behaviour

No Server header in response

Actual behaviour

Server sends these headers:

Content-Type: text/plain; charset=utf-8
Content-Length: 2
Date: Tue, 06 Jun 2017 13:52:03 GMT
Server: Python/3.6 aiohttp/2.1.0

Steps to reproduce

import asyncio
from aiohttp import web


async def http_handler(request):
    return web.Response(text="OK", headers={})


def main():
    loop = asyncio.get_event_loop()
    loop.run_until_complete(loop.create_server(web.Server(http_handler), "127.0.0.1", 8080))
    loop.run_forever()


if __name__ == "__main__":
    main()

Your environment

macOS 10.12.5
Python 3.6.1

$ pip3 freeze
aiofiles==0.3.0
aiohttp==2.1.0
async-timeout==1.2.1
asyncio==3.4.3
chardet==3.0.3
Cython==0.25.2
defusedxml==0.5.0
google-api-python-client==1.6.2
httplib2==0.10.3
jira==1.0.10
llvmlite==0.16.0
magicmemoryview==0.1.5
multidict==2.1.6
numba==0.31.0
numexpr==2.6.2
numpy==1.12.0
oauth2client==4.1.0
oauthlib==2.0.2
pbr==3.0.1
pyasn1==0.2.3
pyasn1-modules==0.0.8
requests==2.13.0
requests-oauthlib==0.8.0
requests-toolbelt==0.7.1
rsa==3.4.2
six==1.10.0
tornado==4.4.2
uritemplate==3.0.0
websockets==3.3
yarl==0.10.2

[asvetlov]

You cannot disable SERVER header but could pass any value to it:

async def http_handler(request):
    return web.Response(text="OK", headers={'Server': 'noname'})

Or you could drop Server header at all on reverse proxy.

[sp-1234]

So, aiohttp is designed to be used _only_ behind reverse proxy, right?
If that's true then I understand it.

[kxepal]

I think that would be a classic security through obscurity. You should keep you software up to date and dont' let your application have a security holes.

[asvetlov]

@sp-1234 requirements for aiohttp are very close to any python web server.
It might be used without any frontend for simple cases. But in practice you'll probably put it behind reverse proxy for very many reasons.

[sp-1234]

@kxepal nobody said that I won't be updating software. But from Defense in Depth™ principle, it's not good to make attacker's job easier if somehow updated software becomes _not enough_ for a moment.

@asvetlov OK, it sounds plausible 😎

[Qix-]

The conclusions made here are nonsense; as a developer I should have the ability to control something as subjective as the Server response header, including the ability to shut it off entirely.

This has nothing to do with security - this is seemingly the maintainers trying to plug their own software.

Would you accept a PR to allow disabling it?

[asvetlov]

Would you accept a PR to allow disabling it?

Disable what?

[Qix-]

The Server response header.

[asvetlov]

Does on_response_prepare signal satisfy your needs?

[Qix-]

@asvetlov No, it appears that Server is added after that signal. Therefore, if "Server" in res.headers: del res.headers["Server"] does not work, and setting it to None results in a concatenation error later on in the aiohttp response pipeline.

[asvetlov]

Aha, I see.
If you insist that Server should be deleted please make a PR that splits StreamRequest._start in two methods: one makes all header preparations and called before the signal raising, another one is bare writer.write_headers() call which is executed after on_response_prepare().

[Qix-]

@asvetlov Alternatively, could we just add a removedefault() method from CIMultiDict in the multidict package? Then the on_response_prepare handler can just do res.headers.removedefault("Server") which is just as descriptive.

[asvetlov]

removedefault is... unusual.
What the method should do?

[asvetlov]

Reopened since we have a discussion here.
P.S. I hate necromancy in general and reviving already closed issues after years of been closed.

[Qix-]

https://github.com/aio-libs/aiohttp/blob/master/aiohttp/web_response.py#L384

This is where it's being set. multidict uses a default if no keys have been set in the dictionary, which gives the effect of a normal dictionary in the case no multi-value-single-key entries have been submitted to the dictionary.

I would imagine there hasn't been a usecase for removing a default until now, but IMO it's a very solid usecase.

P.S. I hate necromancy in general and reviving already closed issues after years of been closed.

Yes, same, but a new issue would have duplicated most of the conversation here. I necro'd since there's already the context and I disagreed with the outcome. Further, on Github, this is less of an issue organizationally than on traditional, archaic boards.

This was the most apt place to raise dissent in my opinion, hence the necro. I hope it didn't cause too much of an inconvenience.

[asvetlov]

multidict.setdefault(k, v) has the following pseudo-code:

if k not in multidict:
    multidict.add(k, v)

What is pseudo-code for multidict.removedefault(...)?

[Qix-]

Ah I understand now, what I wrote earlier was nonsense.

Submitting a PR with your proposed fixes @asvetlov :)