Aiohttp: broken interface normalize_path_middleware

[vir-mir]

Long story short

middleware normalize_path_middleware is not working properly

Expected behaviour

I appeal to /paymethods, there is an endpoint /paymethod

/paymethod -> 200
/paymethods -> 301 -> /paymethod

settings: normalize_path_middleware(remove_slash=True, append_slash=False)

if switch off
/paymethod -> 200
/paymethods -> 404

I think the problem is in the last if merge_slashes and remove_slash

            if merge_slashes:
                paths_to_check.append(re.sub('//+', '/', path))
            if append_slash and not request.path.endswith('/'):
                paths_to_check.append(path + '/')
            if remove_slash and request.path.endswith('/'):
                paths_to_check.append(path[:-1])
            if merge_slashes and append_slash:
                paths_to_check.append(
                    re.sub('//+', '/', path + '/'))
            if merge_slashes and remove_slash:
                merged_slashes = re.sub('//+', '/', path)
                paths_to_check.append(merged_slashes[:-1])

aiohttp == 3.5.4

Your environment

server

[aio-libs-bot]

GitMate.io thinks the contributor most likely able to help you is @asvetlov.

Possibly related issues are https://github.com/aio-libs/aiohttp/issues/3086 (add_static breaks normalize_path_middleware), https://github.com/aio-libs/aiohttp/issues/1178 (Websocket interface change break existing code), https://github.com/aio-libs/aiohttp/issues/1292 (broken interface for response.url), https://github.com/aio-libs/aiohttp/issues/1583 (normalize_path_middleware tests failures), and https://github.com/aio-libs/aiohttp/issues/445 (Is break in finally intentional?).

[bigbag]

if we change last condition
if merge_slashes and remove_slash:
on
if merge_slashes and remove_slash and path.endswith('/'):
all will be correct.

[kxepal]

async def test_bug_3669(aiohttp_client):
    async def paymethod(request):
        return web.Response(text="OK")

    async def paymethods(request):
        raise web.HTTPPermanentRedirect('/paymethod')

    app = web.Application()
    app.router.add_route('GET', '/paymethods', paymethods)
    app.router.add_route('GET', '/paymethod', paymethod)
    app.middlewares.append(
        web.normalize_path_middleware(append_slash=False, remove_slash=True)
    )

    client = await aiohttp_client(app, server_kwargs={'skip_url_asserts': True})

    resp = await client.get('/paymethods')
    assert resp.status == 200
    assert resp.url.path == '/paymethod'

@vir-mir, this test case passed. How is it different from your's case?

[vir-mir]

@kxepal Why did the final URL in the test change?

you send /paymethods
assert resp.url.path == '/paymethod'

[kxepal]

@vir-mir Because of redirect? Eventually, it causes new request to new location.

[vir-mir]

async def test_bug_3669(aiohttp_client):
    async def paymethod(request):
        return web.Response(text="OK")

    app = web.Application()
    app.router.add_route('GET', '/paymethod', paymethod)
    app.middlewares.append(
        web.normalize_path_middleware(append_slash=False, remove_slash=True)
    )

    client = await aiohttp_client(app, server_kwargs={'skip_url_asserts': True})

    resp = await client.get('/paymethods')
    assert resp.url.path == '/paymethod'

[kxepal]

So no redirect involved. That's make a sense. Thanks!

[kxepal]

Hm..autoclose failed. Fixed in master via https://github.com/aio-libs/aiohttp/commit/8260eda522ce0e999642fdcf842370efe48d13a7