Aiohttp: Request.json() and request.text() return None

[rubenvdham]

Long story short

I cannot seem to get the text or json response from a response object.

Expected behaviour

I would be able to extract the json via response.json()

Actual behaviour

response.json() gives a None type
response.text() doesn't give anything either

Steps to reproduce

Run this:

wordpress_url = "https://kominherstel.nl/wp-json/wp/V2/pages/"



async def fetch(session, url):
    async with session.get(url) as response:
        return response

async def main(wordpress_url):
    async with ClientSession() as session:
        response = await fetch(session, wordpress_url)
        print(await response.json())

loop = asyncio.get_event_loop()

loop.run_until_complete(main(wordpress_url))

In a browser "https://kominherstel.nl/wp-json/wp/V2/pages/" will load.

Your environment

aiohttp==2.3.10
OS: Ubuntu 16.04 LTS

[asvetlov]

You cannot fetch response's body after exiting from async with session.get(...) context manager.

[rubenvdham]

Thanks for the advice!
I tried the following instead:

wordpress_url = "https://kominherstel.nl/wp-json/wp/V2/pages/"


async def main(url):
    async with ClientSession() as session:
        async with session.get(url) as response:
            print(await response.json())
            print(await response.text())

            print(response.text)
            print(response.json)

loop = asyncio.get_event_loop()

loop.run_until_complete(main(wordpress_url))

returns the following:

<bound method ClientResponse.text of <ClientResponse(https://kominherstel.nl/wp-json/wp/V2/pages/) [200 OK]>
<CIMultiDictProxy('Date': 'Sat, 07 Apr 2018 20:41:13 GMT', 'Server': 'Apache', 'Vary': 'Cookie', 'X-Robots-Tag': 'noindex', 'Link': '<https://kominherstel.nl/wp-json/>; rel="https://api.w.org/"', 'X-Content-Type-Options': 'nosniff', 'Access-Control-Expose-Headers': 'X-WP-Total, X-WP-TotalPages', 'Access-Control-Allow-Headers': 'Authorization, Content-Type', 'Expires': 'Thu, 19 Nov 1981 08:52:00 GMT', 'Cache-Control': 'no-store, no-cache, must-revalidate', 'Pragma': 'no-cache', 'X-WP-Total': '8', 'X-WP-TotalPages': '1', 'Allow': 'GET', 'Set-Cookie': 'PHPSESSID=9u9an4n6tqt7hkp431ukf5qkk5; path=/', 'Upgrade': 'h2c', 'Connection': 'Upgrade', 'Transfer-Encoding': 'chunked', 'Content-Type': 'application/json; charset=UTF-8')>
>
<bound method ClientResponse.json of <ClientResponse(https://kominherstel.nl/wp-json/wp/V2/pages/) [200 OK]>
<CIMultiDictProxy('Date': 'Sat, 07 Apr 2018 20:41:13 GMT', 'Server': 'Apache', 'Vary': 'Cookie', 'X-Robots-Tag': 'noindex', 'Link': '<https://kominherstel.nl/wp-json/>; rel="https://api.w.org/"', 'X-Content-Type-Options': 'nosniff', 'Access-Control-Expose-Headers': 'X-WP-Total, X-WP-TotalPages', 'Access-Control-Allow-Headers': 'Authorization, Content-Type', 'Expires': 'Thu, 19 Nov 1981 08:52:00 GMT', 'Cache-Control': 'no-store, no-cache, must-revalidate', 'Pragma': 'no-cache', 'X-WP-Total': '8', 'X-WP-TotalPages': '1', 'Allow': 'GET', 'Set-Cookie': 'PHPSESSID=9u9an4n6tqt7hkp431ukf5qkk5; path=/', 'Upgrade': 'h2c', 'Connection': 'Upgrade', 'Transfer-Encoding': 'chunked', 'Content-Type': 'application/json; charset=UTF-8')>
>

The first 2 print(await response.json()) and print(await response.text()) should work. From the documentation (https://docs.aiohttp.org/en/stable/client_quickstart.html#json-response-content) it seems the json() method should raise an exception when it fails decoding. However, no exception is raised at all, and there's clearly json at the fetched URL.

[asvetlov]

Sorry, your code doesn't fit attached output.
I've totally missed what behavior do you expect.

[rubenvdham]

Finally, I found the problem when running the code on windows instead of Ubuntu. On my windows machine, it ran as I expected. The problem: I was still running python 3.5.2 and aiohttp version 2.3.10 on my Ubuntu machine. My bad, I should have checked the version number beforehand:S

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a [new issue] for related bugs.
If you feel like there's important points made in this discussion, please include those exceprts into that [new issue].