httpx cannot process http/1.1 responses with headers and body delimited by \n\n or \n\n\n\n (but requests can)

Created on 3 Nov 2020  路  6Comments  路  Source: encode/httpx

Checklist

  • [x] The bug is reproducible against the latest release and/or master.
  • [x] There are no similar issues or pull requests to fix it yet.

Describe the bug

httpcore treats an http/1.1 response with Headers and Body delimited by \n\n as invalid

To reproduce


server code
Just run this socket-based server and run the client

```python
import datetime

import locale

import socket

locale.setlocale(locale.LC_TIME, "en_US")

full_response = """HTTP/1.1 200 OK\r
Date: {right_now}\r
Accept-Ranges: bytes\r
Connection: close\r
Content-type: text/plain

Some body\r\n"""

if __name__ == "__main__":
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

try:
sock.bind(("127.0.0.1", 8080))
sock.listen(5)

  print(f"Started server on http://127.0.0.1:8080/")

  (conn, addr) = sock.accept()

  try:
      data = conn.recv(4096)
      print(f"income> {data}")

      right_now = datetime.datetime.utcnow().strftime(
          "%a, %d %b %Y %H:%M:%S GMT"
      )

      full_response = full_response.format(right_now=right_now)
      conn.send(full_response.encode())
  finally:
      conn.close()

finally:
sock.detach()

```

# client code

with httpx.Client() as client:
    client.get("http://localhost:8080/")

Expected behavior

The client above should work as the following

s = requests.Session()
s.get("http://localhost:8080/)

or the curl snippet:

$ curl -v http://localhost:8080/

Actual behavior

httpx client fails with an error


stacktrace

```
/Users/krotose1/.pyenv/versions/httpx/bin/python /Users/krotose1/workspace/skv/httpx/wotk_with_bugs.py
Traceback (most recent call last):
File "/Users/krotose1/workspace/skv/httpx/httpx/_exceptions.py", line 326, in map_exceptions
yield
File "/Users/krotose1/workspace/skv/httpx/httpx/_client.py", line 861, in _send_single_request
(status_code, headers, stream, ext) = transport.request(
File "/Users/krotose1/workspace/skv/httpcore/httpcore/_sync/connection_pool.py", line 218, in request
response = connection.request(
File "/Users/krotose1/workspace/skv/httpcore/httpcore/_sync/connection.py", line 105, in request
return self.connection.request(method, url, headers, stream, ext)
File "/Users/krotose1/workspace/skv/httpcore/httpcore/_sync/http11.py", line 72, in request
) = self._receive_response(timeout)
File "/Users/krotose1/workspace/skv/httpcore/httpcore/_sync/http11.py", line 131, in _receive_response
event = self._receive_event(timeout)
File "/Users/krotose1/workspace/skv/httpcore/httpcore/_sync/http11.py", line 167, in _receive_event
event = self.h11_state.next_event()
File "/Users/krotose1/.pyenv/versions/3.8.5/lib/python3.8/contextlib.py", line 131, in __exit__
self.gen.throw(type, value, traceback)
File "/Users/krotose1/workspace/skv/httpcore/httpcore/_exceptions.py", line 12, in map_exceptions
raise to_exc(exc) from None
httpcore.RemoteProtocolError: peer unexpectedly closed connection

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/Users/krotose1/workspace/skv/httpx/wotk_with_bugs.py", line 11, in
resp = client.get("http://127.0.0.1:8080")
File "/Users/krotose1/workspace/skv/httpx/httpx/_client.py", line 907, in get
return self.request(
File "/Users/krotose1/workspace/skv/httpx/httpx/_client.py", line 733, in request
return self.send(
File "/Users/krotose1/workspace/skv/httpx/httpx/_client.py", line 767, in send
response = self._send_handling_auth(
File "/Users/krotose1/workspace/skv/httpx/httpx/_client.py", line 805, in _send_handling_auth
response = self._send_handling_redirects(
File "/Users/krotose1/workspace/skv/httpx/httpx/_client.py", line 837, in _send_handling_redirects
response = self._send_single_request(request, timeout)
File "/Users/krotose1/workspace/skv/httpx/httpx/_client.py", line 861, in _send_single_request
(status_code, headers, stream, ext) = transport.request(
File "/Users/krotose1/.pyenv/versions/3.8.5/lib/python3.8/contextlib.py", line 131, in __exit__
self.gen.throw(type, value, traceback)
File "/Users/krotose1/workspace/skv/httpx/httpx/_exceptions.py", line 343, in map_exceptions
raise mapped_exc(message, **kwargs) from exc # type: ignore
httpx.RemoteProtocolError: peer unexpectedly closed connection

Process finished with exit code 1

```

Debugging material

The pcap file from this comment contains an original request-response, used to investigate this ticket


Hexdump of original requests-response

```
00000000 47 45 54 20 2f 61 78 69 73 2d 63 67 69 2f 70 77 GET /axi s-cgi/pw
00000010 64 67 72 70 2e 63 67 69 3f 61 63 74 69 6f 6e 3d dgrp.cgi ?action=
00000020 67 65 74 20 48 54 54 50 2f 31 2e 31 0d 0a 48 6f get HTTP /1.1..Ho
00000030 73 74 3a 20 31 30 2e 30 2e 30 2e 32 35 34 0d 0a st: 10.0 .0.254..
00000040 55 73 65 72 2d 41 67 65 6e 74 3a 20 70 79 74 68 User-Age nt: pyth
00000050 6f 6e 2d 72 65 71 75 65 73 74 73 2f 32 2e 32 34 on-reque sts/2.24
00000060 2e 30 0d 0a 41 63 63 65 70 74 2d 45 6e 63 6f 64 .0..Acce pt-Encod
00000070 69 6e 67 3a 20 67 7a 69 70 2c 20 64 65 66 6c 61 ing: gzi p, defla
00000080 74 65 0d 0a 41 63 63 65 70 74 3a 20 2a 2f 2a 0d te..Acce pt: /.
00000090 0a 43 6f 6e 6e 65 63 74 69 6f 6e 3a 20 6b 65 65 .Connect ion: kee
000000A0 70 2d 61 6c 69 76 65 0d 0a 41 75 74 68 6f 72 69 p-alive. .Authori
000000B0 7a 61 74 69 6f 6e 3a 20 44 69 67 65 73 74 20 75 zation: Digest u
000000C0 73 65 72 6e 61 6d 65 3d 22 72 6f 6f 74 22 2c 20 sername= "root",
000000D0 72 65 61 6c 6d 3d 22 41 58 49 53 5f 30 30 34 30 realm="A XIS_0040
000000E0 38 43 41 43 43 35 41 34 22 2c 20 6e 6f 6e 63 65 8CACC5A4 ", nonce
000000F0 3d 22 30 30 30 30 31 32 38 66 59 37 39 37 38 39 ="000012 8fY79789
00000100 30 31 63 61 64 39 63 66 64 61 35 36 39 63 33 30 01cad9cf da569c30
00000110 33 66 63 31 63 35 30 39 64 37 33 66 37 30 64 64 3fc1c509 d73f70dd
00000120 34 22 2c 20 75 72 69 3d 22 2f 61 78 69 73 2d 63 4", uri= "/axis-c
00000130 67 69 2f 70 77 64 67 72 70 2e 63 67 69 3f 61 63 gi/pwdgr p.cgi?ac
00000140 74 69 6f 6e 3d 67 65 74 22 2c 20 72 65 73 70 6f tion=get ", respo
00000150 6e 73 65 3d 22 32 64 30 33 36 37 64 66 36 31 39 nse="2d0 367df619
00000160 30 66 34 39 32 63 32 64 31 64 39 39 36 64 31 65 0f492c2d 1d996d1e
00000170 65 32 30 30 64 22 2c 20 71 6f 70 3d 22 61 75 74 e200d", qop="aut
00000180 68 22 2c 20 6e 63 3d 30 30 30 30 30 30 30 31 2c h", nc=0 0000001,
00000190 20 63 6e 6f 6e 63 65 3d 22 38 61 64 32 37 34 39 cnonce= "8ad2749
000001A0 33 35 38 62 61 39 39 38 39 22 0d 0a 0d 0a 358ba998 9"....
00000000 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d HTTP/1.1 200 OK.
00000010 0a 58 2d 46 72 61 6d 65 2d 4f 70 74 69 6f 6e 73 .X-Frame -Options
00000020 3a 20 73 61 6d 65 6f 72 69 67 69 6e 0d 0a 44 61 : sameor igin..Da
00000030 74 65 3a 20 54 68 75 2c 20 32 39 20 4f 63 74 20 te: Thu, 29 Oct
00000040 32 30 32 30 20 31 31 3a 33 37 3a 34 30 20 47 4d 2020 11: 37:40 GM
00000050 54 0d 0a 41 63 63 65 70 74 2d 52 61 6e 67 65 73 T..Accep t-Ranges
00000060 3a 20 62 79 74 65 73 0d 0a 43 6f 6e 6e 65 63 74 : bytes. .Connect
00000070 69 6f 6e 3a 20 63 6c 6f 73 65 0d 0a 41 75 74 68 ion: clo se..Auth
00000080 65 6e 74 69 63 61 74 69 6f 6e 2d 49 6e 66 6f 3a enticati on-Info:
00000090 20 71 6f 70 3d 61 75 74 68 2c 20 72 73 70 61 75 qop=aut h, rspau
000000A0 74 68 3d 22 37 39 65 62 61 37 39 63 62 63 30 30 th="79eb a79cbc00
000000B0 64 38 61 35 32 37 62 31 63 31 39 64 34 62 34 33 d8a527b1 c19d4b43
000000C0 65 35 31 31 22 2c 20 63 6e 6f 6e 63 65 3d 22 38 e511", c nonce="8
000000D0 61 64 32 37 34 39 33 35 38 62 61 39 39 38 39 22 ad274935 8ba9989"
000000E0 2c 20 6e 63 3d 30 30 30 30 30 30 30 31 0d 0a 43 , nc=000 00001..C
000000F0 6f 6e 74 65 6e 74 2d 74 79 70 65 3a 20 74 65 78 ontent-t ype: tex
00000100 74 2f 70 6c 61 69 6e 0a 0a t/plain. .
00000109 72 6f 6f 74 3d 22 72 6f 6f 74 22 0a 62 69 6e 3d root="ro ot".bin=
00000119 22 72 6f 6f 74 2c 62 69 6e 2c 64 61 65 6d 6f 6e "root,bi n,daemon
00000129 22 0a 64 61 65 6d 6f 6e 3d 22 72 6f 6f 74 2c 62 ".daemon ="root,b
00000139 69 6e 2c 64 61 65 6d 6f 6e 22 0a 73 79 73 3d 22 in,daemo n".sys="
00000149 72 6f 6f 74 2c 62 69 6e 22 0a 74 74 79 3d 22 73 root,bin ".tty="s
00000159 63 68 65 64 75 6c 65 64 2c 69 6d 61 67 65 64 2c cheduled ,imaged,
00000169 70 74 7a 61 64 6d 22 0a 64 69 73 6b 3d 22 72 6f ptzadm". disk="ro
00000179 6f 74 2c 61 78 69 73 6e 73 2c 61 63 74 69 6f 6e ot,axisn s,action
00000189 65 6e 67 69 6e 65 64 2c 73 74 72 65 61 6d 65 72 engined, streamer
00000199 2c 69 6d 61 67 65 64 2c 6d 6f 74 69 6f 6e 2c 74 ,imaged, motion,t
000001A9 61 6d 70 65 72 69 6e 67 22 0a 6c 70 3d 22 64 61 ampering ".lp="da
000001B9 65 6d 6f 6e 22 0a 6d 65 6d 3d 22 22 0a 6b 6d 65 emon".me m="".kme
000001C9 6d 3d 22 22 0a 77 68 65 65 6c 3d 22 72 6f 6f 74 m="".whe el="root
000001D9 22 0a 6d 61 69 6c 3d 22 22 0a 61 75 64 69 6f 3d ".mail=" ".audio=
000001E9 22 73 74 72 65 61 6d 65 72 22 0a 73 68 61 64 6f "streame r".shado
000001F9 77 3d 22 73 74 72 65 61 6d 65 72 22 0a 75 74 6d w="strea mer".utm
00000209 70 3d 22 73 74 72 65 61 6d 65 72 22 0a 76 69 64 p="strea mer".vid
00000219 65 6f 3d 22 64 61 65 6d 6f 6e 2c 63 61 70 62 75 eo="daem on,capbu
00000229 66 64 2c 64 62 75 73 32 65 76 65 6e 74 32 2c 70 fd,dbus2 event2,p
00000239 74 6f 64 2c 73 74 72 65 61 6d 65 72 2c 69 6d 61 tod,stre amer,ima
00000249 67 65 64 2c 6d 6f 74 69 6f 6e 2c 74 61 6d 70 65 ged,moti on,tampe
00000259 72 69 6e 67 2c 70 74 7a 61 64 6d 22 0a 66 74 70 ring,ptz adm".ftp
00000269 3d 22 22 0a 76 69 65 77 65 72 3d 22 72 6f 6f 74 ="".view er="root
00000279 2c 61 6e 6f 6e 79 6d 6f 75 73 22 0a 6f 70 65 72 ,anonymo us".oper
00000289 61 74 6f 72 3d 22 72 6f 6f 74 22 0a 61 64 6d 69 ator="ro ot".admi
00000299 6e 3d 22 72 6f 6f 74 22 0a 73 79 73 74 65 6d 3d n="root" .system=
000002A9 22 72 6f 6f 74 22 0a 70 74 7a 3d 22 72 6f 6f 74 "root".p tz="root
000002B9 22 0a 61 6e 6f 6e 79 6d 6f 75 73 3d 22 61 6e 6f ".anonym ous="ano
000002C9 6e 79 6d 6f 75 73 22 0a 74 65 6d 70 6c 61 74 65 nymous". template
000002D9 3d 22 70 74 7a 61 64 6d 22 0a 63 72 79 70 74 6f ="ptzadm ".crypto
000002E9 3d 22 73 74 75 6e 6e 65 6c 2c 73 74 63 6c 69 65 ="stunne l,stclie
000002F9 6e 74 22 0a 67 70 69 6f 3d 22 69 6f 64 2c 6c 65 nt".gpio ="iod,le
00000309 64 2c 65 6e 76 69 72 6f 6e 6d 65 6e 74 2c 61 63 d,enviro nment,ac
00000319 74 69 6f 6e 65 6e 67 69 6e 65 64 2c 73 63 68 65 tionengi ned,sche
00000329 64 75 6c 65 64 2c 73 74 72 65 61 6d 65 72 2c 69 duled,st reamer,i
00000339 6d 61 67 65 64 2c 61 63 64 2c 6d 65 64 69 61 63 maged,ac d,mediac
00000349 6c 69 70 63 67 69 2c 70 74 7a 61 64 6d 22 0a 75 lipcgi,p tzadm".u
00000359 73 65 72 73 3d 22 22 0a 6d 65 73 73 61 67 65 62 sers="". messageb
00000369 75 73 3d 22 6d 65 73 73 61 67 65 62 75 73 22 0a us="mess agebus".
00000379 62 77 3d 22 62 77 22 0a 6d 6f 74 69 6f 6e 3d 22 bw="bw". motion="
00000389 6d 6f 74 69 6f 6e 22 0a 65 76 65 6e 74 3d 22 65 motion". event="e
00000399 76 65 6e 74 2c 61 63 74 69 6f 6e 65 6e 67 69 6e vent,act ionengin
000003A9 65 64 2c 74 72 69 67 67 65 72 64 2c 73 74 72 65 ed,trigg erd,stre
000003B9 61 6d 65 72 22 0a 73 74 72 65 61 6d 65 72 3d 22 amer".st reamer="
000003C9 73 74 72 65 61 6d 65 72 2c 73 74 6f 72 61 67 65 streamer ,storage
000003D9 2c 6d 65 64 69 61 63 6c 69 70 63 67 69 22 0a 61 ,mediacl ipcgi".a
000003E9 78 69 73 6e 73 3d 22 61 78 69 73 6e 73 22 0a 6d xisns="a xisns".m
000003F9 6c 64 3d 22 6d 6c 64 22 0a 69 6f 64 3d 22 69 6f ld="mld" .iod="io
00000409 64 22 0a 6c 61 6e 67 5f 68 61 6e 64 6c 65 72 3d d".lang_ handler=
00000419 22 6c 61 6e 67 5f 68 61 6e 64 6c 65 72 22 0a 70 "lang_ha ndler".p
00000429 74 7a 61 64 6d 3d 22 70 74 7a 61 64 6d 22 0a 67 tzadm="p tzadm".g
00000439 74 6f 75 72 64 3d 22 67 74 6f 75 72 64 22 0a 75 tourd="g tourd".u
00000449 70 6e 70 3d 22 75 70 6e 70 22 0a 72 65 6e 64 65 pnp="upn p".rende
00000459 7a 76 6f 75 73 3d 22 72 65 6e 64 65 7a 76 6f 75 zvous="r endezvou
00000469 73 22 0a 73 74 75 6e 6e 65 6c 3d 22 73 74 75 6e s".stunn el="stun
00000479 6e 65 6c 22 0a 69 6d 61 67 65 64 3d 22 69 6d 61 nel".ima ged="ima
00000489 67 65 64 22 0a 61 63 64 3d 22 61 63 64 22 0a 74 ged".acd ="acd".t
00000499 72 69 67 67 65 72 64 3d 22 74 72 69 67 67 65 72 riggerd= "trigger
000004A9 64 22 0a 74 61 6d 70 65 72 69 6e 67 3d 22 74 61 d".tampe ring="ta
000004B9 6d 70 65 72 69 6e 67 22 0a 73 74 6f 72 61 67 65 mpering" .storage
000004C9 3d 22 61 63 74 69 6f 6e 65 6e 67 69 6e 65 64 2c ="action engined,
000004D9 73 74 72 65 61 6d 65 72 2c 73 74 6f 72 61 67 65 streamer ,storage
000004E9 22 0a 73 65 73 73 69 6f 6e 63 67 69 3d 22 73 65 ".sessio ncgi="se
000004F9 73 73 69 6f 6e 63 67 69 22 0a 65 6e 76 69 72 6f ssioncgi ".enviro
00000509 6e 6d 65 6e 74 3d 22 65 6e 76 69 72 6f 6e 6d 65 nment="e nvironme
00000519 6e 74 22 0a 77 73 64 64 3d 22 77 73 64 64 22 0a nt".wsdd ="wsdd".
00000529 77 73 64 3d 22 77 73 64 2c 77 73 64 64 2c 73 63 wsd="wsd ,wsdd,sc
00000539 68 65 64 75 6c 65 64 22 0a 63 61 70 62 75 66 64 heduled" .capbufd
00000549 3d 22 63 61 70 62 75 66 64 22 0a 63 65 72 74 63 ="capbuf d".certc
00000559 67 69 3d 22 63 65 72 74 63 67 69 22 0a 6c 65 64 gi="cert cgi".led
00000569 3d 22 6c 65 64 22 0a 77 73 61 75 74 68 3d 22 77 ="led".w sauth="w
00000579 73 64 2c 73 63 68 65 64 75 6c 65 64 2c 73 74 72 sd,sched uled,str
00000589 65 61 6d 65 72 22 0a 73 74 63 6c 69 65 6e 74 3d eamer".s tclient=
00000599 22 73 74 63 6c 69 65 6e 74 22 0a 61 63 74 69 6f "stclien t".actio
000005A9 6e 65 6e 67 69 6e 65 64 3d 22 61 63 74 69 6f 6e nengined ="action
000005B9 65 6e 67 69 6e 65 64 22 0a 73 63 68 65 64 75 6c engined" .schedul
000005C9 65 64 3d 22 73 63 68 65 64 75 6c 65 64 22 0a 64 ed="sche duled".d
000005D9 62 75 73 32 65 76 65 6e 74 32 3d 22 64 62 75 73 bus2even t2="dbus
000005E9 32 65 76 65 6e 74 32 22 0a 63 65 72 74 3d 22 77 2event2" .cert="w
000005F9 73 64 2c 61 63 74 69 6f 6e 65 6e 67 69 6e 65 64 sd,actio nengined
00000609 22 0a 70 74 6f 64 3d 22 70 74 6f 64 22 0a 64 65 ".ptod=" ptod".de
00000619 70 64 3d 22 64 65 70 64 22 0a 76 69 72 74 75 61 pd="depd ".virtua
00000629 6c 69 6e 70 75 74 64 3d 22 76 69 72 74 75 61 6c linputd= "virtual
00000639 69 6e 70 75 74 64 22 0a 77 77 77 3d 22 77 77 77 inputd". www="www
00000649 22 0a 6e 6f 67 72 6f 75 70 3d 22 6d 65 64 69 61 ".nogrou p="media
00000659 63 6c 69 70 63 67 69 22 0a 64 69 67 75 73 65 72 clipcgi" .diguser
00000669 73 3d 22 72 6f 6f 74 22 0d 0a s="root" ..

```

As you can see, here is \n\n delimiter between headers and body
00000100 74 2f 70 6c 61 69 6e 0a 0a t/plain. .

Environment

  • OS: OS X
  • Python version: Python 3.8.5
  • HTTPX version: 0.16.1 (from 07229b8dff61d3e9d819244d4835f547cae67b4d)

Additional context

You can check https://github.com/Kane610/axis/issues/55 for more details and pcap files

external htt1.1 requests-compat

All 6 comments

I found the rule how http/1.1 should determine the length of response: link

As the server response contains Connection: close header, we probably could use this rule:

When a message-body is included with a message, the transfer-length of that body is determined by one of the following (in order of precedence):
...
5.By the server closing the connection. (Closing the connection cannot be used to indicate the end of a request body, since that would leave no possibility for the server to send back a response.)

New investigations

Trying to limit the "server code" snippet I found that the server response seems a bit malformed:


Server response hexdump

00000000 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d HTTP/1.1 200 OK. 00000010 0a 44 61 74 65 3a 20 54 75 65 2c 20 30 33 20 4e .Date: T ue, 03 N 00000020 6f 76 20 32 30 32 30 20 31 30 3a 31 30 3a 35 35 ov 2020 10:10:55 00000030 20 47 4d 54 0d 0a 41 63 63 65 70 74 2d 52 61 6e GMT..Ac cept-Ran 00000040 67 65 73 3a 20 62 79 74 65 73 0d 0a 43 6f 6e 6e ges: byt es..Conn 00000050 65 63 74 69 6f 6e 3a 20 63 6c 6f 73 65 0d 0a 41 ection: close..A 00000060 75 74 68 65 6e 74 69 63 61 74 69 6f 6e 2d 49 6e uthentic ation-In 00000070 66 6f 3a 20 71 6f 70 3d 61 75 74 68 2c 20 72 73 fo: qop= auth, rs 00000080 70 61 75 74 68 3d 22 34 31 38 65 30 33 66 65 39 pauth="4 18e03fe9 00000090 63 38 36 63 34 32 63 32 30 64 61 61 62 32 38 37 c86c42c2 0daab287 000000A0 61 32 35 31 62 38 33 22 2c 20 63 6e 6f 6e 63 65 a251b83" , cnonce 000000B0 3d 22 37 33 31 34 63 38 39 63 32 61 37 62 64 62 ="7314c8 9c2a7bdb 000000C0 37 36 22 2c 20 6e 63 3d 30 30 30 30 30 30 30 31 76", nc= 00000001 000000D0 0d 0a 43 6f 6e 74 65 6e 74 2d 74 79 70 65 3a 20 ..Conten t-type: 000000E0 74 65 78 74 2f 70 6c 61 69 6e 0a 0a text/pla in.. 000000EC 72 6f 6f 74 3d 22 72 6f 6f 74 22 0d 0a 62 69 6e root="ro ot"..bin 000000FC 3d 22 72 6f 6f 74 2c 62 69 6e 2c 64 61 65 6d 6f ="root,b in,daemo 0000010C 6e 22 0d 0a 64 61 65 6d 6f 6e 3d 22 72 6f 6f 74 n"..daem on="root

As you can see, headers and body are delimited using 0a 0a ~ \n\n instead of CRLF sequence

Having found that I tuned up the server snippet, and got that the following server caused the httpx exception:


bad_server.py

```python
import datetime

import locale

import socket

locale.setlocale(locale.LC_TIME, "en_US")

full_response = """HTTP/1.1 200 OK\r
Date: {right_now}\r
Accept-Ranges: bytes\r
Connection: close\r
Content-type: text/plain

Some body\r\n"""

if __name__ == "__main__":
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

try:
    sock.bind(("127.0.0.1", 8080))
    sock.listen(5)

    print(f"Started server on http://127.0.0.1:8080/")

    (conn, addr) = sock.accept()

    try:
        data = conn.recv(4096)
        print(f"income> {data}")

        right_now = datetime.datetime.utcnow().strftime(
            "%a, %d %b %Y %H:%M:%S GMT"
        )

        full_response = full_response.format(right_now=right_now)
        conn.send(full_response.encode())
    finally:
        conn.close()

finally:
    sock.detach()

```

but the following is accepted by httpx:


good_server.py

```python
import datetime

import locale

import socket

locale.setlocale(locale.LC_TIME, "en_US")

full_response = """HTTP/1.1 200 OK\r
Date: {right_now}\r
Accept-Ranges: bytes\r
Connection: close\r
Content-type: text/plain\r
\r
Some body\r\n"""

if __name__ == "__main__":
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

try:
    sock.bind(("127.0.0.1", 8080))
    sock.listen(5)

    print(f"Started server on http://127.0.0.1:8080/")

    (conn, addr) = sock.accept()

    try:
        data = conn.recv(4096)
        print(f"income> {data}")

        right_now = datetime.datetime.utcnow().strftime(
            "%a, %d %b %Y %H:%M:%S GMT"
        )

        full_response = full_response.format(right_now=right_now)
        conn.send(full_response.encode())
    finally:
        conn.close()

finally:
    sock.detach()

```

What the difference?

In bad_server.py headers and body are delimited by \n\n (\n\n\n\n raises exception too) whereas in good_server.py the \r\n\r\n is used to that.

rfc2616 says that

[request and response] ... consist of a start-line, zero or more header fields (also known as "headers"), an empty line (i.e., a line with nothing preceding the CRLF) indicating the end of the header fields, and possibly a message-body.

so looks like the server violates this rule. The only thing which is worrying me is curl and requests accepts \n\n as a delimiter

I found this issue in h11: https://github.com/python-hyper/h11/issues/7

That's what we might expect from h11 to make https://github.com/encode/httpx/issues/1378 fixed. On other hand, looks like the server violates rfc.

I'm a little confused here, since the description...

httpcore treat http/1.1 response without neither Content-Length nor chunked encoding as invalid, whereas requests and curl successfully read the response

Doesn't seem to match the title?...

httpx cannot process http/1.1 responses with headers and body delimited by \n\n or \n\n\n\n

Also it would be helpful to be a little more careful with the wording, in your comment here...

I found this bug in h11: https://github.com/python-hyper/h11/issues/7

We shouldn't rightly describe that as a "bug" in any sense - h11 is doing the correct thing, it's the non-spec-compliant servers that have the bug. It would be more correct to describe it as an "open issue" to determine if h11 ought to deal with their broken behaviour or not. 馃檪

You are right. Having added new comments and updated the title, I forget to update the original issue description

Also it would be helpful to be a little more careful with the wording, in your comment here...

yes. I didn't want to offend anyone, I have got confused with the words. Having said "a bug" I meant "an issue in the tracker"

Was this page helpful?
0 / 5 - 0 ratings