Akka-http: Implement server-side HTTP/2 support

[jrudolph]

This collect issues documenting the progress of the HTTP/2 PoC.

• [x] implement a first version of the handshake (mostly setup ALPN, #223)
• [ ] setup server-side API entry points for HTTP/2 servers (#246)
• [x] model constants and data types from the spec
• [x] model HTTP/2 framing protocol events (#245)
• [x] binary protocol parser / renderer (#247)
• [x] setup implementation architecture (#249)
• [x] model high-level HTTP/2 events (#248)
• [x] implement stream handling (stream state machine, flow control) (#250)
• [x] translation between HTTP/2 events and existing akka-http model (#252)
  
  
  
  • [x] implement / reuse twitter HPACK to read/write headers (#224)
  
  
  • [x] read HEADERS, CONTINUATION, and DATA to parse requests (#384)
  
  
  • [x] write HEADERS, CONTINUATION, and DATA to render responses
  
  
  • [x] convert headers into modeled headers (#755)
  
  
• [x] setup testing infrastructure to allow high-level declaration of expected behavior
• [ ] respect all SETTINGS
• [x] support HTTP/1.1 & HTTP/2 on the same port
• [x] setup automated conformancy testing (#251) (might be out-of-scope but helpful for debugging, measuring practical progress)

Out-of-scope for this initial effort:

• client side
• optimized / prioritized stream output handling (which / how many data to put on the TCP connection when multiple streams have outgoing data)
• h2c
• server-push
• performance optimizations

[jrudolph]

Our first sprint for implementing HTTP/2 is done. Unfortunately, we had much less time working on it than we hoped for various reasons. Luckily, we still managed to build a (somewhat) working prototype that we (and everyone else interested) can now built upon.

Here's an overview of the current state:

What works:

• simple request / response cycle works with multiple concurrent streams (tested on chromium)
• overall architecture / data structures / entry point setup
• frame parsing / rendering
• a simple form of (de)multiplexing request/response streams
• support an API very similar to the existing one (only bindAndHandleAsync so far)
• support parsing requests into the existing HTTP model (request entities not yet supported)
• support rendering responses from the existing HTTP model

Mandatory features that are missing:

• Proper management of all aspects of the stream state machine
• Respect all limits and settings like flow control
• Receive request entities (e.g. POST requests)
• Switch between HTTP/2 + HTTP/1.1 depending on client needs
• Documentation
• API should be mostly consistent / the same as existing HTTP (currently only bindAndHandleAsync is supported)
• TLS setup will need some extras (either jetty-alpn or netty-openssl)

Optional features missing

• Stream priorities
• PUSH_PROMISE support
• Clever logic to manage outgoing data windows, and optimisations specific to HTTP/2 behaviour
  Better cipher performance for required ciphers in HTTP/2 (see http://netty.io/wiki/requirements-for-4.x.html) by either JDK getting a better implementation or relying on openssl (we’ve got a PoC to piggy-back on netty’s openssl support)
• Automatic testing against HTTP/2 conformance suite
• Client-side implementation

The current implementation can be found in https://github.com/akka/akka-http/tree/wip-http2. A basic example can be found in https://github.com/akka/akka-http/blob/wip-http2/akka-http2-support/src/test/scala/akka/http/scaladsl/Http2ServerTest.scala.

We plan to follow up on this work quite soon to get it into a state where we can confidently make a first release. We strive to have all the infrastructure in place by then which would simplify it for everyone interested to test and contribute to further development. Our current focus is to release Akka HTTP as a stable module in the coming days, starting it off with an RC1 very soon. The HTTP/2 work is not a requirement for the first stable release from the new repository - we’ll get back to it at a later point in time, and – in the meantime – invite you to help us out by contributing the missing bits and pieces.

[jrudolph]

Initial version working. Remaining work in smaller tickets. Closing...