Grpc-gateway: support HttpBody in streaming responses

[srenatus]

In http.proto, it says

// This message can be used both in streaming and non-streaming API methods in
// the request as well as the response.

What I would like to achive is to have an rpc MyMethod(Req) (stream HttpBody) return lines of _bare content_, like, CSV, or JSON streaming.

What we currently get is:

# generate some data
$ curl http://127.0.0.1:8080/v1/example/a_bit_of_everything/bulk -d '{}'
{}
$ curl http://127.0.0.1:8080/v1/example/a_bit_of_everything/bulk -d '{}'
{}
$ curl http://127.0.0.1:8080/v1/example/a_bit_of_everything/bulk -d '{}'
{}
$ curl -v http://127.0.0.1:8080/v1/example/a_bit_of_everything/httpbody
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET /v1/example/a_bit_of_everything/httpbody HTTP/1.1
> Host: 127.0.0.1:8080
> User-Agent: curl/7.62.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Grpc-Metadata-Content-Type: application/grpc
< Grpc-Metadata-Count: 3
< Date: Thu, 20 Jun 2019 11:09:10 GMT
< Transfer-Encoding: chunked
<
{"result":{"content_type":"application/csv-stream","data":"JSFkKHN0cmluZz05ZDg0MTA3MzljMzdhMWIwZGIxNTlkNzY3NzUxOTA1YWQ2M2JmNDBjNDRhMTc3MGMpLDA="}}
{"result":{"content_type":"application/csv-stream","data":"JSFkKHN0cmluZz05ZTg0MTA3MzljMzdhMWIwZGIxNTlkNzY3NzUxOTA1YWQ2M2JmNDBjNDRhMTc3MGMpLDE="}}
{"result":{"content_type":"application/csv-stream","data":"JSFkKHN0cmluZz05Zjg0MTA3MzljMzdhMWIwZGIxNTlkNzY3NzUxOTA1YWQ2M2JmNDBjNDRhMTc3MGMpLDI="}}
* Connection #0 to host 127.0.0.1 left intact

(This was created using that branch)

What would be nice was to get a response like

< Content-Type: application/csv-stream
< Grpc-Metadata-Content-Type: application/grpc
< Grpc-Metadata-Count: 3
UUID1,0
UUID2,1
UUID3,2

i.e., the lines of content, unencoded, and not wrapped in {"result": ... }

[johanbrandhorst]

@wimspaargaren

I think this should be possible with a type assertion in ForwardResponseStream like we're already doing for unary messages.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[srenatus]

Still seems like a valuable thing to support. That said, I have not immediate need for it and won't be implementing it. So I guess closing is fine with me 😃

[npuichigo]

@theRealWardo Sorry to bother you. Has this feature already been supported with streaming httpbody?

[johanbrandhorst]

I don't think this is supported.

[theRealWardo]

so my fork does do this - https://github.com/theRealWardo/grpc-gateway/compare/07b39baf50b9eba1a2328e64d28d2fe78bf24fb0...theRealWardo:master

disclaimer: its been a while since I tried to rebase that code on top of master so... your mileage may vary with all of this.

the custom marshaller I wrote HttpBodyMarshaler detects the HttpBody message and handles it appropriately. in a way this is "supported" because I think there's a way to have a custom marshaller in your code that can be set up using WithMarshalerOption in the runtime package (you probably want to override the MIMEWildcard mime type if you want this for all requests/responses).

[johanbrandhorst]

Ah cool. Maybe we could add an example to the docs?

[npuichigo]

@theRealWardo Do you still work on merging your code to master branch?

[npuichigo]

@johanbrandhorst When will an interface param be added to the ContentType() function of Marshal interface so as to fit HttpBody message better. (set the content type with proper grpc response)

[johanbrandhorst]

I think the existing httpbody marshaller supports this? https://github.com/grpc-ecosystem/grpc-gateway/blob/master/runtime/marshal_httpbodyproto.go#L29