Grpc-gateway: protoc-gen-swagger: support maps in query parameters

[george-oakling]

Hi guys,
having following definition:

message Test {
    map<string, string> filters = 1;
}

service TestService {
    rpc TestRpc(Test) returns (Test) {
        option (google.api.http) = {
            get: "/test"
        };
    }
}

Which goes through swagger generator and ends up without the map being mentioend as parameters in request:

"/test": {
      "get": {
        "operationId": "TestRpc",
        "responses": {
          "200": {
            "description": "A successful response.",
            "schema": {
              "$ref": "#/definitions/offersTest"
            }
          }
        },
        "tags": [
          "TestService"
        ]
      }
    }

I thought that I can add map params to the query... but I dont think so now and I cant tell if it's a bug... KR Jirka

[johanbrandhorst]

Thanks for the bug report! A few questions:

1. What does the generated gateway look like? Is it trying to parse the map from the query? If so, that would be a large indicator that we should be supporting this and that it's a bug in the swagger generator.
2. Does it _work_ even if you disregard the generated swagger docs?

[george-oakling]

test.proto:

message TestRequest {
    map<string, string> filters = 1;
}

message TestResponse {
}

service TestSvc {
    rpc TestRpc(TestRequest) returns(TestResponse) {
        option (google.api.http) = {
            get: "/offers"
        };
    }
}

test.pb.gw.go

var (
    filter_TestSvc_TestRpc_0 = &utilities.DoubleArray{Encoding: map[string]int{}, Base: []int(nil), Check: []int(nil)}
)

func request_TestSvc_TestRpc_0(ctx context.Context, marshaler runtime.Marshaler, client TestSvcClient, req *http.Request, pathParams map[string]string) (proto.Message, runtime.ServerMetadata, error) {
    var protoReq TestRequest
    var metadata runtime.ServerMetadata

    if err := req.ParseForm(); err != nil {
        return nil, metadata, status.Errorf(codes.InvalidArgument, "%v", err)
    }
    if err := runtime.PopulateQueryParameters(&protoReq, req.Form, filter_TestSvc_TestRpc_0); err != nil {
        return nil, metadata, status.Errorf(codes.InvalidArgument, "%v", err)
    }

    msg, err := client.TestRpc(ctx, &protoReq, grpc.Header(&metadata.HeaderMD), grpc.Trailer(&metadata.TrailerMD))
    return msg, metadata, err

}
...
func RegisterTestSvcHandlerClient(ctx context.Context, mux *runtime.ServeMux, client TestSvcClient) error {

    mux.Handle("GET", pattern_TestSvc_TestRpc_0, func(w http.ResponseWriter, req *http.Request, pathParams map[string]string) {
        ctx, cancel := context.WithCancel(req.Context())
        defer cancel()
        inboundMarshaler, outboundMarshaler := runtime.MarshalerForRequest(mux, req)
        rctx, err := runtime.AnnotateContext(ctx, mux, req)
        if err != nil {
            runtime.HTTPError(ctx, mux, outboundMarshaler, w, req, err)
            return
        }
        resp, md, err := request_TestSvc_TestRpc_0(rctx, inboundMarshaler, client, req, pathParams)
        ctx = runtime.NewServerMetadataContext(ctx, md)
        if err != nil {
            runtime.HTTPError(ctx, mux, outboundMarshaler, w, req, err)
            return
        }

        forward_TestSvc_TestRpc_0(ctx, mux, outboundMarshaler, w, req, resp, mux.GetForwardResponseOptions()...)

    })

    return nil
}

test.swagger

      "get": {
        "operationId": "TestRpc",
        "responses": {
          "200": {
            "description": "A successful response.",
            "schema": {
              "$ref": "#/definitions/offersTestResponse"
            }
          }
        },
        "tags": [
          "TestSvc"
        ]
      }

[johanbrandhorst]

I can't tell if that is or isn't parsing the query for the map, could you test running it and confirm?

[george-oakling]

Yep, working on it :)

[george-oakling]

Request to:
/test?filters[jirka]=test&filters[wow]=test2

And there I have log.Println, which prints:
filters:<key:"jirka" value:"test" > filters:<key:"wow" value:"test2" >

So yes, its getting data, but the protoc-swagger-gen ommit the definitions from request...

[george-oakling]

Also the code in the GRPC server looks like this to make everything clear:

func(s *Server) TestRpc(ctx context.Context, req *offers.TestRequest) (*offers.TestResponse, error){
    log.Println(req)
    return &offers.TestResponse{}, nil
}

[johanbrandhorst]

Thanks, that narrows down the scope of this bug. Would you be interested in trying to submit a fix yourself? I would recommend starting to look at https://github.com/grpc-ecosystem/grpc-gateway/blob/master/protoc-gen-swagger/genswagger/template.go#L95 and maybe add some log messages and rebuild the generator. It would be very welcome!

[george-oakling]

I'll try, but not before next week... I "hacked" this by using POST request with full body request which works perfectly, but definitely will look at the source asap.

[sum2000]

Hi, I can take a look at this bug since we have a requirement to use this kind of pattern. Thanks

[s4nji]

Note that it is not possible to have objects or maps in query parameter according to OpenAPI-v2 spec (which is what is available at the time of writing, protoc-gen-openapiv2):

Since the parameter is not located at the request body, it is limited to simple types (that is, not an object). The value MUST be one of "string", "number", "integer", "boolean", "array" or "file".

This issue can probably be merged into https://github.com/grpc-ecosystem/grpc-gateway/issues/441 which allows object type query parameters.