Grpc-gateway: YAML support for Swagger annotations

[dmgarland]

Hi there, first of all great project thanks for all your hard work.

I'm experimenting with the swagger doc generation and have got things to work fine in the proto file. I am interested in keeping all of the swagger / rest stuff out of the proto file and use the associated YAML file that can be used to configure endpoints, as per
https://cloud.google.com/endpoints/docs/grpc/grpc-service-config

Is it possible to add the protogen-swagger annotation messages in the YAML file instead? I'd like to write something like:

# API Reference: https://cloud.google.com/endpoints/docs/grpc-service-config/reference/rpc/google.api#google.api.Service

type: google.api.Service
config_version: 3

name: api.vega.trading# API Reference: https://cloud.google.com/endpoints/docs/grpc-service-config/reference/rpc/google.api#google.api.Service

type: google.api.Service
config_version: 3
info:
  title: "A Bit of Everything";
  version: "1.0";
  contact: 
    name: "gRPC-Gateway project";
    url: "https://github.com/grpc-ecosystem/grpc-gateway";
    email: "[email protected]";

[achew22]

I think this is reasonably well documented at https://grpc-ecosystem.github.io/grpc-gateway/docs/grpcapiconfiguration.html. Can you check that out and report back what additional information you need or how we could make that easier?

[dmgarland]

For instance, how would one accomplish the following in YAML:

option (grpc.gateway.protoc_gen_swagger.options.openapiv2_swagger) = {
  info: {
    title: "Example API";
    description: "A description";
    version: "1.0";
    contact: {
      url: "https://example.org";
      email: "[email protected]";
    };
  };
}

[achew22]

What have you tried already? What documentation have you looked at?

[ivucica]

Support for YAML API definition is exciting, but somewhat basic: https://github.com/grpc-ecosystem/grpc-gateway/pull/521

It doesn't process most of the fields available in google.api.Service (which is very large: see the documentation, see the proto file). As far as I can tell, it only supports google.api.Http-type http key.

I believe you can only split out your routing proto options into YAML. I don't believe you can currently define API title, description, version, etc. using the YAML file.

[johanbrandhorst]

Sounds like there's work that could be done here.

[gensmusic]

need help too.

[alexisvisco]

So there is no documentation at all ? Because this:

 type: google.api.Service
 config_version: 3

 http:
   rules:
   - selector: example.YourService.Echo
     post: /v1/example/echo
     body: "*"

It's not a documentation ... It's a snippet.

This link can be useful: https://cloud.google.com/endpoints/docs/grpc/grpc-service-config#annotations_http_rules_only but does not answer to all my questions.

https://github.com/grpc-ecosystem/grpc-gateway/blob/master/examples/proto/examplepb/a_bit_of_everything.proto#L15

This annotations, with responses, security... can we do that ?

[johanbrandhorst]

Swagger annotations are currently not supported in the yaml files, only inlined in the proto files. We would love your help to improve the documentation, how can I help you contribute updates?

[jasonewang]

I'm interested in taking on adding YAML support for Swagger/OpenAPIv2 options.

I took a look at protoc-gen-openapiv2/internal/template.go and it looks like I'd just need to extend the extract...From...Descriptor functions to look up YAML added options. Does this sound like the right track?

To go into more detail, my plan is to:

1. Come up with a schema to represent the OpenAPI options

For the sake of example, it could look like

syntax = "proto3";

package grpc.gateway.protoc_gen_openapiv2.options;

import "protoc-gen-openapiv2/options/openapiv2.proto";

message FileOption {
  string selector = 1; // ex: file.proto
  Swagger option = 2;
}

message MethodOption {
  string selector = 1; // ex: Service.Method
  Operation option = 2;
}

message MessageOption {
  string selector = 1; // ex: Message
  Schema option = 2;
}

message ServiceOption {
  string selector = 1; // ex: Service
  Tag option = 2;
}

message FieldOption {
  string selector = 1; // ex: Message.Field
  JSONSchema option = 2;
}

message Configuration {
  repeated FileOption file = 1;
  repeated MethodOption method = 2;
  repeated MessageOption message = 3;
  repeated ServiceOption service = 4;
  repeated FieldOption field = 5;
}

1. Add a new descriptor.Registry option to parse the YAML and add the parsed options to the registry.
2. Extend the option extracting functions to look up options in the registry.
   
   
   
   • It would probably be easier to change the signature of the extract functions to take grpc-gateway's descriptor types because those have methods for getting the fully qualified names of types and fields.
   
   

Finally, where is the grpc.gateway.protoc_gen_openapiv2.options.Tag option parsed? I see it used in a_bit_of_everything.proto but can't find it in the generator code.

Any guidance or feedback would be greatly appreciated.

[johanbrandhorst]

Hi @jasonewang, that's great! I think you need to start here: https://github.com/grpc-ecosystem/grpc-gateway/blob/v2/internal/descriptor/grpc_api_configuration.go. The annotations used are sourced from https://github.com/grpc-ecosystem/grpc-gateway/blob/v2/internal/descriptor/apiconfig/apiconfig.proto. If you want to add more of them, you'd probably have to start here. As you can see, we're actually turning the YAML into JSON before protojson.Unmarshaling it to the proto type. It's pretty self contained to this file so it shouldn't be too bad.

You may even be able to add a test first and do a TDD style iteration (https://github.com/grpc-ecosystem/grpc-gateway/blob/v2/internal/descriptor/grpc_api_configuration_test.go).

[ivucica]

In #1665 this adds a _new_ file rather than incompatibly forking the google.api.Service. I like that approach.