Protobuf: Python ParseFromString not parsing custom options

[navsaini]

I am trying to write a python plugin, but ParseFromString behavior is not letting me get custom options that are written in .proto files. I am doing the following to read the .proto file:

data = sys.stdin.read()

request = plugin.CodeGeneratorRequest()
request.ParseFromString(data)

However, if I print the request, this is the output:

message_type {
  name: "Hello"
  field {
    name: "greeting"
    number: 1
    label: LABEL_REQUIRED
    type: TYPE_BOOL
    json_name: "greeting"
  }
  field {
    name: "name"
    number: 2
    label: LABEL_REQUIRED
    type: TYPE_STRING
    json_name: "name"
  }
  field {
    name: "number"
    number: 3
    label: LABEL_OPTIONAL
    type: TYPE_INT32
    json_name: "number"
  }
  options {
  }
}

Even though I have defined an option and used it in my message as follows:

import "google/protobuf/descriptor.proto";

extend google.protobuf.MessageOptions {
    string my_option = 51234;
}
message Hello {
    bool greeting = 1;
    string name = 2;
    int32 number = 3;

    option (my_option) = "telephone";
}

I noticed that the Go equivalent of ParseFromString, proto.Unmarshal, works correctly and parses the options, but the Python ParseFromString does not.

[xfxyjwf]

In your python code, you need to import the .pb2.py file for the proto file that defines the custom option.

[navsaini]

Does that imply that I need the protoc-generated pb2.py file to get the options? I'm trying to write a plugin that generates code in another language (not Python), so having to generate those pb2.py files seems like it should be unnecessary since I (or the client) won't be using it for anything except accessing the custom options. @xfxyjwf

[xfxyjwf]

Yes, you will need the protoc generated pb2.py file. Since your plugin needs to access the options, presumably these options are part of your plugin rather than supplied by your clients. Is it not the case?

If these custom options are actually defined by the users of your plugin, accessing them will be very tricky. I'm not sure if python has the ability to do that, but in C++ you have to build a DescriptorPool on top of the FileDescriptorSet and use DyanamicMessage to reparse the message to get access to the custom options.

[navsaini]

Say I have two files, one defined by me called option.proto that has a custom-defined option. The other file, called example.proto, will be defined by the client and use the custom option defined by me in option.proto. Do I need to then have both option_pb2.py AND example_pb2.py to get the custom option? If so, I know you can access the options via the Extensions API defined here:
(https://developers.google.com/protocol-buffers/docs/proto#customoptions) and using the following syntax in python
value = my_proto_file_pb2.MyMessage.DESCRIPTOR.GetOptions().Extensions[my_proto_file_pb2.my_option]
Is there a way to do this WITHOUT generating example_pb2.py?
Thank you so much for your help! @xfxyjwf

[xfxyjwf]

You only need option_pb2.py in order to access the custom options.

[navsaini]

That worked! Thanks so much for your help! Didn't realize that simply not importing that file would make the ParseFromString method not recognize those options.

[navsaini]

You can find more information about it here -
https://developers.google.com/protocol-buffers/docs/pythontutorial#compiling-your-protocol-buffers

Basically you have to define your .proto file with the definition of your
datatype and then use the protobuf compiler to generate that file. The
options and arguments for the command are in the provided link.

On Tue, Jun 19, 2018 at 6:32 AM, Shiv Kumar [e-mail]
wrote:

It might be a silly question but how will we generate that option_pb2.py
file?

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/google/protobuf/issues/3321#issuecomment-398399658,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AMTu4iJFs88_ObizIg_dk2klrDzQ7uPuks5t-P18gaJpZM4OP3-e
.