Serving: Where do the .proto and *_pb2.py files come from?

[eriophora]

The documentation for TF Serving mentions the *.proto files, but does not do much in the way of explaining what they are or how they may be modified. For instance, I'm trying to do a regression task. I tried to modify the *.proto file to do this but it was not clear how.

Further, the creation of *_pb2.py files is never mentioned, but is apparently a necessary step if you wish to change anything. I did this by:

protoc --proto_path=serving/tensorflow_serving --python_out=serving/tensorflow_serving/custom_net serving/tensorflow_serving/custom_net/cutom_net_inference.proto

but the custom_net_inference_pb2.py file it output did not have the same functions as inception_inference_pb2.py; namely, it stopped after protoc_insertion_point.

Furthermore, the line:

service InceptionService { // Classifies an JPEG image into classes. rpc Classify(InceptionRequest) returns (InceptionResponse); }

is Classify a function? Namely the function in inception_inference.cc? Is it necessary to modify these names if the net performs a regression? My understanding is that the signatures themselves should change but the function names are irrelevant, so custom_net_inference.cc has much the same function names but just uses tensorflow::serving::RegressionSignature signature;

[viksit]

@eriophora You need to install grpc. See instructions there. Once done modifying proto files, invoke it this way.

protoc -I ./ --python_out=. --grpc_out=. --plugin=protoc-gen-grpc=`which grpc_python_plugin` ./custom_net_inference.proto

Remember that you need to install the C++ extension for grpc/protobuf as well as the grpcio module for python via pip. The latter does NOT contain the grpc python plugin, which you'll need.

HTH.

[eriophora]

Ah, I see that my call to protoc was incorrect!

Thank you for this, I will install grpc now and see if it works.

Is this protoc call documented somewhere on the TF-Serving website that I've missed? Or is it assumed they'd be sufficiently familiar with it?

[viksit]

At the moment, it's not as adequately documented as it should be :)

[fangweili]

Thank you, @viksit for the answer. I will update the tutorial document for instructions of updating the grpc service file.

inception_inference.cc implements async gRPC service. If you change the rpc name to Custom, then you need to also change service_->RequestClassify to service_->RequestCustom. As for method Classify, you should probably rename it accordingly for readability, but it's not syntactically required.

[eriophora]

Yeah that's what I've done. I think part of my confusion was because (and correct me if I'm wrong) Classify refers to the functions defined (or at least referenced?) in the *_pb2.py, and since my protoc call was incorrect, those functions weren't being built.

[fangweili]

If you implement synchronous gRPC service interface instead, then you will need to implement the same-name method Classify (see mnist_inference.cc for an example). *_pb2.py contains both synchronous and asynchronous service stubs, that's why you find Classify reference in it.

[eriophora]

@viksit good call on the C++ extensions for grpc. According to this, in order to install gRPC for C++, you need to build from source--however, the the TF serving docs refer you to these instructions. This will probably need to be changed too, @fangweili

[eriophora]

@viksit 's solution worked. I'm closing this issue.

For those who want to replicate this in the future, you need to install gRPC from source and (possibly) also via pip (I suspect both):

git clone https://github.com/grpc/grpc.git
cd grpc
git submodule update --init
make
[sudo] make install
# now you can install grpcio
[sudo] pip install grpcio

Then, navigate to the directory where the custom net files and execute (as per @viksit):

protoc -I ./ --python_out=. --grpc_out=. --plugin=protoc-gen-grpc=`which grpc_python_plugin` ./<your_prototext>.proto

If, in the terminal

which grpc_python_plugin

returns nothing, you have not install gRPC for C++

[viksit]

Thanks for the updates!

[nirbd]

@eriophora what do you mean "navigate to the directory where the custom net files " ? which folder exactly?