Protobuf: Add support for printing Javalite to JSON

[alexhilman]

Affects: javalite

We are encountering a limitation with json formating of proto messages on Android: JsonFormat doesn't work with javalite protos.

Our use case is stuffing message objects into an Elastic cluster by transforming the messages into json. This works for our server-side apps(which use normal protos), but not for Android. For example:
JsonFormat.printer().print(myLiteProto)
results in a compile error.

We have considered the following work arounds:

1. Sending the proto via grpc to a server which would then do json transformation and stuff it into elastic
2. Using or writing a protoc plugin to generate POJOs with a set of javalite-to-POJO transformers; then using Gson or Jackson to transform the POJO to json
3. Writing a custom gson/jackson plugin to directly translate GeneratedMessageLite objects into json
4. Using normal full-protos in Android

Obviously these are not ideal and option 4 seems to be the least-painful. Is javalite-to-json transformation something which could be added?

[xfxyjwf]

JSON support in Java lite is something we are considering but haven't decided yet. Protobuf binary format is smaller and more efficient than JSON and that's what we generally recommend. In your case, is it possible to use protobuf binary format on both the client and server?

[alexhilman]

For normal device-to-server communications (where we own the code on both devices) we already use the binary format with GRPC services.

For this use case (and a few others like it) we want to log the data to an Elastic cluster which works with JSON documents. It is for that reason which we need to have a proto-to-json conversion.

[ritikrishu]

+1 to this. We've a different use case where we need to convert the protobuf into JSON and pass it to a Javascript VM to do some stuff...but we are stuck on the conversion itself.

[baneyue]

We are also eager to support the protobuf-lite JSON conversion function, because the full package of this feature, Lite package has no reason not to support. We use it on Android.

[Globegitter]

Another use-case, we want to store protobufs in postgres but also being able to do some basic querying on it so we want to make use of postgres' JSONB column and store the proto as json there.

[hamberluo]

Any updates?

[wooseopkim]

https://github.com/protocolbuffers/protobuf/blob/d61aede/java/core/src/main/java/com/google/protobuf/MessageLite.java#L44-L46

Use MessageLite instead when you only need the subset of features which it supports -- namely, nothing that uses descriptors or reflection.

MessageLites per se does not have descriptor information, and that doesn't sound very hopeful. Is this still possible given that MessageLites don't have descriptors?

[wooseopkim]

I managed to implement this (very roughly) with reflection, but not really sure about its stability or performance yet.

[ian-ellis]

We are facing the same issue here. We are using protobuf for our analytics - mainly as a spec across platforms (ios, android, front end backend). However, we need to deliver the protos to segments SDK in the ios and Android clients which needs key value pairs. We simply need a reliable way to get field names - all other descriptions are unnecessary.

I have implemented this using reflection on the builder class (filtering getter methods and reformatting names) but it seems VERY unstable and risky.

We cannot use the full java protobuf as we have a dependency on firebase (which most Android apps would) which uses lite and they are not compatible. Any way to simply get the names of the fields would be hugely appreciated - a converter to JSON would be good enough as from there we could convert to whatever we liked

[wooseopkim]

@ian-ellis I stopped working on my implementation, but what about getting fields metadata from .proto file? This might be working without patches to Protobuf libraries. Never tried this myself though.

[laiyuejin99]

any update or timeline for this?