Protobuf: ptypes: Timestamp<=>time.Time conversion instructions are hard to find

[ahmetb]

The time.Time does not play well with protobuf google.protobuf.Timestamp field.

Currently I am doing:

now := time.Time()
s := int64(time.Seconds()) // from 'int'
n := int32(time.Nanoseconds()) // from 'int'

ts := &timestamp.Timestamp{Seconds:s, Nanos:n}

which looks ugly and I'm not sure if I'm actually doing something bad while converting ints back and forth.

Why don't we just have a helper like func From(time.Time) timestamp.Timestamp?

[ahmetb]

This actually goes two-ways. I am failing to assigntimestamp.Timestamp back into time.Time. I can easily wrap this and define the conversion methods, but these should be in the library.

[ahmetb]

and I'm not sure if I'm actually doing something bad

yes I was doing something wrong so turns out the right methods are:

• now.Unix() this is int64 so it fits
• now.Nanosecond() this is int so I have to convert to int32
  
  
  
  • or maybe int32(now.UnixNano()-1000000*now.Unix()) from int64 ugh
  
  

[bcmills]

https://godoc.org/github.com/golang/protobuf/ptypes#TimestampProto

[broady]

@bcmills perhaps we should refer to that utility function from the docs in the timestamp proto? Other languages have complete code samples.

[ahmetb]

@bcmills I would expect this method to be in ptypes/timestamp.

[joelpresence]

Is there a reason to use google.protobuf.Timestamp in a message instead of a plain int64 nanos since epoch, if both the producer and consumer of my message are in go?

[ahmetb]

Is there a reason to use google.protobuf.Timestamp in a message instead of a plain int64 nanos since epoch, if both the producer and consumer of my message are in go?

That defeats the purpose of Protocol Buffers and the existence of Timestamp type. Representation of a type should be codified in the protos or protobuf spec, not on a specific platform/architecture.

[joelpresence]

@ahmetb well, I'd say "that defeats the purposes of Protocol Buffers" is a bit strong. As I see it, the purposes of Protocol Buffers are:

1. Cross-platform compatibility, like JSON.
2. Faster marshaling and unmarshaling than JSON.
3. Less memory-intensive marshaling and unmarshalling than JSON.
4. Smaller message payload than JSON.
5. Well-documented "schema" compared to JSON.

Since our platform is entirely golang, we care about 2-5 10x more than we care about 1. So how would using an int64 nanos since epoch "defeat the purpose of Protocol Buffers"? protobuf's Timestamp also smears leap seconds, which not all usages may need or want or even accept.

I was just looking for some comparisons of int64 nanos since epoch vs protobuf Timestamp.

I'm curious as to your thoughts? I'm not looking to start a flame war.

Thanks.

[ahmetb]

Your arguments are valid, however we're at a place where this type is already offered and there are already many APIs (such as Google Cloud APIs) relying on it. Therefore it's probably a bit difficult to roll that back.

I initially wrote the conversion functions myself and deployed, only to find out they are provided in some other package already. That's when I discovered my implementation is buggy, as in it doesn't handle all the cases perfectly.

In your project/company, you can just use int64 if you're certain that you can reliably convert it to in-memory structures back and forth in your platform of choice. I personally prefer not to rely on that as it sounds like it may break and doesn't include timezones etc.

[bohdantrotsenko]

@joelpresence

Protocol buffers do introduce a (de)serializing penalty, however small it is by itself or in comparison to JSON.

But, after dealing with lots of timestamps used by different teams, I came to appreciate imposed standards.

For instance, google.protobuf.Timestamp carries UTC time; and some people forget to set the time zone, it's good that the default conversion does that.

While your arguments are valid and int64 could "just work", there's still a value in having a separate struct for a timestamp.

[joelpresence]

Thanks @bohdantrotsenko

[JulienBreux]

Thanks @bohdantrotsenko

[paranoiacblack]

So... any advice on how to make these instructions more easy to find? I knew Timestamp was a WKT and that WKTs are in the ptypes so I also found the related TimestampProto conversion functions easily. Please re-open if you have a particular suggestion for this.

[cybrcodr]

You can use the google.protobuf.Timestamp mentioned in this thread. To use that in your .proto file, add --

import "google/protobuf/timestamp.proto";

and then use google.protobuf.Timestamp as the type of your field.

[gdegas]

Question here, I defined my type of google.protobuf.Timestamp but I want the unix seconds value. Are those not the same?

[neild]

google.protobuf.Timestamp is a message defined here:
https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/timestamp.proto

message Timestamp {
  // Represents seconds of UTC time since Unix epoch
  // 1970-01-01T00:00:00Z. Must be from 0001-01-01T00:00:00Z to
  // 9999-12-31T23:59:59Z inclusive.
  int64 seconds = 1;

  // Non-negative fractions of a second at nanosecond resolution. Negative
  // second values with fractions must still have non-negative nanos values
  // that count forward in time. Must be from 0 to 999,999,999
  // inclusive.
  int32 nanos = 2;
}