Google-cloud-ruby: Add support for Bigtable

[mbrukman]

Could you please add support for Bigtable?

Support for the Bigtable Admin API can be added to the master branch as it does not require any additional work beyond just calling the gRPC APIs. We can start with the Bigtable Data API on a separate branch and merge to master once it's ready for general consumption; Bigtable protocol requires additional development beyond just calling the gRPC APIs.

Thanks!

/cc: @jgeewax, @sduskis

[quartzmo]

@mbrukman

We would be very happy to add support for Bigtable! The only issue of course is understanding how to prioritize it relative to our existing pipeline. I assume you are hoping for Q1 2017?

Bigtable protocol requires additional development beyond just calling the gRPC APIs.

Can you provide links and/or more detail on this? I see that google-cloud-python and google-cloud-node have recently added Bigtable packages in master. Can they serve as examples for us?

[mbrukman]

@quartzmo

The only issue of course is understanding how to prioritize it relative to our existing pipeline. I assume you are hoping for Q1 2017?

Yes, please, though I'm not going to object to anything happening in Q4 2016. :-)

Bigtable protocol requires additional development beyond just calling the gRPC APIs.

Can you provide links and/or more detail on this? I see that google-cloud-python and google-cloud-node have recently added Bigtable packages in master. Can they serve as examples for us?

The google-cloud-python repo is more complete than google-cloud-node, so I would use that as a model. That said, the Bigtable implementation in google-cloud-go is even better.

There are several things above and beyond gRPC support that are necessary:

• reconstructing streaming replies from the server
• batch read/write operations are not atomic, issuing a batch of ops may cause some to succeed while others fail; the caller needs to parse the response and re-issue the failed ones until some timeout or number of failures are exceeded, with some backoff
• some performance optimizations were done in other languages, such as connection pooling, this is language-specific as to what needs to be done

@sduskis and @garye may have additional comments.

[mbrukman]

@quartzmo -- any updates on this? Can we get started with a set of generated classes from gRPC protobufs and proceed from there?

[quartzmo]

@swcloud @geigerj What do you think?

[geigerj]

Hi @mbrukman -- I'm going to follow up with you internally about getting BigTable on the client library pipeline. At the moment, there is no ETA for support.

[quartzmo]

Thanks @geigerj!

[geigerj]

Note: I'm going to leave this issue open to track the feature requests beyond gRPC, but this will be blocked on basic GAPIC support, which is tracked internally.

[jochenseeber]

Is there any news on this? I really would be able to use BigTable :-)

[geigerj]

@jochenseeber Sorry, no updates yet.

[geigerj]

Closing, as this is being tracked internally. The ETA for support is currently 2017Q4.

[mbrukman]

Thanks for the update, @geigerj! We're all looking forward to it.

[mbrukman]

Looks like this was implemented in PR https://github.com/GoogleCloudPlatform/google-cloud-ruby/pull/1786 — thank you!

[sduskis]

We will need to either reopen this issue, or create a new one. The Cloud Bigtable team would like to flesh out a hand-written client. The GAPIC client isn't enough by itself to make a usable client.

[retnuh]

Is there an update on this? Is there a new issue tracking further work, or will updates be posted here?

[quartzmo]

@geigerj Can we reopen this issue? It states:

Bigtable protocol requires additional development beyond just calling the gRPC APIs.

[geigerj]

@quartzmo Sure.

@retnuh No updates so far, but we expect to begin work early next year. The new ETA is by the end of 2018Q1.

[quartzmo]

Thanks @geigerj !

[retnuh]

Thanks for the update @geigerj !

[blowmage]

This is still on track for 2018Q1.

[scottrobertson]

Is this still on track for end of Q1?

[sduskis]

We ought to have a functional Cloud Bigtable client by the end of the quarter. I don't think that the Q1 client will be production ready. Given our experience with other languages, it is highly likely that we will have to work on reliability and performance in subsequent quarters.

[sduskis]

We're still working on this. We have plenty of code written, and we have some PRs (#1990 for an improved admin and #2001 for the data API). We need to get them reviewed. We'll still have a bit of engineer work for a fully reliable production grade client. We'll do what we can to move that forward in Q2.

[sduskis]

This is almost done. We should be close to Alpha soon.

[quartzmo]

@sduskis Can I close this issue now, since the release is imminent, and the PR(s) is/are a better place to continue discussion? Because of its age, this issue is out of SLO.

[quartzmo]

Closing, because a considerable amount of Bigtable support has been merged into master branch. See #2132, #2102, #2001, et al.

[lolripgg]

@quartzmo Got an ETA on when google-cloud-bigtable will be available via RubyGems?

[quartzmo]

Got an ETA on when google-cloud-bigtable will be available via RubyGems?

@sduskis Any ETA?