Google-cloud-go: bigtable: SingleColumnValueFilter equivalent

[maxcleme]

Currently, we are looking for a way to filter rows based on a specific cell value, but we also want the entire row content to be returned in case of a match. Which is not the case when using ValueFilter.

Is there any plan for a SingleColumnValueFilter equivalent in the go client ?

[vdlbk]

I experienced exactly the same issue with ValueFilter. It should be very nice to have this feature in the go client.
There is a workaround by filtering myself the values but it doesn't suit me (I obviously don't want to pull data I won't use)

[sduskis]

@vdlbk, you can see this adapter code about how the Cloud Bigtable HBase adapter converts ValueFilters to Cloud Bigtable requests.

[sduskis]

@maxcleme, You can see this adapter code for how Java constructs a Cloud Bigtable request out of a SingleColumnValueFilter.

FWIW, #1399 is related to this issue.

[maxcleme]

@sduskis , thanks for this insight, these comments really help me out to create an equivalent in the Go client.

bigtable.ConditionFilter(
    bigtable.ChainFilters(
        bigtable.FamilyFilter(fn),
        bigtable.ColumnFilter(cn),
        bigtable.ValueFilter(v),
    ),
    bigtable.LatestNFilter(1),
    nil,
)

As you can see, I use LatestNFilter as a workaround in the trueFilter since it suits my use case and return the entire row, but I couldn't find the pass_all filter.

After some investigation, the pass_all is implemented in the google.golang.org/genproto/googleapis/bigtable/v2 package but it is not present in cloud.google.com/go/bigtable as a Filter implementation.

I tried to implement my own based on how others filters are written :

func PassAllFilter(p bool) bigtable.Filter { return passAllFilter(p) }

type passAllFilter bool

func (paf passAllFilter) String() string { return fmt.Sprintf("passAllFilter(%t)", bool(paf)) }

func (paf passAllFilter) proto() *btpb.RowFilter {
    return &btpb.RowFilter{Filter: &btpb.RowFilter_PassAllFilter{PassAllFilter: bool(paf)}}
}

Sadly, this cannot works since the proto method from Filter interface is not exported.

Have you any idea how I could implement the pass_all filters or should I create a PR for it ?

Thanks :)

[sduskis]

@jadekler, can you please remove me and add @kolea2 instead?

[igorbernstein2]

Hi @maxcleme,

We will gladly take a PR that adds the the passAll filter in filters.go. Please see https://github.com/googleapis/google-cloud-go/blob/master/CONTRIBUTING.md for the process.

As a side note, I would recommend to make the passAllFilter a unit struct though.

Something like:

func PassAllFilter() bigtable.Filter { return passAllFilter{} }

type passAllFilter struct {}

func (paf passAllFilter) String() string { return "passAllFilter()") }

func (paf passAllFilter) proto() *btpb.RowFilter {
    return &btpb.RowFilter{Filter: &btpb.RowFilter_PassAllFilter{PassAllFilter: true}}
}

[maxcleme]

@igorbernstein2 , @kolea2

I just submit a PR with changes according to our discussion, however it seems that the Gerrit build is failing.

Honestly I don't get why, everything is working properly from my machine by using :

./run-tests.sh [COMMIT]

Gerrit link

PS : Don't mind all patches, I screwed up with the go.mod & go.sum files during the first push.

[igorbernstein2]

Looks like its an unrelated pubsub failure on go112:

=== RUN TestPublishTimeout
FATAL: 2019/05/13 12:52:23 grpc: Server.RegisterService after Server.Serve for "google.pubsub.v1.Publisher"
FAIL cloud.google.com/go/pubsub 8.584s

@kolea2, can you get someone from the yoshi team to take a look?

[maxcleme]

Contribution has been merged, I guess we can close it now?

https://github.com/googleapis/google-cloud-go/commit/741d62a9e2a86526f0e5a7bbc556c12bc1843b0c

[uschen]

@maxcleme @igorbernstein2 it seems there are quite a lot features supported by grpc are not exposed by the library (probably due to misuse concerns???)

I wonder if it is possible to expose the go Bigtable client's interface, e.g. Filter, RowSet. in this way, custom implementations can be done via the proto() *btpb.* methods.

[igorbernstein2]

Can you clarify what features are missing? In general we would like to avoid exposing raw protobufs on the client surface.

edit:
Lets close this issue and discuss missing features in a different issue

[jadekler]

@uschen It's entirely acceptable to simply use the underlying library for things that you very specifically need. The manual layers above the generated underlying libraries try to capture something like 90-95% functionality.

Closing this issue for now - please file a new issue if you have a specific feature request, per @igorbernstein2 .