K8s-config-connector: Bigtable API write quota rapidly depleted

[Bj0rnen]

Describe the bug
After upgrade to version 1.10.0, Cloud Bigtable Admin API Instance and cluster write requests quota spiked massively. A single completely normal BigtableInstance declaration with 3 clusters which was created successfully and was completely in sync with the spec, was suddenly causing on the order of 1 write request every couple of minutes to the API. After rolling back to KCC version 1.9.1, it was making no write requests at all, as we expected, since it already matches configuration.

ConfigConnector Version
1.10.0 (rollback to 1.9.1 fixed the issue)

To Reproduce
Seemingly, all BigtableInstance objects result in this behavior. The sample one in this repo should make this happen, we think.

YAML snippets:
https://github.com/GoogleCloudPlatform/k8s-config-connector/blob/da9769c8d0b815368ce5d9317b6bd78ef49cab1c/samples/resources/bigtableinstance/bigtable_v1beta1_bigtableinstance.yaml#L15-L28

[spew]

Thanks for reporting this, we have identified an issue in our test for this scenario that prevented us from noticing. In addition, we are fixing the BigTableInstance resource such that it will not continuously reconcile.

[Bj0rnen]

Thanks, that's great to hear!

[xiaobaitusi]

This issue should be address in version 1.13.1. Let us know if it's still a problem.

[goldfishy]

Yay! Thank you! 🥳