K8s-config-connector: Establishing private IP for SQLInstance

[marcoderama]

I want to create a SQLInstance with a private IP. I've seen https://github.com/GoogleCloudPlatform/k8s-config-connector/issues/55 but don't really follow and have additional questions.

I already have a VPC network with private service connections (cloudsql-postgres-googleapis-com and servicenetworking-googleapis-com) in my project. (I _think_ these were setup automatically as described in https://cloud.google.com/sql/docs/postgres/configure-private-ip#prerequisites)

• Can I create a config-connector SQLInstance to use a private IP from that pre-existing connection? (I tried using ipConfiguration/privateNetworkRef/external: cloudsql-postgres-googleapis-com but no-go.)

I tried doing things using a ComputeNetwork, ComputeAddress, and ServiceNetworkingConnection to no avail. A network is created and an IP range is allocated, but no private service connection is setup.

The cnrm-controller-manager-0 log says:

{"level":"error","ts":1582744214.0397074,"logger":"kubebuilder.controller","msg":"Reconciler error","controller":"sqlinstance-controller","request":"mynamespace/sqlinstance-sample-damn11","error":"Update call failed: error applying desired state: Error, failed to update instance settings for : googleapi: Error 400: The incoming request contained invalid data., invalidRequest"

The GCP console "Operations and logs" for the instance shows:

Failed to create subnetwork. Please create Service Networking connection with service 'servicenetworking.googleapis.com' from consumer project 'xxxx' network 'dev-network-mynetwork' again.

So, to summarize:

• Can I use an existing private service connection from Config-Connector?
• If not, what am I missing in creating such a connection using the Config-Connector CRs?

Fyi, I'm using config-connector 1.2.0. My cluster is a local kind cluster at this time, but with gcp-key installed and proper namespace annotation in place. All other config-connector operations are working fine.

Any help/pointers would be appreciated. Let me know what other info you might need. Thank you!

[marcoderama]

I managed to get this working with a newly minted network using config-connector 1.2.0 on a GKE cluster. I'm still wondering if there's a way to use a pre-existing (sub)network though...

[marcoderama]

And figured that out. We just had to craft an external referent. e.g. Given that we have a pre-existing network dev-network, the SQLInstance spec needed to include:

spec:
  settings:
    ipConfiguration:
      privateNetworkRef:
        external: projects/<project>/global/networks/dev-network

[spew]

Hi @marcoderama, I'm glad you solved your problem. It would help us a lot if you could take a look at our documentation and tell us how you could find the 'external' section easier or what we could change to make this easier for you:

https://cloud.google.com/config-connector/docs/how-to/creating-resource-references

[marcoderama]

Thanks for following up with that link. Definitely on me for not having found that. That said, it still wouldn't have helped _me_ figure out how to form the value for a privateNetworkRef. I would have followed the link to https://cloud.google.com/apis/docs/overview and then been hopelessly lost. (I would have started with Networking APIs which doesn't seem to contain the magic.) Even knowing now what I'm looking for, I can't find it. I may be missing a necessary conceptual understanding though.

When working on this, I tried an external ref of dev-network but that failed. It was only after looking at someone else's terraform file for building the network in the first place that I saw what looked like a likely (and ultimately correct) candidate.

I typically look to the Config Connector resource reference page when crafting my yaml. In this case, the note for the SQLInstance external field points to https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names which doesn't really offer any help all.

So, a plain ol' example of an external ref for a privateNetwork is all I needed. It'd be great if I could find (a link to) that easily in the reference doco. And my apologies if that is out there and I missed it somehow. (Ultimately, that's why I posted https://github.com/GoogleCloudPlatform/k8s-config-connector/issues/99#issuecomment-591733713. Maybe that will help someone else.)

[spew]

Hi @marcoderama thanks for your very thoughtful feedback. I agree that figuring out which fields to set is hard as things exist today.

If we had an example in [1] that made use of dev-network do you think you would have looked there and found it? That would be the easiest thing for us to do right now. I do agree that our documentation of individual fields needs a lot of work to be more useful to you.

[1] https://github.com/GoogleCloudPlatform/k8s-config-connector/tree/master/resources/sqlinstance

[marcoderama]

I actually looked there so, yeah, an example there would have done the trick. (Imho, the simple examples are really helpful to get going, but it might be nice to also have complex examples that show every field in use (to the extent possible). Just my opinion though.)

Thanks again. (We can close this now if you want.)

[marcoderama]

Question answered thanks. Closing.

[rk185151]

Hi @marcoderama thanks for your very thoughtful feedback. I agree that figuring out which fields to set is hard as things exist today.

If we had an example in [1] that made use of dev-network do you think you would have looked there and found it? That would be the easiest thing for us to do right now. I do agree that our documentation of individual fields needs a lot of work to be more useful to you.

[1] https://github.com/GoogleCloudPlatform/k8s-config-connector/tree/master/resources/sqlinstance

URL does not work - is there an example to keep a private network and no public network for a cloudsql instance?

Thank you!

[jcanseco]

Hi @rk185151, I believe this sample should help you get started.

[rk185151]

@jcanseco, thank you. That sample was helpful. I have learned ipv4Enabled: false made the database instance private.