Cloudsql-proxy: Any cpu and memory limits recommendations for kubernetes usage?

[pdecat]

Hi,

are there any cpu and memory limits recommendations for kubernetes usage?

That could be worth mentioning in the README.md.

[Carrotman42]

It completely depends on how you use it, i.e. connection/query patterns.

CPU usage is proportional to the I/O rate.

Memory usage is proportional to the number of active connections through
the proxy at a specific time.

The best way to figure out the limits for the Proxy in conjunction with
your specific application is to test it under high, medium, and low loads
(on your application) to get an idea of how your application acts in
regards to data rate and connection count under different scenarios.

[pdecat]

Hi @Carrotman42, thanks for the advice.

Could there be a way to configure the maximum number of active connections the proxy accepts?
That would somehow put a limit on the memory it uses at any time, isn't it?
But that's probably out of its scope as it would probably then act as kind of a connection pool.

FWIW, at the time being, we're deploying the cloudsql-proxy as a sidecar container in our autoscaled application pod. From time to time, some cloudsql-proxy containers are OOM killed as they hit the current memory limit we have set.
In addition to tuning this limit, we are thinking to move it outside of this pod as a service or a DaemonSet to have less cloudsql-proxy containers with higher limits.

[Carrotman42]

There is no way to limit connections via the Proxy, it will blindly attempt to connect to the database any time something connects connects to it. As you alluded to, it is up to your application to deal with connection tracking, pooling, and limiting as appropriate.

That being said, I'm not completely opposed to having a hard-limit in the Proxy to help people prevent applications from OOMing the proxy if it is negatively affecting you. I suppose that per-connection denial is better than blindly accepting connections until something decides the Proxy shouldn't hold as much memory as it does (which would cause all existing connections to be dropped, which can be a frustrating failure mode).

[ye]

CloudSQL does published a doc denoting connection limits based on different size of the GCP VMs: https://cloud.google.com/sql/docs/mysql/diagnose-issues though.

What metrics to look for other than CPU/Mem to look for to benchmark? If it's queries per second, does CloudSQL PostgreSQL dashboard provides that?

[kurtisvg]

Closing - this issue hasn't been updated in some time and appears to be abandoned. If you feel it requires more attention feel free to reopen.

In terms of connection pooling, it is probably best to handle it inside your application at the moment.

[mikedickey]

Just for the record, not using resource requests for this is not only bad practice, but also breaks some documented GKE features. In particular, HPA doesn't work as documented here for any pods that include cloud-sql-proxy as per its documentation.

Regardless of how you deploy it (via Web console or YAML), an HPA will always report "failed to get cpu utilization: missing request for cpu" (or similar for memory), contrary to GKE's docs. It took a bit of digging to discovery that my HPA was doing this because my pod included a cloud-sql-proxy container which lacked resource requests.

I believe it would be far better for all the examples to include resource requests in the YAML examples. There is no way for most people to know what are reasonable values to use. And I don't see anything about this in the doc link above claiming to provide guidance. At a minimum, the examples should note the problem and provide a link to guidance.