Aws-iam-authenticator: Constructing many clients error

Created on 7 Apr 2020 · 16Comments · Source: kubernetes-sigs/aws-iam-authenticator

Hello all, recently we've enabled aws-iam-authenticator and I'm seeing an error message whenever I run anything that uses --all-namespaces (i.e. kubectl --context cluster get pods --all-namespaces).

The error message in question is:

W0407 12:20:16.744257   32782 exec.go:203] constructing many client instances from the same exec auth config can cause performance problems during cert rotation and can exhaust available network connections; 1001 clients constructed calling "aws-iam-authenticator"

kubectl version v1.18.0
aws-iam-auth version: {"Version":"v0.5.0","Commit":"1cfe2a90f68381eacd7b6dcfa2bf689e76eb8b4b"}
OSX: 10.14.6

lifecyclrotten

Source

arukaen

👍9

Most helpful comment

/reopen
This is still an issue Nothing has been resolved.

HeavensRegent on 5 Nov 2020

👍12

All 16 comments

This is actually a warning message, and is mainly targeted for resource usage when using client certificate auth. Kubectl/client-go will only invoke aws-iam-authenticator once and reuse the token for multiple connections.

micahhausler on 13 Apr 2020

Is there a way to turn off this message?

arukaen on 17 Apr 2020

You might be able to set --stderrthreshold=ERROR (see klog), but if that doesn't work you'd have to file an issue on kubernetes.

micahhausler on 17 Apr 2020

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot on 16 Jul 2020

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

fejta-bot on 15 Aug 2020

I'm getting about 2000 lines of this error also when running kubectl today. There is no --stderrthreshold option for aws-iam-authenticator. Is there any other fix?

max-rocket-internet on 2 Sep 2020

My errors looked like this:

W0902 17:12:19.796919   25579 exec.go:203] constructing many client instances from the same exec auth config can cause performance problems during cert rotation and can exhaust available network connections; 1384 clients constructed calling "aws-iam-authenticator"

So I just filtered it out with grep, but this is quite hacky:

$ kubectl describe pods 2>&1 | grep -v W0902

max-rocket-internet on 2 Sep 2020

👍5

This warning appears continuously when running https://github.com/vmware-tanzu/octant

sysadmiral on 28 Sep 2020

👍6

@sysadmiral I am getting the same warning using octant. Know of any work around?

dunjoye4real on 2 Oct 2020

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

fejta-bot on 1 Nov 2020

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot on 1 Nov 2020

/reopen
This is still an issue Nothing has been resolved.

HeavensRegent on 5 Nov 2020

👍12

@HeavensRegent: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen
This is still an issue Nothing has been resolved.

k8s-ci-robot on 5 Nov 2020

philoserf on 19 Nov 2020

Also getting this and commented on #91913 with more information.

ajcann on 7 Dec 2020

Seeing this (slightly different message so adding for Google) when using aws to load data from S3 in initContainer:

connections; 1377 clients constructed calling "aws"
W1218 16:36:11.240927 1982077 exec.go:271] constructing many client instances from the same exec auth config can cause performance problems during cert rotation and can exhaust available network
connections; 1378 clients constructed calling "aws"
W1218 16:36:11.240941 1982077 exec.go:271] constructing many client instances from the same exec auth config can cause performance problems during cert rotation and can exhaust available network
connections; 1379 clients constructed calling "aws"