Kubespray: Feature Request - Configure private docker registries

[sermilrod]

FEATURE REQUEST:
Hi,

I see that there is nothing in the kargo repo related to docker login stuff and more precisely nothing about the ability to make kubernetes pull images from private registries such as ECR.

I have a solution for this that currently works for ECR and CoreOS and that could be extended to different registries solutions (azure one, custom private registry on premises...) and to other Operating Systems.

I would like to discuss the best way of including my work into Kargo as I do not know if my implementation is the most suitable to this project organisation or if it requires more work to deal with things I might have missed.

I created a separated role to tackle it and I achieved it by using kubernetes secrets (https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/):
kubectl create secret docker-registry regsecret --docker-server=<your-registry-server> --docker-username=<your-name> --docker-password=<your-pword> --docker-email=<your-email>

To get the docker password I have to run the docker login command to authenticate against the registry and this could be a registry of any type. In the specific case of AWS ECR the authentication expires every 12h so it is compulsory to renew the AWS ECR login token. To overcome this situation I have a systemd service that runs every 6h and it automatically renews the token so kubernetes is able to pull images at any time from the private registry.

Please let me know what you think.

Thank you very much in advance!
-->

Environment:

• Cloud provider or hardware configuration:
  On premises HyperV

• OS (printf "$(uname -srm)\n$(cat /etc/os-release)\n"):

Linux 4.9.24-coreos x86_64
NAME="Container Linux by CoreOS"
ID=coreos
VERSION=1353.7.0
VERSION_ID=1353.7.0
BUILD_ID=2017-04-26-2154
PRETTY_NAME="Container Linux by CoreOS 1353.7.0 (Ladybug)"
ANSI_COLOR="38;5;75"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://issues.coreos.com"

• Version of Ansible (ansible --version):

ansible 2.3.0.0
  config file = 
  configured module search path = Default w/o overrides
  python version = 2.7.6 (default, Oct 26 2016, 20:30:19) [GCC 4.8.4]

Kargo version (commit) (git rev-parse --short HEAD):
8eb60f5

Network plugin used:
Calico

Copy of your inventory file:

shared-k8-master-eu01 ansible_ssh_host=shared-k8-master-eu01.domain.com
shared-k8-master-eu02 ansible_ssh_host=shared-k8-master-eu02.domain.com
shared-k8-etcd-eu01 ansible_ssh_host=shared-k8-etcd-eu01.domain.com
shared-k8-etcd-eu02 ansible_ssh_host=shared-k8-etcd-eu02.domain.com
shared-k8-etcd-eu03 ansible_ssh_host=shared-k8-etcd-eu03.domain.com
shared-k8-worker-eu01 ansible_ssh_host=shared-k8-worker-eu01.domain.com
shared-k8-worker-eu02 ansible_ssh_host=shared-k8-worker-eu02.domain.com
shared-k8-worker-eu03 ansible_ssh_host=shared-k8-worker-eu03.domain.com

[kube-master]
shared-k8-master-eu01
shared-k8-master-eu02

[etcd]
shared-k8-etcd-eu01
shared-k8-etcd-eu02
shared-k8-etcd-eu03

[kube-node]
shared-k8-worker-eu01
shared-k8-worker-eu02
shared-k8-worker-eu03

[k8s-cluster:children]
kube-master
kube-node

Command used to invoke ansible:

ansible-playbook -u core --private-key=$HOME/.ssh/id_rsa --become cluster.yml -D --tags registry

new tag registry is the tag I assigned for the new feature in my current implementation

Output of ansible run:
https://gist.github.com/sermilrod/c13bb75a149d0e5aa2820e4bd0ddd04c

[PPTrue-love]

hello, I'm interested in your "feature", I would like to deploy a remote private registry in my cluster installed in customer premises, can you give me some hints please? Regards

[deimosfr]

+1

[ant31]

/lifecycle frozen
/remove-lifecycle stale

[gutek9]

According to latest changes in DockerHub (https://www.docker.com/blog/scaling-docker-to-serve-millions-more-developers-network-egress/), maybe it's worth considering to add such feature? For example, when I provision a lot of environments using Kubespray, I am reaching regularly anonymous limit of pulls.

Maybe some way of providing templated config.json? Or providing credentials with group_vars (I know, not secure really)? Or maybe from command line during Kubespray execution?