Kops: Not able to download K8s components from gcr.io

[jepsenwan]

Since from China region, we are not able to pull Docker images from gcr.io registry.
But in KOPS cluster cluster.spec the K8s services like kube-apiserver, all need to download from gcr.io, therefore no any K8s components are running on the nodes and master created by KOPS.
image: gcr.io/google_containers/kube-controller-manager:v1.7.0.
Would you figure out if I can change the docker downloading repository before update the cluster.
Thank you very much in advance!

[chrislovecnm]

With the assets API value you are able to set you own docker registry. No documented yet since it is still a work in progress. You still need a workstation that can upload them to a registry

[jepsenwan]

Thanks. Do you mean that I can download the cluster components from my private registry or from docker registry by altering the cluster.spec file? Is this file created after updating cluster with --yes or It exists right after the 'create cluster' command?

[KashifSaadat]

The Assets API values are here: https://github.com/kubernetes/kops/blob/0620cce89e6b5d2f72c05e4cedce3d268f4edca7/pkg/apis/kops/cluster.go#L142-L151

You can modify your cluster spec after initially running the create command (before running update), just run kops edit cluster <cluster-name>.

[chrislovecnm]

Thanks for the assist @KashifSaadat

Also look at the documentation on using manifests / yaml form your cluster. It is in the docs directory. The file repository is not fully implemented yet, and the container registry should work. Needs to be a public registry, and kops will stage the containers using the computer that is running kops, because of that you need docker running on the computer that you run kops on, since kops pulls and pushed the containers for you.

Still a very new feature, let us know if you have problems!

[chrislovecnm]

@jepsenwan looking through the code we still need a couple more PRs to get in before this is functional

[jepsenwan]

@KashifSaadat Thanks for reminding me of this, I am gonna give it a try.

[qiuyuzhou]

@jepsenwan You could set the http proxy. https://github.com/kubernetes/kops/blob/master/docs/http_proxy.md
I have install k8s cluster in region cn-north-1 last week with this approach.
You must use a master version kops.

[jepsenwan]

@KashifSaadat I tried issuing 'kops edit cluster xx', but there is no any objects regarding kube-apiserver, kube-proxy inside. If there is any other way I can specify image registry of these objects? Thanks.

[jepsenwan]

@chrislovecnm Thanks. I have got the instance that is running kops installed with docker. But it seems to still haven't download anything...

[jepsenwan]

@qiuyuzhou, Thanks buddy. I have tried editing cluster, adding this object to cluster configuration. But it merely returned "/tmp/kops-edit-cmge1yaml" 66L, 1528C written A copy of your changes has been stored to "/tmp/kops-edit-cmge1yaml".

My kops version is Version 1.7.0 (git-e04c29d). Is there any step I took wrong? Or I need to start over...

Edit cancelled, no valid changes were saved.

[qiuyuzhou]

@jepsenwan You need build the latest version kops by yourself. The http proxy feature haven't been released.

• Install go lang
• Clone kops master branch to your $GOPATH/src/k8s.io/
• $ make
• Then check kops version. The kops binary file should be in $GOPATH/bin.

$ kops version
Version 1.7.1-beta.1 (git-189fa3c85)

You can set docker registry mirrors:

docker:
    logDriver: ""
    registryMirrors:
    - https://abc.mirror.aliyuncs.com

Exclude the s3 and docker mirror

egressProxy:
  excludes: amazonaws.com.cn,aliyuncs.com
  httpProxy:
    host: a.b.c.d
    port: 8080

[KashifSaadat]

Hi @jepsenwan. As mentioned above, you'll need to build kops from source to get the http proxy feature as it was newly introduced to the codebase and so hasn't yet made it into a release.

The Assets Spec sits at the root of the ClusterSpec, so isn't defined within the KubeAPIServer or KubeProxy. These Specs aren't defined by default on a new standard cluster creation, you can just type it directly in when editing the cluster. On save it will warn you if there is a typo, but best to check against the API definitions (https://godoc.org/k8s.io/kops/pkg/apis/kops#ClusterSpec).

As @chrislovecnm mentioned, the Assets FileRepository is not fully functional yet (related PR: #3254).

[jepsenwan]

@qiuyuzhou, Thanks for the detailed workaround. But I couldn't download them even though I have applied a docker accelerator from Aliyun and used it by kops edit cluster $NAME.

[jepsenwan]

@qiuyuzhou.Would you recommend any useful web proxy? Thanks again.

[qiuyuzhou]

@jepsenwan
You must use a http proxy in order to download docker images from google registry 'gcr.io'.
The aliyun mirror is only the mirror of DockerHub.

Prepare your proxy servers like below:

|privoxy -> sslocal |-->|GFW|--> |ssserver|--> The Freeworld

• Shadowsocks
• Privoxy

[chrislovecnm]

/assign

Will close once assest API is finished

[xiaods]

any easy way to alter the gcr.io to private registry.

[emacsliu1107]

I make it just now....

1. Download shadowsocks.
2. Get a ss account and connect ss.
3. Download privoxy.
4. Set up forward rule, default to 8118.
5. Set docker proxy, and set http(s)_proxy to http://localhost:8118 (according to 4)
6. Restart docker daemon and docker.
7. Enjoy it.

[chrislovecnm]

Assets API is completed, and there are also options to redirect the docker registry, going to close this for now.

[joelittlejohn]

@chrislovecnm I'm using the containerRegistry option with great success. Is the fileRepository feature usable in kops 1.8?