Dubbo: how to use dubbo in docker

[463535160]

目前我有一个疑惑，dubbo的服务消费者跟提供者都用容器去包装，然后发布在容器云里面。一般来说不应该去修改容器镜像内的东西，但是服务的发现会用网卡ip信息去实现。网上有说要改host文件或者该dubbo的配置文件，但都不太现实，如果是基于弹性伸缩的情况下，每次都得去改配置，而且注册的都是用宿主机的ip，很显然是不正确的。有没有办法通过DNS做域名映射实现服务注册以及服务消费呢？或者dubbo团队能否提供一些镜像案例给我们参考参考。

[beiwei30]

pls. check https://github.com/dubbo/dubbo-docker-sample

[463535160]

Hi~This demo project I have been found,but this always used the host.How to do with out the host ip?We can use docker‘s virtual network link to network between the zookeeper and the provider. Thank you for your reply.

[chickenlj]

but this always used the host.How to do with out the host ip?We can use docker‘s virtual network link to network between the zookeeper and the provider.

What do you mean by without the host ip and use docker‘s virtual network link to network, i don't understand, could explain in more details.

[463535160]

Thank you for your reply.If the elastic expansion of k8s is used. And the provider containers are in the same host. When beiwei30 is used, there may be a collision between host port and host port. I need to use the network in the container to interact. Implement the service registration and discovery of provider and consumer in the container.

[chickenlj]

I think i understand your requirement, could you please send an email to the apache dev mailing list: [email protected]? Remember to bring this issue link so people can learn the detailed context.

[463535160]

Ok.The email has been sended.Glad to contact with you.

[vangoleo]

@463535160 I have worked with Dubbo, Docker and k8s, and maybe I can give you some suggestion.
As far as I know, Docker is just a container engine, with very simple network ability. If you just use Docker itself, the only way to implement interaction between multiple container on different machines, is to use the Host network mode. The container will share the Host's IP and Port. Dubbo will register the IP of Host to the registry(e.g. zookeeper).
The above solution has a problem, the container share the Host's IP and port, so every container must have different ports, which lead to extra configuration work. Meanwhile, it is hard to implement auto scaling.
Docker is not designed to resolve these problems, and here comes the container Orchestration tools, for example, Docker swarm, Google Kubernetes(k8s).
In k8s, we can use some network component to setup a virtual network:

• each container has an unique IP, which is called Pod IP
• any two container can communicate with each other directly, even they are in different hosts.
  I use Flannel to implement above network.
  So if you deploy a containerized dubbo application to k8s cluster, the app will register the Pod IP to Zookeeper. When a new container is deployed to k8s, k8s will assign a Pod IP to it, and ensure the Pod IP is unique, and it can reach to all the other containers directly in the k8s cluster. So when you want to scale up, you just need to deploy new container to k8s. In fact, just one command in k8s.

So in k8s cluster:

• service registration and discovery is implemented by Dubbo(zookeeper) itself.
• the network communication is implemented by k8s(flanneld).

You also mentioned the DNS, k8s also have a DNS plugin, out of box. The DNS is used to map a k8s service name to a k8s service IP, also known as Cluster IP. The Service is in fact a load balancer in the front of several container replicas. So you can use a readable service name, for example, zookeeper to visit the zookeeper cluster, instead of the Service IP.
If you are interested in these, you can find more details by your self. Or you can have a look at my wiki:
https://github.com/vangoleo/wiki/tree/master/k8s

Hope this may help you.

[463535160]

Thanks for you help.I'm trying it now.When I slove it.I will share each other.

[xhuiinit]

@vangoleo ,I also have worked with Dubbo, Docker and k8s，and have some questions：
With docker and k8s, zookeeper is deployed under docker. After the container is restarted, the virtual IP of zookeeper will change.
Dubbo is configured with the service address of zookeeper, such as: zookeeper://zk-headless. Kube-system :2181,
The parsed virtual IP is still rebooting the previous IP;Then the dubbo connection is not connected to zookeeper.
Do you have any problems like this?

[463535160]

@xhuiinit If zookeeper be down, consumer will build a lock file.Also it can be used without zookeeper

[463535160]

@vangoleo It just to be used in container .While no-container consumer want to link container provider，it couldn't be work.If we use the DNS proxy. Maybe it could be sloved.

[chickenlj]

I think we should have a guidance of how dubbo can be deployed in docker or K8S.

Dubbo itself should also learn from Service Mesh, learn how to cooperate with fundamental infrastructures like K8S.

[xhyspring]

Now I begin to deploy dubbo/zookeeper in k8s, very appreciate if have guidance of how dubbo can be deployed in docker or K8S.

[MouGG]

@chickenlj Hi, your link https://github.com/dubbo/dubbo-docker-sample 404, would you please provide an avaliable link?

[xhkyyy]

@463535160
I am trying to deploy dubbo on kubernetes using the docker image.
There is some description on how to deploy it: https://gist.github.com/tnndwc/bbb5c6d85b4e52f833d75670e2913bd2
(I pass the environment variables DUBBO_IP_TO_REGISTRY、DUBBO_PORT_TO_REGISTRY to shell script which runs the dubbo service using docker.)

[ralf0131]

@MouGG Please check here: https://github.com/dubbo/dubbo-samples/tree/master/dubbo-samples-docker

[objcoding]

@463535160 你好，可以了解一下docker swarm集群模式下的overlay跨主机网络模型，在跨主机网络模型下，容器之间是可以通信的，意味着即使dubbo将docker容器的ip地址注册到zookeeper，只要在同一个跨主机网段下，是可以做到通信的。

[objcoding]

@463535160 同样地，在k8s下集群模式下每个容器都有一个pod ip，在k8s集群模式下每个pod同样可以互相通信。
也就是说，如果你简单地用docker，那么你就要额外地配置，将宿主机作为注册ip地址，如果你用了一些集群方案，比如k8s、swarm等，这些工具已经帮你想好解决方案了。