Spring-cloud-netflix: Need to tell eureka client running in Docker to use host's IP

Created on 10 Jul 2015 · 36Comments · Source: spring-cloud/spring-cloud-netflix

I am running a Spring Cloud application inside a Docker container.
While I can tell Eureka to use the exposed port (e.g. EUREKA_INSTANCE_NON_SECURE_PORT: 8081) , it doesn't seem to let you tell it to register using the host's ip address.
Instead, eureka registers the instance using the container's ip address.
That fine unless I want to access the container from another host.
What would be great is the ability to set an environment variable such as EUREKA_INSTANCE_NON_SECURE_HOST: docker_host_ip_address

Source

cer

Most helpful comment

And what about:

eureka.instance.prefer-ip-address=true
eureka.instance.ip-address=

brenuart on 11 Jul 2015

👍9 👎5

All 36 comments

I wonder if eureka.instance.hostname will work. If not, then we'll have to do something to fix it.

spencergibb on 11 Jul 2015

And what about:

eureka.instance.prefer-ip-address=true
eureka.instance.ip-address=

brenuart on 11 Jul 2015

👍9 👎5

Difficult to tell from looking at the class in isolation. Depends on
whether getHostName(refresh) is called with refresh=true or false.

cer on 11 Jul 2015

@cer maybe an option to disable the refresh

spencergibb on 13 Jul 2015

I haven't looked at Spring Cloud enough to know precise what refresh is
intended to do.
However, I think that a user specified host name/ip address should always
take precedence.

cer on 13 Jul 2015

Maybe I don't get it, but setting eureka.instance.hostname to the desired value should work (at least it does on our side).
If you set eureka.instance.prefer-ip-address=true, then the registration with Eureka will instead use the IP address specified by eureka.instance.ip-address (which defaults to host's IP after resolving).

@cer: the user specified value takes precedence over the default values. Do you see a different behaviour?

brenuart on 13 Jul 2015

@brenuart we used to ignore the refresh, but then I did 9149319bd73f228896d52b733894447170e4f26f which will refresh at somepoint in netflix code. @cer, I agree, we just have trouble knowing when the value was user specified or not, so either another user specified hostname field or the flag to not refresh.

spencergibb on 13 Jul 2015

@spencergibb Ok - I see. Sorry, I wasn't aware of that change.
To be honest, we used to override part of the EurekaInstanceConfigBean to add some validation and other useful default values/behaviour. To make it short, we moved the initialisation of default values out of the config bean to the method that creates it. The config bean is now a simple POJO and we are not "hit" by your latest change...
I should definitely create a PR with those changes for you to see and pickup what you believe is interesting.

brenuart on 13 Jul 2015

@cer @spencergibb Having a slightly more complicated version of the same problem. I am experiencing the same issue for the host. On top of that the container is launched with -P meaning a random host port is assigned. With the current setup there is no way to get that random port assignment. The port is being pulled from server.port.

Another note, about assigning something within eureka.instance.hostname or eureka.instance.ip-address is that if this is a docker cluster in our case AWS ECS. The host and IP can't be statically configured within properties.

ccit-spence on 26 Jul 2015

Something that can be done from docker is just using the docker run -h hostname or ip Eureka will pick up the host name since it it written to the etc/hosts file

Not sure yet how to handle this for a cluster.

ccit-spence on 26 Jul 2015

Update: Further testing with AWS ECS even with adding the hostname to the properties it does not work. Unless all instances are docker containers i.e. Eureka, Zuul and UI and API services on the same host. Not sure If I am missing something. It seems that Ribbon is looking for something different than what Eureka is showing via the UI. Even if I could get it to work. As soon as the docker container switched to a different EC2 container the old hostname would be invalid.

ccit-spence on 26 Jul 2015

What do you mean by "docker container switched to a different EC2 container" exactly?

cer on 19 Aug 2015

@cer In ECS you have a cluster of EC2 instances. Due to the nature of EC2 unless you specify a Service to run on only one EC2 instance it can appear anywhere within the cluster.

That being said I now have ECS working for Spring Cloud. The solution is to curl the hostname of the host from within the Docker container during its boot process and assign the hostname to an environment variable. Then add the discovered hostname to the Eureka instance metadata.

ccit-spence on 19 Aug 2015

@cer look at this - #30
I've got same problem - AWS, Docker and I've resolved it using proposed custom configurations.

rozhok on 25 Aug 2015

@rozhok Are you doing something like the below? This is how I managed to get ECS to work.

entrypoint.sh

#!/bin/sh
export HOST=$(curl --retry 5 --connect-timeout 3 -s 169.254.169.254/latest/meta-data/local-hostname)
export LOCAL_IP=$(curl --retry 5 --connect-timeout 3 -s 169.254.169.254/latest/meta-data/local-ipv4)
exec "$@"

Dockerfile

FROM privaterepo/oracle-java8

RUN bash -c 'apt-get -qq update'
RUN bash -c 'DEBIAN_FRONTEND=noninteractive apt-get install -y curl'

EXPOSE 8080

COPY *.jar target/app.jar

COPY entrypoint.sh /usr/local/bin/entrypoint.sh

RUN touch target/app.jar

RUN chmod +x /usr/local/bin/entrypoint.sh

ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]

CMD java -jar target/app.jar

ccit-spence on 25 Aug 2015

👍1

No, I've created custom Eureka Config so I don't need to setup additional scripts in Dockerfile. All stuff are described in #30 .

rozhok on 25 Aug 2015

Another one approach described in Spring Cloud Netflix docs:

@Bean
@Profile("docker")
public EurekaInstanceConfigBean eurekaInstanceConfig() {
    EurekaInstanceConfigBean config = new EurekaInstanceConfigBean();
    AmazonInfo info = AmazonInfo.Builder.newBuilder().autoBuild("eureka");
    config.setDataCenterInfo(info);
    info.getMetadata().put(AmazonInfo.MetaDataKey.publicHostname.getName(), info.get(AmazonInfo.MetaDataKey.publicIpv4));
    config.setHostname(info.get(AmazonInfo.MetaDataKey.publicHostname));
    config.setIpAddress(info.get(AmazonInfo.MetaDataKey.publicIpv4));
    config.setNonSecurePort(port);
    return config;
}

This will put proper IP (public in my case, but you may change it to local) and port into Eureka registry.
Don't forget to add proper spring profile into Java env props, like this:

FROM java:8
EXPOSE 14862
ENV JAVA_OPTS -Dspring.profiles.active=docker
ADD build/distributions/portal.tar /
ENTRYPOINT ["/portal/bin/portal"]

rozhok on 27 Aug 2015

👍4

@cer @rozhok @brenuart @ccit-spence anyone still having issues that we need to address?

spencergibb on 11 Nov 2015

My solution works fine for me.

ccit-spence on 11 Nov 2015

@spencergibb I'm using solution described above. A little bit hackish but fine at the moment.

rozhok on 11 Nov 2015

I'm fine with this.

brenuart on 11 Nov 2015

It can be used to do my own server not AWS？

@Bean
@Profile("docker")
public EurekaInstanceConfigBean eurekaInstanceConfig() {
    EurekaInstanceConfigBean config = new EurekaInstanceConfigBean();
    AmazonInfo info = AmazonInfo.Builder.newBuilder().autoBuild("eureka");
    config.setDataCenterInfo(info);
    info.getMetadata().put(AmazonInfo.MetaDataKey.publicHostname.getName(), info.get(AmazonInfo.MetaDataKey.publicIpv4));
    config.setHostname(info.get(AmazonInfo.MetaDataKey.publicHostname));
    config.setIpAddress(info.get(AmazonInfo.MetaDataKey.publicIpv4));
    config.setNonSecurePort(port);
    return config;
}

Dreampie on 22 Dec 2015

👍2

@Dreampie not, it won't be used on your own infrastructure since it using AWS-specific SDK.

rozhok on 22 Dec 2015

@rozhok thank you. and how can eureka client registered with host ip，not docker container ip？if use --net=host always return 127.0.0.1

Dreampie on 22 Dec 2015

Use some env variables passed into docker on startup time.

rozhok on 22 Dec 2015

@rozhok but if this machine's ip changed, must recreate container?

Dreampie on 22 Dec 2015

Sure.

rozhok on 22 Dec 2015

Or trigg some API insinde container to reload eureka configuration.
Something like /api/eureka/changeIp etc. There is a lot of ways to solve
this task but non of them are obvious or elegant. Sad.

rozhok on 22 Dec 2015

💦thanks

Dreampie on 22 Dec 2015

@Dreampie Using the custom Eureka config as in your post, what do the application.yml and bootstrap.yml look like?

asarkar on 27 Dec 2015

is there any update about this issue?

cswanghan on 10 Apr 2017

@cswanghan no, it was closed in 2015

spencergibb on 12 Apr 2017

@spencergibb im still facing the issue

momentum123 on 7 May 2018

@momentum123
me too

wenfei3 on 6 Sep 2018

It can be used to do my own server not AWS？

@Bean
@Profile("docker")
public EurekaInstanceConfigBean eurekaInstanceConfig() {
    EurekaInstanceConfigBean config = new EurekaInstanceConfigBean();
    AmazonInfo info = AmazonInfo.Builder.newBuilder().autoBuild("eureka");
    config.setDataCenterInfo(info);
    info.getMetadata().put(AmazonInfo.MetaDataKey.publicHostname.getName(), info.get(AmazonInfo.MetaDataKey.publicIpv4));
    config.setHostname(info.get(AmazonInfo.MetaDataKey.publicHostname));
    config.setIpAddress(info.get(AmazonInfo.MetaDataKey.publicIpv4));
    config.setNonSecurePort(port);
    return config;
}

EurekaInstanceConfigBean is private in that package .. not sure how this would be implemented.