Spring-cloud-netflix: Eureka warning even when defaultOpenForTrafficCount is set to 0

[issam03]

EMERGENCY! EUREKA MAY BE INCORRECTLY CLAIMING INSTANCES ARE UP WHEN THEY'RE NOT. RENEWALS ARE LESSER THAN THRESHOLD AND HENCE THE INSTANCES ARE NOT BEING EXPIRED JUST TO BE SAFE

This message appears after certain time when the server is deployed using this configuration:

eureka.server.eviction-interval-timer-in-ms: 60000
eureka.client.registerWithEureka = false
eureka.client.fetchRegistry = false
eureka.instance.hostname = localhost
eureka.client.serviceUrl.defaultZone = http://${eureka.instance.hostname}:${server.port}/eureka/
eureka.server.defaultOpenForTrafficCount = 0

The same issue is reported in https://github.com/spring-cloud/spring-cloud-netflix/issues/1312 but this does not solve the problem.

When i use: eureka.server.defaultOpenForTrafficCount = 0 in the dashboard the Renews threshold start as 0 but:

• The Renews threshold is not incremented after the registration of a new service
• Im still having the error message

[spencergibb]

How many instances do you have?

[issam03]

We have only one instance

[spencergibb]

I think the math is hard for just one instance. If you are going into production with just one instance, eureka probably isn't needed.

[issam03]

We decided to go for only one instance for the moment, we need eureka so that services can talk with each other without specific ip adresses.
So with only one instance, there is no solution to avoid this warning ?

[spencergibb]

Eureka is overkill for one instance. You will need to set eureka.server.enableSelfPreservation=false, otherwise, if that instance ever needed to be replaced, it would never be replaced (hence the warning).

[ivanmir]

Even with the enableSelfPreservation=false and just one instance Eureka states the same message in red. perhaps the message text should be revised to something more clarifying. A red message tells me there is something wrong that needs to be fixed.

[spencergibb]

It is bad. enableSelfPreservation=false isn't for production. The warning above is bad.

[chefhoobajoob]

I have a single Eureka server instance, deployed into a Cloud Foundry environment that supports container networking, a minimum of 3 client instances (can grow dynamically based on demand), and I have the same problem as @issam03. I'm also using enableSelfPreservation=false (I think - see server configuration below).

All I want is for the expiration of instances when an instance is killed. Under my current configuration setup, this never happens, and the server is always displaying the BIG RED WARNING.

Here is the client-side configuration I'm using:

eureka.instance.leaseRenewalIntervalInSeconds=1
eureka.instance.leaseExpirationDurationInSeconds=2

Here is the minimal eureka server application.yml file I'm using:

eureka:
    client:
        serviceUrl:
           defaultZone: http://localhost:8761/eureka/

spring:
    application:
        name: eureka-server

...and I'm trying to control most server configuration strictly through environment variables. I'm not sure I have these all named correctly, but here is the Cloud Foundry application manifest I'm using:

applications:
- name: eureka-server
  instances: 1
  health-check-http-endpoint: /
  env:
    EUREKA_CLIENT_SERVICEURL_DEFAULTZONE: http://eureka-server.cf.lab.net/
    EUREKA_SERVER_ENABLESELFPRESERVATION: false
    SERVER_PORT: 8080
#    EUREKA_SERVER_DEFAULTOPENFORTRAFFICCOUNT: 0
    EUREKA_CLIENT_REGISTERWITHEUREKA: false
    EUREKA_CLIENT_FETCHREGISTRY: false

With this configuration, the server successfully deploys, but is always displaying the BIG RED WARNING as soon as the first instance registers, and it's always showing Lease expiration enabled as false. If I kill off one of the service instances, the number of instances on the eureka server doesn't change.

Have I named the environment variables correctly for the server configuration? Is there anything else I can try? Setting defaultOpenForTrafficCount=0 gets rid of the warning, but instances are still never expired.

[spencergibb]

There will be a big red warning, either that you are below the safe threshold, or that enable self-preservation is disabled.

Check /configprops to see that self preservation is actually set.

[chefhoobajoob]

I can't seem to get a response from /configprops. Do I need to build the server a specific way to add support for that?

These are the gradle dependencies:

dependencies {
    compile 'org.springframework.boot:spring-boot-starter-web:1.5.7.RELEASE'
    compile 'org.springframework.cloud:spring-cloud-starter-eureka-server'

    testCompile 'junit:junit:4.12'
}

This is the Application class:

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.discovery.EnableDiscoveryClient;
import org.springframework.cloud.netflix.eureka.server.EnableEurekaServer;

@EnableDiscoveryClient
@EnableEurekaServer
@SpringBootApplication
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

}

[spencergibb]

please read spring boots docs on actuator.

[chefhoobajoob]

I think there must be a conspiracy against me learning how the server is configured.

I added this dependency:

    compile "org.springframework.boot:spring-boot-starter-actuator"

...and added this section to Application.yml:

management:
  endpoints:
    web:
      exposure:
        include: "info, health, configprops"

...and after deploying the server, only the /info and/health url's return a response. /configprops acts like there's no such route. Did I miss something? This was my reference.

[spencergibb]

No need for quotes. I usually just do '*' (which does need quotes).

[chefhoobajoob]

Yes - saw that and currently trying it...

[chefhoobajoob]

Dangit - same results with '*'. What does configprops actually report? Does it depend on something I've omitted?

[spencergibb]

It says what actually got bound to configuration properties objects. The same problem we are trying to solve.

[chefhoobajoob]

Crikey. I'm at a loss for how to proceed with troubleshooting. Can I poke the server into dumping this info some other way, like via service logs or something?

[spencergibb]

I don't know either

[chefhoobajoob]

Hang on - this might be a clue:
https://stackoverflow.com/a/48150600/1299008

That is the response I'm getting when I hit /configprops:

Whitelabel Error Page
This application has no explicit mapping for /error, so you are seeing this as a fallback.

Thu Jun 14 00:43:24 UTC 2018
There was an unexpected error (type=Unauthorized, status=401).
Full authentication is required to access this resource.

[spencergibb]

you need to login or disable security for actuator.

[chefhoobajoob]

yes - trying with security disabled

[chefhoobajoob]

Success!

So at eurekaClientConfigBean.properties.propertyResolver.systemEnvironment I see this:

         "EUREKA_SERVER_ENABLESELFPRESERVATION": "false",

...at eurekaClientConfigBean.properties.propertyResolver.systemProperties I see this:

      "enableSelfPreservation": true,

...and at eurekaServerConfig.properties I see this:

      "enableSelfPreservation": true,

...so something about the environment variable setting isn't working. What should I change?

For now, I'll try and make all the config changes in Application.yml to see if I can get a config that works. If I find one, I'd like to figure out how to get the environment variable config to work, though obviously that's a separate issue.

[chefhoobajoob]

OK - I think my basic requirements are met with these settings and a single eureka server instance:

eureka:
    client:
      serviceUrl:
         defaultZone: http://localhost:8761/eureka/
      registerWithEureka: false
      fetchRegistry: false
    server:
      enableSelfPreservation: false

I do see the behavior reported in the original issue where the renews threshold does not get updated after clients register if defaultOpenForTrafficCount=0. The threshold stays at 0.

However, the BIG RED WARNING does _not_ appear in my deployment with 3 registered instances.

It also does not expire instances when services shutdown and are no longer renewing their leases.

Now, all I need is to figure out how to set these configuration properties using environment variables. Any suggestions here would be welcomed.