Spring-boot-admin: spring boot admin does not detect service after restarting spring boot admin

Created on 26 Feb 2021  路  7Comments  路  Source: codecentric/spring-boot-admin

spring boot 2.4.3
spring cloud 2020.0.1
spring boot admin 2.4.0

I have an environment with two services.
They are exchange-api-test and admin-test(is a spring-boot-admin project)
They all are deployed and managed by Docker images and Kubernetes.
I also use admin-test in order to monitor them all and spring-cloud-kubernetes to discover all the services automatically.

It works perfectly whenever I restart the exchange-api-test project (kubectl rollout restart deployment exchange-api-deployment), admin-test discovers every service.
image
My problem comes when I restart admin-test project(kubectl rollout restart deployment admin-deployment), exchange-api-test is shown as OFFLINE in the SBA and it does not change its status.
image
applaction.yml (in admin-test project)

server:
 port: 8080
spring:
 profiles:
   active: test
 application:
   name: admin-${spring.profiles.active}
 cloud:
   kubernetes:
     discovery:
       all-namespaces: true
       service-labels:
         env: ${spring.profiles.active}
     reload:
       enabled: true
management:
 endpoints:
   web:
     exposure:
       include: "*"
 endpoint:
   health:
     show-details: always
bug

Most helpful comment

We also hit this bug after upgrading to SBA 2.4.0 using Spring Cloud Kubernetes Discovery. I noticed that events got "lost" and as such, certain AbstractEventHandlers (like EndpointDetectionTrigger) were not triggered at all. Subsequently, services are reported as down or endpoints are not discovered.

I traced this back to InstanceEventPublisher, where after the Reactor Core upgrade to 3.4.x, the new sinks API was used. However, the new sinks API has one major difference with the previous API (UnicastProcessor), in the sense that serialization (i.e. concurrent access) is _not_ handled for you. Instead, the emission fails silently (adding orThrow() will reveal the errors).

I can fix it locally by either retrying the emission (using a EmitFailureHandler) or by adding synchronized to InstanceEventPublisher#publish. Happy to provide a PR for this if you want, @SteKoe.

All 7 comments

Hi @star4j, have you added @EnableScheduling to the main Application class of your Spring Boot Admin implementation (see #1506)?

See also:
https://cloud.spring.io/spring-cloud-kubernetes/1.1.x/reference/html/#secrets-propertysource

Hi @star4j, have you added @EnableScheduling to the main Application class of your Spring Boot Admin implementation (see #1506)?

See also:
https://cloud.spring.io/spring-cloud-kubernetes/1.1.x/reference/html/#secrets-propertysource

I have added @Enablescheduling
image

We are seeing a lot of our services appearing as 'applications.unknown' instead of 'up' after upgrading to Spring Boot Admin 2.4.0
(also on Kubernetes)

I tried spring boot admin 2.3.1, it is no problem

image

the same problem after upgrading to 2.4.0

We also hit this bug after upgrading to SBA 2.4.0 using Spring Cloud Kubernetes Discovery. I noticed that events got "lost" and as such, certain AbstractEventHandlers (like EndpointDetectionTrigger) were not triggered at all. Subsequently, services are reported as down or endpoints are not discovered.

I traced this back to InstanceEventPublisher, where after the Reactor Core upgrade to 3.4.x, the new sinks API was used. However, the new sinks API has one major difference with the previous API (UnicastProcessor), in the sense that serialization (i.e. concurrent access) is _not_ handled for you. Instead, the emission fails silently (adding orThrow() will reveal the errors).

I can fix it locally by either retrying the emission (using a EmitFailureHandler) or by adding synchronized to InstanceEventPublisher#publish. Happy to provide a PR for this if you want, @SteKoe.

Was this page helpful?
0 / 5 - 0 ratings