Spring-boot-admin: How to notify and automatically remove offline instances?

[hotstepper13]

Our setup is SBA 1.5.3 with enabled hipchat notification.

If an application change state to UP/DOWN/OFFLINE we got a message in Hipchat.
Because the identifiers for applications are always changes (kubernetes environment) an offline application will never come back again with the same identifier. Therefore we would like to remove the offline applications automatically from SBA.

I have tried setting spring.boot.admin.auto-deregistration in the clients, which works well. Instead of removing the OFFLINE application it never reaches the state because it deregisters actively.

But the deregistration did not trigger a notification in hipchat.

So, question 1:
Is there a way to configure in SBA in order to automatically remove offline instances (after a period of time)?

Question 2:
How can we ensure to enable notification for deregistration?

Thanks in advance.

[joshiste]

q1: SBA does not include such a functionality. You could write a scheduled tasks that removes offline instances from the application registry.

q2: the notifiers are just reacting on a status change. Deregistration is not a status change, but you can extend the notifier and override the shouldNotify method to act on a deregistration event.

[RemiBou]

@hotstepper13 @joshiste
I got the same problem running a docker swarm. I got the following solution (inspired by InfoUpdateTrigger) :

public class DeregisterOfflineTrigger extends AbstractEventHandler<InstanceEvent> {
    private final IntervalCheck intervalCheck;

    private InstanceRegistry instanceRegistry;

    public DeregisterOfflineTrigger(Publisher<InstanceEvent> publisher, InstanceRegistry instanceRegistry) {
        super(publisher, InstanceEvent.class);
        this.instanceRegistry = instanceRegistry;
        this.intervalCheck =
            new IntervalCheck("deregister", this::updateInfo, Duration.ofMinutes(5), Duration.ofMinutes(1));
    }

    @Override
    protected Publisher<Void> handle(Flux<InstanceEvent> publisher) {

        Scheduler scheduler = Schedulers.newSingle("offline-deregister");
        return publisher.subscribeOn(scheduler).filter(
            event -> event instanceof InstanceStatusChangedEvent && ((InstanceStatusChangedEvent) event)
                .getStatusInfo().isOffline()).flatMap(event -> this.updateInfo(event.getInstance()))
            .doFinally(s -> scheduler.dispose());
    }

    private Mono<Void> updateInfo(InstanceId instance) {

        return instanceRegistry.deregister(instance).then();

    }

    @Override
    public void start() {
        super.start();
        this.intervalCheck.start();
    }

    @Override
    public void stop() {
        super.stop();
        this.intervalCheck.stop();
    }

    public void setInterval(Duration updateInterval) {
        this.intervalCheck.setInterval(updateInterval);
    }

    public void setLifetime(Duration infoLifetime) {
        this.intervalCheck.setMinRetention(infoLifetime);
    }
}

And the registration of the bean

@Configuration
public class DeregisterOfflineConfiguration {

    @Bean(initMethod = "start", destroyMethod = "stop")
    @ConditionalOnMissingBean
    public DeregisterOfflineTrigger deregisterOfflineTrigger(InstanceRegistry registry,
        Publisher<InstanceEvent> events) {
        DeregisterOfflineTrigger trigger = new DeregisterOfflineTrigger(events, registry);
        trigger.setInterval(Duration.ofMinutes(1));
        trigger.setLifetime(Duration.ofMinutes(1));
        return trigger;
    }
}

So far it seems to be working :) I still have the offline notifications on Slack but these instance are not anymore displayed on the UI.