Spring-boot-admin: Support X-CF-APP-INSTANCE http header for Pivotal Cloud Foundry

[VincSch]

Hey folks,
were using Spring Boot Admin combined with Eureka deployed in Cloud Foundry.
If we scale an application Eureka picks up all of your running instances identified by there instance id (hash value). Spring Boot also discovers those hosts, but it can't reach them. In order to make that working I have to configure the hostname and securePort in the Eureka Client settings, because the app has only a single route and is reachable via port 80. The problem now is that it will not detect multiple instances of the same app (cause its using the single route and the go router is doing RR).

Cloud Foundry supports the 'X-CF-APP-INSTANCE' header which can be send in the request by using the app 'GUID:InstanceNumber' (see https://docs.cloudfoundry.org/devguide/deploy-apps/routes-domains.html).

Is it possible to configure something like that today? If not can this feature be included?
At the moment we will try to implement this feature on our own. We're just trying to get a feeling about the complexity.

[joshiste]

Hmm... how the sba knows which instances are available and what value to put in the corresponding header?

[joshiste]

Currently there is no feature like that (as we don't use cf currently). But if you do a PR on it I'm keen on merging it.

[ankurkaps]

@joshiste We are also using sba in cloud foundry environment and ran into same problem.

One way possibly could be for the admin client to pass app guid & instance number as additional metadata to SBA during client registration as these are readily available in the environment of each instance, which could be stored and used during making a request back to the clients.

Do you think this is something that is feasible ?

[joshiste]

You can achive that using the current SBA version by following some easy steps:

1. Configure the clients to submit their cf-app-instance id in the metadata
2. Add your own implementation of the ApplicationIdGenerator to the server. It should take the id from the metadata into account
3. Add your own implementation of the HttpHeadersProvider to the server which sets theX-CF-AP-INSTANCE header

[ankurkaps]

Thanks, will try this out.

[ankurkaps]

@joshiste Tried the approach using custom application id generator and header provider. It work quite well SBA 1.5.4.

[joshiste]

Nice to hear about! Would you like to share the implementation and a small guide for the docs via PR?

[joshiste]

@ankurkaps wanna share your implementation for this?

[ankurkaps]

@joshiste Sure. Since, I did this for work, I need to get this cleared before I can create a PR. I am in the process of getting this done. Will create a PR after that.

[iferca]

@ankurkaps any progress with the PR, I'm needing the same thing and it would be great if I could take advantage of your progress instead of starting it form scratch, let me know if I can help

[ankurkaps]

@iferca Great to hear that, I should have something shortly.

[shakuzen]

We're also interested in this support and are willing to help. As far as I understand, without this support, there is no way for the instance status in SBA to be accurate in a Cloud Foundry environment since it isn't necessarily being routed to the instance it thinks it is (unless there's only one instance, of course).

[CAFxX]

To help with this we also requested to include the number of running instances in the CF HTTP responses: https://github.com/cloudfoundry/gorouter/issues/197

[tetsushiawano]

We're also interested in this support and now planning to support this by the end of the week. So if @ankurkaps will take more time as is ok with this, we can prepare a PR instead.

[hauerio]

We are interested, as well. So I wonder what is the status? Is any help needed?

[tetsushiawano]

I'm now coding right now. Once finished, I will create a PR. hopefully today.

[VincSch]

Hey guys, we managed to implement this future without any changes to SBA core. We wanted to share this approach with you which only includes writing two beans and setting up the Eureka client in order to pass the right PCF instance id! Thanks to @patrickmarschallek!

Here we go:

Provide custom header.

import de.codecentric.boot.admin.model.Application;
import de.codecentric.boot.admin.web.client.BasicAuthHttpHeaderProvider;
import org.springframework.http.HttpHeaders;
import org.springframework.http.MediaType;
import org.springframework.util.StringUtils;

import java.util.Collections;

/**
 * @author [email protected]
 *
 * Provides a custom header for Cloud Foundry which identifies which instance to call.
 * The go-router in CloudFoundry will forward the request to the instance with the
 * according id in the header.
 */
public class CloudFoundryInstanceIdHttpHeaderProvider extends BasicAuthHttpHeaderProvider {

    private static final String CF_APP_INSTANCE_HEADER = "X-CF-APP-INSTANCE";

    @Override
    public HttpHeaders getHeaders(Application application) {
        HttpHeaders headers = super.getHeaders(application);
        String appInstanceId = application.getMetadata().get("instanceId");
        if (StringUtils.hasText(appInstanceId)) {
            headers.set(CF_APP_INSTANCE_HEADER, appInstanceId);
        }
        return headers;
    }

}

Generate Hash based on URL.

import de.codecentric.boot.admin.model.Application;
import de.codecentric.boot.admin.registry.ApplicationIdGenerator;

import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;

/**
 * Generates an SHA-1 Hash based on the applications url.
 */
public class CloudFoundryApplicationUrlIdGenerator implements ApplicationIdGenerator {

    private static final char[] HEX_CHARS = {
            '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c', 'd', 'e', 'f'
    };

    @Override
    public String generateId(Application a) {
        try {
            MessageDigest digest = MessageDigest.getInstance("SHA-1");
            String unique = a.getMetadata().getOrDefault("instanceId","") + a.getHealthUrl();
            byte[] bytes = digest.digest(unique.getBytes(StandardCharsets.UTF_8));
            return new String(encodeHex(bytes, 0, 8));
        } catch (NoSuchAlgorithmException e) {
            throw new IllegalStateException(e);
        }
    }

    private char[] encodeHex(byte[] bytes, int offset, int length) {
        char chars[] = new char[length];
        for (int i = 0; i < length; i = i + 2) {
            byte b = bytes[offset + (i / 2)];
            chars[i] = HEX_CHARS[(b >>> 0x4) & 0xf];
            chars[i + 1] = HEX_CHARS[b & 0xf];
        }
        return chars;
    }
}

Expose the beans above. We also made it configurable by using ConditionalOnProperty.
If SBA gets deployed in a different environment (eg Kubernetes) the PCF feature can be turned off using properties.

@Configuration
@SpringBootApplication
@EnableAdminServer
@EnableDiscoveryClient
public class SpringBootAdmin {

    public static void main(String[] args) {
        SpringApplication.run(SpringBootAdmin.class, args);
    }

    @Bean
    @ConditionalOnProperty("cloud.foundry.enabled")
    public HttpHeadersProvider cfhttpHeadersProvider() {
        return new CloudFoundryInstanceIdHttpHeaderProvider();
    }

    @Bean
    @ConditionalOnProperty("cloud.foundry.enabled")
    public ApplicationIdGenerator applicationIdGenerator() {
        return new CloudFoundryApplicationUrlIdGenerator();
    }
}

Configure Eureka client:

# ----------------------------------------
# EUREKA PROPERTIES
# ----------------------------------------
eureka:
  instance:
    leaseRenewalIntervalInSeconds: 10
    hostname: ${vcap.application.uris[0]}
    nonSecurePort: 80
    instanceId: ${vcap.application.application_id}:${vcap.application.instance_index}

And thats it! This fixed the issue for us without touching the SBA code at all.
Have a great one.

Cheers,
@VincSch & @PatrickMarschallek

[patrickmarschallek]

Thanks @VincSch that you did this for me. I couldn't find the time to properly write it down here.
cheers
Patrick

[tetsushiawano]

Sorry, I'm stuck right now with spring-boot-admin-client to automatically set the following 4 lines and cant make PR today.
https://github.com/tetsushiawano/spring-boot-admin-client-app/blob/master/src/main/resources/application.yml#L5-L8

Can someone help me solve it?
I created the EnvironmentPostProcessor to achieve this but I'm current struggling to solve the issue of@Value annotation is not working and the value is not passed to the variable.

https://github.com/tetsushiawano/spring-boot-admin/blob/feature/support_cf_v2/spring-boot-admin-client/src/main/java/de/codecentric/boot/admin/client/config/CloudFoundryEnvironmentPostProcessor.java#L34-L41

The discussion about the implementation is described below.
https://github.com/codecentric/spring-boot-admin/pull/650

Thank you in advance.

[joshiste]

Hmm I guess you can't use the @Value because, there is no BeanPostProcessor which resolves this annotation (as the environment is still in instantiation). Get rid of the three fields and extract the values inside the postProcessEnvironment() from the environment parameter.

[tetsushiawano]

Thanks for advice! I will try it today!