Azure-sdk-for-java: [BUG] Azure Keyvault Starter for Spring Boot no longer works with 2.4

Created on 17 Nov 2020 · 13Comments · Source: Azure/azure-sdk-for-java

Describe the bug
When using Spring Boot 2.4 and Azure Keyvault Starter to grab application properties on launch - it fails to startup and hangs at the point where the keyvault requests start.

Exception or Stack Trace

To Reproduce

Download a 2.4 Spring Boot app at start.spring.io (I chose Kotlin and Gradle)
Add necessary dependencies

implementation(platform("com.microsoft.azure:azure-spring-boot-bom:2.3.5"))
implementation(group = "com.microsoft.azure", name = "azure-keyvault-secrets-spring-boot-starter")

add information to your application-default.properties

azure.keyvault.uri = https://***.vault.azure.net/
azure.keyvault.client-id = ***
azure.keyvault.client-key = ***
azure.keyvault.tenant-id = ***

(try to) start the application

Expected behavior
Application starts up correctly.

Setup (please complete the following information):

OS: macOS
IDE : IntelliJ
Version of the Library used: see above

Additional context
This has worked fine with e.g. 2.3.

Following up the discussion in https://github.com/Azure/azure-sdk-for-java/issues/14909#issuecomment-726736141

Client azure-spring azure-spring-keyvault bug customer-reported

Source

jonasbark

👍1

Most helpful comment

Hi @jonasbark , we are now planning for a release of Azure Spring Boot to be compatable with Spring Boot 2.4.0, but there is a jar conflicting problem between Spring boot 2.4.0 and current Azure sdk libraries and we are discussing with other service team members. So the release timeline is uncertain yet. We will try to solve it as soon as possible.

yiliuTo on 10 Dec 2020

👍3

All 13 comments

@vcolin7 sorry to bother - but are there any news yet? Can I do more to help?

jonasbark on 25 Nov 2020

Hi @jonasbark, I haven't had the chance to look into this lately, I will spend some tie on it tomorrow and will let you know if I need more info. Thanks for the interest :)

vcolin7 on 1 Dec 2020

Hi @jonasbark , thanks for your reporting. I found that this is due to a dependency conflict between keyvault secret starter and spring boot 2.4.0. To quickly fix it, you can add another dependency of com.nimbusds:oauth2-oidc-sdk;7.1.1. We will also fix it in our code, thanks again.

yiliuTo on 2 Dec 2020

👍2

@jonasbark if what @yiliuTo suggested fixed your problem feel free to just close this issue :)

vcolin7 on 9 Dec 2020

@jonasbark if what @yiliuTo suggested fixed your problem feel free to just close this issue :)

I can now use it again but I would not consider this issue fixed unless a new version is provided by you. I still wonder if we're the only company trying to use the library with spring boot 2.4, seeing that this issue hadn't got much traction.

Can we expect a proper fix soon?

jonasbark on 9 Dec 2020

You are right @jonasbark, we should close this until a fix is in the codebase. I think @yiliuTo would be better suited to provide an answer about the time frame.

vcolin7 on 10 Dec 2020

yiliuTo on 10 Dec 2020

👍3

@jonasbark - I see this issue too. Trying to access few sensitive properties from keyvault as property source during initialization of Spring boot application. It hangs and doesn't move forward. Java version 1.8 Spring boot- 2.4.1, MacOS Catalina, IntelliJ. This only happens when i use azure-keyvault-secrets-spring-boot-starter. Have not got time to dig into because used a work around suggested by someone in this thread.

sarveshkaushal on 19 Dec 2020

Due to dependency compatibility issue, Azure Spring KeyVault integration simply going in infinite loop. Some of the code is swallowing exceptions.

@yiliuTo .. Please consider it as an enhancement before releasing next version.
It would be great to throw out exceptions so that users will know about it.

I had to debug code to extract this stack trace. Hope it will help you in fixing it.

java.lang.NoSuchMethodError: com.nimbusds.jwt.JWTClaimsSet.toJSONObject()Lnet/minidev/json/JSONObject;
    at com.nimbusds.oauth2.sdk.auth.JWTAuthenticationClaimsSet.parse(JWTAuthenticationClaimsSet.java:166)
    at com.nimbusds.oauth2.sdk.auth.JWTAuthentication.<init>(JWTAuthentication.java:139)
    at com.nimbusds.oauth2.sdk.auth.PrivateKeyJWT.<init>(PrivateKeyJWT.java:251)
    at com.nimbusds.oauth2.sdk.auth.PrivateKeyJWT.parse(PrivateKeyJWT.java:286)
    at com.microsoft.aad.msal4j.ConfidentialClientApplication.createClientAuthFromClientAssertion(ConfidentialClientApplication.java:116)
    at com.microsoft.aad.msal4j.ConfidentialClientApplication.buildValidClientCertificateAuthority(ConfidentialClientApplication.java:107)
    at com.microsoft.aad.msal4j.ConfidentialClientApplication.initClientAuthentication(ConfidentialClientApplication.java:81)
    at com.microsoft.aad.msal4j.ConfidentialClientApplication.<init>(ConfidentialClientApplication.java:68)
    at com.microsoft.aad.msal4j.ConfidentialClientApplication.<init>(ConfidentialClientApplication.java:30)
    at com.microsoft.aad.msal4j.ConfidentialClientApplication$Builder.build(ConfidentialClientApplication.java:147)
    at com.azure.identity.implementation.IdentityClient.getConfidentialClientApplication(IdentityClient.java:238)
    at com.azure.identity.implementation.IdentityClient.lambda$new$1(IdentityClient.java:158)
    at com.azure.identity.implementation.SynchronizedAccessor.lambda$getValue$0(SynchronizedAccessor.java:42)
    at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:44)
    at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
    at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:127)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:127)
    at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210)
    at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1789)
    at reactor.core.publisher.MonoSingle$SingleSubscriber.onComplete(MonoSingle.java:185)
    at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2361)
    at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onSubscribeInner(MonoFlatMapMany.java:150)
    at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:189)
    at reactor.core.publisher.SerializedSubscriber.onNext(SerializedSubscriber.java:99)
    at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onNext(FluxRetryWhen.java:173)
    at reactor.core.publisher.MonoCreate$DefaultMonoSink.success(MonoCreate.java:160)
    at reactor.netty.http.client.HttpClientConnect$HttpIOHandlerObserver.onStateChange(HttpClientConnect.java:389)
    at reactor.netty.ReactorNetty$CompositeConnectionObserver.onStateChange(ReactorNetty.java:638)
    at reactor.netty.resources.DefaultPooledConnectionProvider$DisposableAcquire.onStateChange(DefaultPooledConnectionProvider.java:195)
    at reactor.netty.resources.DefaultPooledConnectionProvider$PooledConnection.onStateChange(DefaultPooledConnectionProvider.java:466)
    at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:613)
    at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:94)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
    at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311)
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
    at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1533)
    at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1294)
    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1331)
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:508)
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:447)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
    at io.netty.channel.kqueue.AbstractKQueueStreamChannel$KQueueStreamUnsafe.readReady(AbstractKQueueStreamChannel.java:544)
    at io.netty.channel.kqueue.AbstractKQueueChannel$AbstractKQueueUnsafe.readReady(AbstractKQueueChannel.java:382)
    at io.netty.channel.kqueue.KQueueEventLoop.processReady(KQueueEventLoop.java:211)
    at io.netty.channel.kqueue.KQueueEventLoop.run(KQueueEventLoop.java:289)
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.lang.Thread.run(Thread.java:748)
java.lang.NoSuchMethodError: com.nimbusds.jwt.JWTClaimsSet.toJSONObject()Lnet/minidev/json/JSONObject;
    at com.nimbusds.oauth2.sdk.auth.JWTAuthenticationClaimsSet.parse(JWTAuthenticationClaimsSet.java:166)
    at com.nimbusds.oauth2.sdk.auth.JWTAuthentication.<init>(JWTAuthentication.java:139)
    at com.nimbusds.oauth2.sdk.auth.PrivateKeyJWT.<init>(PrivateKeyJWT.java:251)
    at com.nimbusds.oauth2.sdk.auth.PrivateKeyJWT.parse(PrivateKeyJWT.java:286)
    at com.microsoft.aad.msal4j.ConfidentialClientApplication.createClientAuthFromClientAssertion(ConfidentialClientApplication.java:116)
    at com.microsoft.aad.msal4j.ConfidentialClientApplication.buildValidClientCertificateAuthority(ConfidentialClientApplication.java:107)
    at com.microsoft.aad.msal4j.ConfidentialClientApplication.initClientAuthentication(ConfidentialClientApplication.java:81)
    at com.microsoft.aad.msal4j.ConfidentialClientApplication.<init>(ConfidentialClientApplication.java:68)
    at com.microsoft.aad.msal4j.ConfidentialClientApplication.<init>(ConfidentialClientApplication.java:30)
    at com.microsoft.aad.msal4j.ConfidentialClientApplication$Builder.build(ConfidentialClientApplication.java:147)
    at com.azure.identity.implementation.IdentityClient.getConfidentialClientApplication(IdentityClient.java:238)
    at com.azure.identity.implementation.IdentityClient.lambda$new$1(IdentityClient.java:158)
    at com.azure.identity.implementation.SynchronizedAccessor.lambda$getValue$0(SynchronizedAccessor.java:42)
    at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:44)
    at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
    at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:127)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:127)
    at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210)
    at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1789)
    at reactor.core.publisher.MonoSingle$SingleSubscriber.onComplete(MonoSingle.java:185)
    at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2361)
    at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onSubscribeInner(MonoFlatMapMany.java:150)
    at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:189)
    at reactor.core.publisher.SerializedSubscriber.onNext(SerializedSubscriber.java:99)
    at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onNext(FluxRetryWhen.java:173)
    at reactor.core.publisher.MonoCreate$DefaultMonoSink.success(MonoCreate.java:160)
    at reactor.netty.http.client.HttpClientConnect$HttpIOHandlerObserver.onStateChange(HttpClientConnect.java:389)
    at reactor.netty.ReactorNetty$CompositeConnectionObserver.onStateChange(ReactorNetty.java:638)
    at reactor.netty.resources.DefaultPooledConnectionProvider$DisposableAcquire.onStateChange(DefaultPooledConnectionProvider.java:195)
    at reactor.netty.resources.DefaultPooledConnectionProvider$PooledConnection.onStateChange(DefaultPooledConnectionProvider.java:466)
    at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:613)
    at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:94)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
    at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311)
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
    at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1533)
    at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1294)
    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1331)
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:508)
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:447)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
    at io.netty.channel.kqueue.AbstractKQueueStreamChannel$KQueueStreamUnsafe.readReady(AbstractKQueueStreamChannel.java:544)
    at io.netty.channel.kqueue.AbstractKQueueChannel$AbstractKQueueUnsafe.readReady(AbstractKQueueChannel.java:382)
    at io.netty.channel.kqueue.KQueueEventLoop.processReady(KQueueEventLoop.java:211)
    at io.netty.channel.kqueue.KQueueEventLoop.run(KQueueEventLoop.java:289)
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.lang.Thread.run(Thread.java:748)

anumalasri on 30 Jan 2021

@anumalasri , thanks for your reporting, and this dependency conflict has been fixed by https://github.com/Azure/azure-sdk-for-java/pull/18817. The keyvault starter to be released can work with spring boot 2.4.0~

yiliuTo on 1 Feb 2021

Hi @jonasbark and @anumalasri , the Azure Keyvault Secret Starter for version 3.3.0 has been released, which is compatable with Spring Boot 2.4. Here is the Maven repo link: https://mvnrepository.com/artifact/com.azure.spring/azure-spring-boot-starter-keyvault-secrets/3.3.0

yiliuTo on 7 Apr 2021

@yiliuTo .. On brief testing with new dependency version (Spring Boot 2.4.4 and Azure KV Starter v3.3.0), The application is working expected.

However, Looks like, the way KeyStore load and process certificates got updated when compared to previous version and new version. We got below error and able to resolve by exporting only Private Key. May be it would be to document that certificate need to be modified when users get upgraded to new version.

Caused by: java.lang.IllegalArgumentException: more than one certificate alias found in input stream
    at com.microsoft.aad.msal4j.ClientCertificate.create(ClientCertificate.java:80)
    at com.microsoft.aad.msal4j.ClientCredentialFactory.createFromCertificate(ClientCredentialFactory.java:45)

anumalasri on 7 Apr 2021

@anumalasri thanks for your reporting, we will look into it and update it soon.

yiliuTo on 9 Apr 2021

Was this page helpful?

0 / 5 - 0 ratings

Related issues

[FEATURE REQ] Document which azure-identity dependencies are optional

knutwannheden · 4Comments

[QUERY] Does Azure Java SDK support FIPS

dkirrane · 4Comments

[BUG] getByResourceGroup throws an NPE when the resource is a Gallery, GalleryImage or GalleryImageVersion

Shanky2304 · 4Comments

Azure Blob - listBlobsByHierarchy().iterableByPage(10) - not providing results based on defined page size

abhikt48 · 3Comments

ShareFileClient returns 404 when retrieves the properties when file name contains umlaut characters

valmol · 4Comments