Azure-sdk-for-java: [QUERY] How to integration test an app using SecretClient

[chris-j-h]

Query/Question
Are there any dev or in-memory/local implementations of Key Vault that can be deployed to enable integration testing of an app using SecretClient, or perhaps other techniques that don't require knowledge of the underlying Key Vault APIs?

Further context
I would like to add integration tests to my code which uses SecretClient::getSecret.

I don't want to create a live Key Vault resource and have to consider the pricing that will incur (particularly when CI could run the tests frequently).

I have identified the following options which do not seem ideal:

• Mock a SecretClient delegate to return a known KeyVaultSecret
  
  
  
  • This is fine for a unit test but doesn't test if the built SecretClient can actually communicate with a Key Vault (e.g. if vault URL and auth credentials are applied correctly)
  
  
• Use com.azure:azure-core-test in PLAYBACK mode to mock the HTTP responses from Key Vault
  
  
  
  • This requires me to have in-depth knowledge of the expected HTTP requests and responses for Key Vault, as well as Azure's challenge-based auth handshake process
  
  
  • This does allow me to verify the SecretClient is being built with the correct vault URL and user-provided auth credentials
  
  
  • But this adds a level of detail to my tests that will make them susceptible to changes to the Key Vault or challenge-based auth APIs
  
  

Ideally, I would create some form of test Key Vault, populate it with secrets, and make sure that my app can retrieve them without having to use live infra or have a dependency on the underlying implementation of SecretClient.

Why is this not a Bug or a feature Request?
User query, not an issue report.

Setup (please complete the following information if applicable):
com.azure:azure-security-keyvault-secrets:4.2.4

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• [x] Query Added
• [x] Setup information Added

[joshfree]

@vcolin7 could you please follow up with @chris-j-h on this inquiry?

[jainswati02]

I am in a similar situation, would like to know if we have any option to use dev or in-memory/local implementations of Key Vault for integration tests.

[madkroll]

Same here. SDK works perfectly, but can't find any test tooling for integration with Azure KeyVault.

[chris-j-h]

Hi @vcolin7 @joshfree any update on this? Seems like there's a fair amount of interest in this area.

[AlexGhiondea]

@jainswati02 @madkroll @chris-j-h looking at this it seems like you are trying to find a way to test that the SDK will return a key from KeyVault. The SDK is already validating those things in our test infrastructure using the azure-core-test framework that our team uses.

We recommend mocking (the way you described above) as a way to unit test your application. For validating the Key Vault communication you could either do live testing or you could use an existing HTTP recording framework to record the communication that your application is doing and then use that in your own tests.

I am not aware of any mechanism to spin up an in-memory Key Vault that you can use in your tests.

Please let me know if I'm misunderstanding your ask.

[vcolin7]

@jainswati02 @madkroll @chris-j-h One way to achieve this without much trouble and having to know a lot about the Key Vault service's requests and responses is to write your tests to include a recording and playback policy in your client's HTTP pipeline, just like we do to develop unit tests for the Key Vault SDK. You would need to leverage the azure-core-test library:

<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-core-test</artifactId>
    <version>1.5.3</version>
</dependency>

Here is some sample code for a very simple test:

import com.azure.core.http.rest.PagedIterable;
import com.azure.core.test.TestBase;
import com.azure.identity.DefaultAzureCredentialBuilder;
import com.azure.security.keyvault.keys.KeyClient;
import com.azure.security.keyvault.keys.KeyClientBuilder;
import com.azure.security.keyvault.keys.models.KeyProperties;
import org.junit.jupiter.api.Test;

public class RecordTest extends TestBase {
    @Test
    public void listKeysTest(){
        KeyClient keyClient = new KeyClientBuilder()
            .credential(new DefaultAzureCredentialBuilder().build())
            .vaultUrl("https://<your-key-vault-name>.vault.azure.net/")
            .addPolicy(interceptorManager.getRecordPolicy()) // Here we add the recording/playback policy
            .buildClient();
        PagedIterable<KeyProperties> propertiesOfKeys = keyClient.listPropertiesOfKeys();
        assert propertiesOfKeys.stream().count() != 0;
    }
}

To record the test results, you would have to run the test against the service at least once by setting the environment variable AZURE_TEST_MODE=RECORD, but after that you can simply change it to AZURE_TEST_MODE=PLAYBACK or remove it entirely, as that's the default value for said variable. Additionally, there is one more test mode (LIVE) for when you want the framework not to record anything and just run against the service.

[chris-j-h]

@AlexGhiondea @vcolin7 thanks for the info about the recording framework. Just to be clear, this requires connecting to a real key vault service at least once to record the expected result, but from that point on the recording can be used in place of the real service?

If so that sounds like it could be what we're looking for, thanks.

[AlexGhiondea]

@chris-j-h that is correct. Please let us know if there is anything else we can help with!

[msftbot[bot]]

Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

[msftbot[bot]]

Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

[chris-j-h]

Closing issue - all queries so far have been answered.