Azure-sdk-for-java: The lease mechanism leads to loss of transferred data

[rkowalski4]

The lease mechanism leads to loss of transferred data

I am working on android application which transfer data to azure storage. I successfully implemented it in Kotlin in following way.

val containerClient: BlobContainerClient = getBlobContainerClient(resourceDirectory)

if (!containerClient.exists()) {
    containerClient.create()
}

containerClient.getBlobClient(resourceName).uploadFromFile(resourcePath)

Then I had to implement lease mechanism. I used BlobLeaseClientBuilder to build
LeaseClient and to acquire lease. Code below.

val containerClient: BlobContainerClient = getBlobContainerClient(resourceDirectory)

if (!containerClient.exists()) {
    containerClient.create()
}

val client: BlobClient = containerClient.getBlobClient(resourceName)

val leaseClient: BlobLeaseClient = BlobLeaseClientBuilder()
    .blobClient(client) 
    // .leaseId(UUID.randomUUID().toString()) - also tested
    .buildClient()

val acquiredLeaseId: String = leaseClient.acquireLease(-1)

client.uploadFromFile(
    resourcePath, //filePath
    null, // parallelTransferOptions
    null, // headers
    null, // metadata
    null, // tier
    BlobRequestConditions().apply { leaseId =  acquiredLeaseId }, // accessConditions
    null // timeout
)

leaseClient.releaseLease()

Uploading data in such way does not work. Only container is created correctly
(containerClient.create()) and azure seems to drop all request when lease is set
(because of incorrect leaseId?)

Environment:

• OS: development: Ubunto 18.04, tested on OnePlus 6T with Android 10
• IDE: Android Studio: 3.5.3
• Version of the library: group: 'com.azure', name: 'azure-storage-blob', version: '12.0.1'

[joshfree]

Thanks for opening this android issue on BlobLeaseClient, @rkowalski4. @rickle-msft and @anuchandy will be able to follow up with you shortly.

/cc @bsiegel as FYI

[rickle-msft]

@anuchandy I ran this code locally and it worked fine. I wonder if there's some combination of Kotlin and Android that might be interfering with the requests?

@rkowalski4 If you use some sort of network capture like Fiddler, are you able to see the uploadFromFile request actually go out and whether there is a lease id header present? Or does just nothing happen when you call this method?

[anuchandy]

thanks, Rick, I guess we need more info.

@rkowalski4 - like Rick mentioned, is it possible to check the network traffic?

1. also any chance you can provide us a minimal repro android project? if you can push that to your Github repo and share it here that will be helpful. Also, leave any external dependencies that the original app uses in this repro project.
2. What is the size of the file that we're trying to upload? Or that doesn't matter?

[rkowalski4]

Thanks guys for answering.

I found out what is wrong with this code. I'm trying to set lease on blob I want to upload. But when I setting up lease (val acquiredLeaseId: String = leaseClient.acquireLease(-1)), blob does not exist so Azure returns 404. So I'm afraid I cannot handle lease in such way for file I want to upload. Either way I need to set lease, because it is not permitted to do anything with files while upload is in progress especially that files bigger than 256MB are chunking (as I find out from the source code).

Because of that, I'm afraid, I can't set up lease for blob I want to upload. Any advice on how to handle this?

I'm not allowed to share a codebase with you, but I'll push some very basic project with this use case within a day If you still need it.

[rickle-msft]

@rkowalski4 oh yup that sounds like it'll do it. Glad you found the source of that issue!

I think there are two ways you could resolve this. You could put an empty blob at the destination and then acquire a lease on that before starting your upload. Then you can overwrite the empty blob with your upload (probably locking on the etag of the empty blob to make sure it's still empty when you overwrite). This would allow you to continue to use leases to protect your upload, but would require an extra request to the service in order to create that empty blob.

Alternatively, you can use optimistic concurrency instead and you can use access condition checks (such as last modified time or etags) to ensure you're updating your data in the way you expect. This would not require the extra call, but it can also result in some other service exceptions that you'll have to know to handle. For instance, writing to the same destination from two different sources can result in one invalidating the block list of the other.

[rickle-msft]

I am going to close this issue as I believe the question and issue have been resolved. Please let us know if you need further support.

[rkowalski4]

Hi, I'm sorry for lack of information. I fixed this out and everything works perfectly. Thank you so much for your support!