Azure-cosmos-dotnet-v2: .NET Standard support for Table API

[SeanFeldman]

CosmosDB Table API is promoted as the replacement for Table Storage.
Table Storage has support for .NET Standard. This library doesn't.
Almost a year ago, it was mentioned that work on .NET Standard 1.6 is in progress and should be shortly released.
As of today, the latest version of the CosmosDB .NET client does not support .NET Standard.

This impedes migration plans for those that have to support .NET Standard.
Please provide an update and/or a milestone.

Update: latest client package name has changed to Microsoft.Azure.CosmosDB.Table

[kirankumarkolli]

@SeanFeldman which API are you consuming (Table API/document API)?

[SeanFeldman]

Table API @kirankumarkolli

[kirankumarkolli]

@SeanFeldman table API standard version is tracked for the end of year.

[SeanFeldman]

Thank you @kirankumarkolli

Not to be a PITA, but could you please either assign some milestone to this issue (I've renamed it to be more accurate) or raise another one? Even a label like table-api would help. Similar to what you've done for the graph API here. Thank you.

[kirankumarkolli]

Labeled it as Table.

[rnagpal]

@SeanFeldman DocumentDB Client SDK has a .NET Standard version which we released last year https://docs.microsoft.com/en-us/azure/cosmos-db/documentdb-sdk-dotnet-core As Kiran mentioned, we would be releasing .NET Standard for Premium Table SDK in the near future.

[SeanFeldman]

Thank you @rnagpal. I was specifically asking about Tables API 🙂
Will track this issue for progresses.

[SeanFeldman]

Microsoft.Azure.CosmosDB.Table client has GA-ed. Would be nice to see status/progress update on this issue. Thanks.

[yaronyg]

Microsoft has two different table offerings, Azure Table storage and Azure Cosmos DB Table API. They each target different audiences but use the same SDK. You can get a clear idea of how to choose which service by looking at https://docs.microsoft.com/en-us/azure/cosmos-db/table-introduction. I'm also tracking down the .net standard issue now with @kirankumarkolli

[SeanFeldman]

Thank you. @yaronyg
Just to be clear - this issue is about .NET Standard support for Azure Cosmos DB Table API.
Any mentioning of the Azure Storage tables is irrelevant 🙂

[yaronyg]

@SeanFeldman we don't currently support .net standard for Tables API but we do intend to eventually fix it.

[SeanFeldman]

@yaronyg any idea on the bulk part when to expect it?

[yaronyg]

@SeanFeldman not sure what you mean by bulk part?

[SeanFeldman]

@yaronyg ETA w/o overcommitting

[yaronyg]

@SeanFeldman Sorry for the slippage, but currently this is planned for the first 1/2 of 2018.

[johlrich]

It looks like a new version of the package hit nuget last week without netstandard support. Any updates to when we can expect to see that?

[yaronyg]

We are still on the same schedule as I previously specified, first 1/2 of 2018.

[paulinventome]

Is there any way to use TableAPI in Cosmo from a dot net core application at the moment, going through the older Windows.Storage libraries? I'm working on a large ish project and if i can't get table support then it will be AWS or similar instead.

[aboo]

Can I get a rain check on this please?

[kirankumarkolli]

Azure Cosmos table API is REST protocol compliant with Azure table storage. .NET standard Widnows storage SDK can be leveraged as an alternative.

[SeanFeldman]

@kirankumarkolli while it's true that the API is a REST protocol at heart, when working with .NET stack, you want to use a .NET SDK client rather than REST API.

You can prepare pizza dough, all the ingredients, and bake the pizza at home. Or can go to the restaurant. Relating to the metaphore, I guess we're paying by waiting @aboo 🙂

[aboo]

Cannot agree more with @SeanFeldman
We've a backlog which makes a lot of sense to do with CosmosDB Tables Api and we've waited so long trying to avoid doing something in parallel with what MS is doing. Just yesterday we've decided to use CosmosDB SQL. Still would be great to have this SDK out ASAP. Thanks.

[TechInceptions]

First 1/2 of 2018 is almost upon us - should we look for alternative approaches?

[MisinformedDNA]

@yaronyg Are we still on track? Is there anything we can do to speed this along? 16 days left...

[yaronyg]

@MisinformedDNA @kirankumarkolli is a better person to ask than me. :)

[manigandham]

Side note:

All data in CosmosDB is stored the same underneath using its proprietary architecture and just exposed with different interfaces, so you can use any of the other APIs to get the same performance and scale. The Cassandra data model is a similar wide-column style to Azure Tables and it supports pretty much all of the same operations. The existing Storage Tables API also isn't going anywhere, and the pricing model is different too.

[joshlang]

Pleeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeease!

[SeanFeldman]

@manigandham if your data is stored using tables, and system is designed to work with Tables, I don't see how any ither API is relevant, regardless of how data is stored internally.

@kirankumarkolli @yaronyg it's June (happy Father's Day!). Is this still on schedule or has it changed? Thanks.

[joshlang]

Is there any system to put a bounty on stuff like this? I'd happy to pay some bitcoin or other crypto if it'd bump it up the priority list?

[TechInceptions]

We are just a few days away from July - and we are still waiting for the .NET Standard support CosmosDB Table API.

[RyanONeill1970]

I noticed CosmosDB support is possibly in the next EF Core 2.2 release. They may be implementing it that way instead.

[MisinformedDNA]

Except that they said, in this thread, that they would release this in the first half of 2018. And then went silent...

[TechInceptions]

Let me just say that EF cannot possibly be the answer.

[joshlang]

I've abandoned this and moved to a different solution. Thanks anyway.

[searbe]

It looks like you can use CloudTable from the Microsoft.WindowsAzure.Storage.Table namespace (i.e. the WindowsAzure Storage package at https://www.nuget.org/packages/WindowsAzure.Storage/). Create your CosmosDB instance, choose Table API when you create it, then:

var storageAccount = CloudStorageAccount.Parse("connectionstring");
var tableClient = storageAccount.CreateCloudTableClient();

(Obviously this only helps if all you're looking for is compatibility with the Table Storage API)

[SeanFeldman]

@searbe that client will not have the APIs specific to CosmosDB. Therefore it's more of a temporary workaround than a permanent solution. I agree with the above that the communication on this issue is poor.

Yes @kirankumarkolli @yaronyg, I'm looking in your direction. It's OK not to have an exact date. It's OK to let know that a date is pushed out. What is not OK is to let it sit w/o updates when customers ping and leave them wondering what's going on. I hope you take some time to update the issue. Thank you.

[joshlang]

Thanks @searbe … Yes, our solution worked using the old storage package, for the most part. There were some pesky issues though, making it an unreliable workaround. Thanks though.

[kirankumarkolli]

Apologies for the inconvenience. We will update this thread with more details and clear ETA in 2 weeks.

[wmengmsft]

We apologize for the inconvenience. We are fully committed to delivering a .Net Standard SDK for Table API. We are tracking for delivery by October or before.

[sinapis]

We had 2 clients that cosmos DB table API would have been perfect for them but we went with an alternative solution because there is no .net core support. This decision actually resulted in us migrating the whole solution from azure to AWS, since splitting our solution across two cloud providers was not an option in both cases, and incurrs latency penalty and other issues. We would really like to come back, but as long as there is no .net core support, we can't.

[joshlang]

We're back on track using Azure Cosmos DB Table API.

But the lack of a .NET Standard library means we need horrible workarounds.

https://github.com/Azure/azure-cosmosdb-dotnet/issues/517 <-- This is a persistent problem that kills us.

Our workaround is to maintain TWO sets of projects. One .NET Standard/Core, and one .NET Framework (with Microsoft.Azure.CosmosDB.Table, which actually does work), and to coordinate code changes via linked source files. It also requires silly things so that it can compile in both projects, like this, because things are in different namespaces:
namespace Microsoft.Azure.CosmosDB.Table { }
namespace Microsoft.Azure.Storage { }// delete this crap once .NET Framework no longer needed
namespace Microsoft.WindowsAzure.Storage.Table { }

One of our specific use cases, just as an example, is to scan various blockchains for deposits. Deposit addresses belonging to our customers are stored in Cosmos. When a Bitcoin block arrives, there might be say 10000 potential deposits. So we fire off 10000 concurrent Cosmos requests to say "Hey, does this deposit address exist?". 99.9% of them will return 404 (and needlessly throw an expensive exception, even though a 404 shouldn't be exceptional - https://github.com/Azure/azure-cosmosdb-dotnet/issues/219 ). Whatever does exist, we continue and do further processing.

It all works fine. Except with the old Azure Storage Table library, 0.01% of requests sometimes take 5 minutes to return (and occasionally it never returns at all unless we disable the network adapter). With the .NET Framework cosmos package, we don't have any problems at all, other than needing to use .NET Framework.

So, all in all, I suppose I'm just begging, in a long-winded way, for this to be done some time soon :)

[MisinformedDNA]

@joshlang Have you tried multi-targeting instead of multiple projects? This blog post has a great walkthrough of everything you should need to do that.

[joshlang]

@MisinformedDNA I had no idea that was possible. I've started implementing it to see if it's suitable, but yes, it definitely would simplify our workaround.

Thanks! :)

[TechInceptions]

@kirankumarkolli, on Jul 9, we were promised an update, as well as an ETA.

It would be much appreciated if we could have an update or an ETA.

Thanks

[manigandham]

@TechInceptions
The next comment by @wmengmsft gives you the update:

We apologize for the inconvenience. We are fully committed to delivering a .Net Standard SDK for Table API. We are tracking for delivery by October or before.

[TechInceptions]

Yes.
I actually cheered that comment.
Time passed.
And I forgot.

October it is.
Thanks.

[pcoppney]

Ok, it is October. Any updates on this?

[wmengmsft]

We are almost there. We expect to push out the preview version of the .NET Standard SDK during the week of 10/15. Really appreciate your patience with us so far waiting for this release.

[pnda489]

Will this preview .NET Standard SDK be released through NuGet?

[pnda489]

We are almost there. We expect to push out the preview version of the .NET Standard SDK during the week of 10/15. Really appreciate your patience with us so far waiting for this release.

We really want to go on and use Cosmos DB. :( Any updates?

[aboo]

Well, not sure what the issue is but personally I'm not very optimistic that this package will be released any time soon. How hard is it to write a client for an API? and CosmosDb is already there and Tables is just a simple dialect of CosmosDB.

I've delayed a part of our project for over 6 months now waiting for this client to be released. We are using CSV files at the moment. I wish we had implemented our own client at the time.

If the team could push whatever they have, the community will finish it.

[wmengmsft]

Hi all, thank you for your patience all this time. We just released a preview of the .NET Standard SDK for Cosmos DB Table API. You can download it here. We aim to pick up the pace of development in the future. Please send us feedback and we will address them as quickly as we can. Thanks!

[aboo]

Just confirming that this work :)

Apparently it only supports Cosmos DB Tables at the moment and if you try it with Storage tables you will get an error saying it's not supported.

Here is a link to the release note page.

I followed the .Net Framework samples and was able to create a table, add an item and query the table successfully.

Would be great if someone from the dev team can share the road map for this package.

Thanks so much.

A bit late but hey late is always better than never ;)

[SeanFeldman]

Given the client is for Azure Cosmos DB Table .NET Standard API, I'd
not expect it to work Storage Tables 🙂

[ggirard07]

Can anyone confirm whether the SDK will be updated to support Azure Storage Table endpoints?

Trying to figure out whether we will have to migrate to CosmosDB instead of Azure Storage at some point to have access to latest SDK updates. I hope no as the CosmosDB pricing is way too different from Azure Storage and does not match our requirements...

[geekcubed]

Please send us feedback and we will address them as quickly as we can. Thanks!

Well @wmengmsft - you asked, so here it is. What a cluster ** this is. We have been using ~DocumentDb~ CosmoDB in production for over 3 years - I'd like to think we have a handle on how to do this Azure stuff by now. I'm in the midst of porting some architecture from Net 4.7 to Net Standard. I _thought_ the hard part was going to be migrating from EF6 to EFCore - turns out I was wrong. Here's a reality check for you:

• Support for Azure Tables is available in the WindowsAzure.Storage package v <= 9.3.2 😐
• <= v.9.3.2 of the storage library is not semantically versioned between platforms - re-targeting from framework to standard breaks a bunch of their SDK calls 🤦‍♂️
• Luckily v9.4 is semver between platforms 😃
• But as of v9.4, support for Azure Tables is not supported & the advice is the use the CosmosDB SDK 😬
• Microsoft.Azure.CosmosDB.Table is not compatible with .net Standard 😭

Azure has grown immeasurably since we first adopted the platform - and quite frankly it looks like it's about to blow apart at the seams. The various platform teams need to sit down in a big room and sort this stuff out.

Right now I'm totally stuck - rewrite a bunch of code to support an older version of the Storage Library, or just park a giant investment in the future of out platform until the technology we depend on gets itself together

[dlouwers]

We are at the point of selecting a cloud provider and services. Azure Table Storage, as well as CosmosDB Tables, are a good fit for one of our core needs but it has to be .NET Standard compliant. When can we expect the .NET Standard support for CosmosDB Tables to be complete and ready for production and is there a definitive period for which it will be maintained?

[wmengmsft]

@dlouwers, the .NET Standard SDK will be maintained for extended period of time (all foreseeable future). The goal is to make it the defacto .NET library for Table API. Currently we are looking to get to GA by early next year (around Jan).

[TechInceptions]

Just start re-implementing my library using the .Net Standards package.

Noticed that the StorageCredentials class was reimplemented, but lacks the constructor that takes a Sas-powered Url to the Azure storage table:

Example:
(New) .Net Standard:
var storageAccount = new CloudStorageAccount(new StorageCredentials(accountName, accountKey), true);

(Old) .Net Framework:
var storageAccount = new CloudStorageAccount(new StorageCredentials(storageTableSasUrl), true);

The StorageCredentials class in "new" exists in the Microsoft.Azure.Cosmos.Table namespace, while the class in "old" exists in the Microsoft.Azure.Storage.Auth namespace.

Are we going to get a constructor that takes a Sas key-powered Uri?

Using a Sas key-powered Uri is a more secure way to access an Azure storage table - it means we don't have to expose the storage account key.

Thoughts?

[vascofernandes]

@geekcubed is very right. This is a complete mess. Also there is a severe lack of proper communication with the community about this.

To add to all the problems, this package https://www.nuget.org/packages/Microsoft.Azure.Cosmos.Table does not support BufferManager, which was available previously.

Microsoft_WindowsAzure_Storage_Table_CloudTableClient_BufferManager

Microsoft_Azure_CosmosDB_Table_CloudTableClient_BufferManager

Moreover, TableRequestOptions does not have the property EncryptionPolicy.

[Fredborg]

Hi,

Been trying out the Microsoft.Azure.Cosmos.Table -Version 0.9.1-preview.

If i new up a .net core console app project, and call methods in my .net standard library that uses the .net standard version of Microsoft.Azure.Cosmos.Table, everything seems to work just fine.

However, calling the exact same methods from my Azure Function fires Storage Exceptions:

Exception while executing function: NewFunction.
Microsoft.Azure.Cosmos.Table: Method not found: 'Void Microsoft.Azure.Documents.Client.DocumentClient..ctor(System.Uri, System.String, Newtonsoft.Json.JsonSerializerSettings, Microsoft.Azure.Documents.Client.ConnectionPolicy, System.Nullable`1)'.

It seems to me, that this must be a versioning issue, with using a constructor method that has changed?

Or it can be a binding redirect issue, that Azure Functions suffers from, especially with versions of Json.

But if I can get it to work fine calling it from a .Net Core app, I really should be able to get it working using Azure functions.

Any help - or ideas?

[SeanFeldman]

Is Table API client going to GA at some point soon? I'm not asking for a specific date, but to have an idea where this is heading. Some customers do not want to touch a nuget that is in it's first preview state. Thank you.

[wmengmsft]

@SeanFeldman , there are a couple items we are actively working through for the new Table API SDK before going GA:

• Adding a few missing APIs and support for Azure Storage endpoints. This will also address the needs of a few folks who are in the process of migrating the application.
• Moving to Microsoft.Azure.Cosmos as a dependency (instead of Microsoft.Azure.DocumentDB). This will bring additional performance improvements and align with our overall .NET clients roadmap.
• Make the code-base ready to be open-sourced.

We are targeting a GA release sometime towards the latter part of this quarter.

[MisinformedDNA]

@wmengmsft Interesting that you have to make your code "ready" to be open-sourced, instead of just developing in the open.

[raghukrishnamoorthy]

@geekcubed I can feel your pain from across the planet.

I need help folks. I have a Azure Bot Service application built on .NET Core that needs to use Azure Table Storage and the Table storage APIs are not supported for .NET Core. What do I do now? I need a solution within a week. What do I tell the stakeholders?

[geekcubed]

@raghukrishnamoorthy - I've just used v9.3.2 of WindowsAzure.Storage : we went with that and have been running it in our dev stack for 3 months with no issues, and will take it production shortly. We'll continue to use it for Table Storage support until CosmosDB get their library sorted (which we already use to connect to CosmosDB itself)

[raghukrishnamoorthy]

@geekcubed Are there any samples using that version of the WindowsAzure.Storage library? I need basic persistence and retrieval. That's all.

[geekcubed]

@raghukrishnamoorthy http://lmgtfy.com/?q=azure+table+storage+WindowsAzure.Storage+tutorial

[raghukrishnamoorthy]

@geekcubed I googled and tried the examples and had some problems running the samples on the docs site. Nonetheless, thanks for the snarky response. I'll poke around some more.

Have a good one :)

[ilya-git]

I am a bit confused on the Azure Table Storage support for this new library. It seems to work in my tests, but the name of the library clearly says it is for CosmosDB.

Will this library officially support both Azure Table Storage and Cosmos DB Table API?

[wmengmsft]

@geekcubed, have you looked at our new .NET Standard Table SDK? It now supports Azure Storage Table endpoints as well.

@raghukrishnamoorthy, I hope the above SDK can solve your needs. Here is a sample available here. Please let us know if you encounter any issues using it.

@ilya-git, yes, this SDK officially supports both Azure Table and Cosmos DB Table API endpoints.

[ilya-git]

Thank you for the answer, is it already possible to use that one in production or is it premature for that?

I have tried using Microsoft.Azure.Cosmos.Table and Microsoft.WindowsAzure.Storage in .net core and it seems that the first one is sometimes unstable against Cosmos Table API but works against storage just fine. It just hangs on read/insert operations without throwing any exception. The second library just works all the time

[TechInceptions]

It now supports Azure Storage Table endpoints as well.

That is a big deal - I have been waiting for Azure Storage Table support for since May 2018.
Thanks :-)

[manigandham]

Is there a reason why the new SDK is a brand new package at Microsoft.Azure.Cosmos.Table instead of updating Microsoft.Azure.CosmosDB.Table?

https://www.nuget.org/packages?q=microsoft.azure.cosmos

This will end up causing more confusion in an already confusing list of SDKs. I highly recommend consolidating these library names while they are still new.

[SeanFeldman]

@manigandham I can only assume that multi-targetting is not possible with Microsoft.Azure.CosmosDB.Table. Hence a new package. Agreed, it's sub-optimal.