Azure-cosmos-dotnet-v2: The authorization token is not valid at the current time (Debug / VS2017 / Docker)

Created on 14 May 2018 · 20Comments · Source: Azure/azure-cosmos-dotnet-v2

Hi,

I am developing an .NET core application that connects to the Azure Document DB. via the DocumentClient, this application is currently running locally in docker(windows) on my laptop.

When executing following code when running in VS2017, in Docker:
this._documentClient = new DocumentClient(new Uri(_configuration["DataReqeuestRepo:CosmosEndpoint"]), _configuration["DataReqeuestRepo:CosmosPrimaryKey"]); await this._documentClient.CreateDatabaseIfNotExistsAsync(new Database { Id = _configuration["DataReqeuestRepo:CosmosDataRequestDbName"] });

I get following error:
Exception thrown: 'Microsoft.Azure.Documents.DocumentClientException' in System.Private.CoreLib.dll Exception thrown: 'Microsoft.Azure.Documents.DocumentClientException' in System.Private.CoreLib.dll Exception thrown: 'Microsoft.Azure.Documents.DocumentClientException' in System.Private.CoreLib.dll fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[0] An unhandled exception has occurred while executing the request Microsoft.Azure.Documents.DocumentClientException: The authorization token is not valid at the current time. Please create another token and retry (token start time: Mon, 14 May 2018 12:48:25 GMT, token expiry time: Mon, 14 May 2018 13:03:25 GMT, current server time: Mon, 14 May 2018 05:03:36 GMT). ActivityId: 516356b5-09d5-47fd-b708-67ed1fdc8865, Microsoft.Azure.Documents.Common/1.22.0.0, Windows/10.0.16299 documentdb-netcore-sdk/1.9.1 at Microsoft.Azure.Documents.Client.ClientExtensions.<ParseResponseAsync>d__1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter1.GetResult()
at Microsoft.Azure.Documents.Client.GatewayServiceConfigurationReader.d__41.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter1.GetResult() at Microsoft.Azure.Documents.Routing.GlobalEndpointManager.<GetDatabaseAccountFromAnyLocationsAsync>d__18.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter1.GetResult()
at Microsoft.Azure.Documents.Client.GatewayServiceConfigurationReader.d__43.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter1.GetResult() at Microsoft.Azure.Documents.Client.DocumentClient.<InitializeGatewayConfigurationReader>d__267.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.Documents.Client.DocumentClient.<GetInitializationTask>d__55.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.Documents.Client.DocumentClient.<EnsureValidClientAsync>d__100.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.Documents.Client.DocumentClient.<ReadDatabasePrivateAsync>d__168.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter1.GetResult()
at Microsoft.Azure.Documents.BackoffRetryUtility1.<>c__DisplayClass1_0.<<ExecuteAsync>b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter1.GetResult()
at Microsoft.Azure.Documents.BackoffRetryUtility1.<ExecuteRetry>d__3.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.Azure.Documents.BackoffRetryUtility1.d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter1.GetResult() at Microsoft.Azure.Documents.BackoffRetryUtility1.d__1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter1.GetResult() at Microsoft.Azure.Documents.Client.DocumentClient.<CreateDatabaseIfNotExistsPrivateAsync>d__106.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter1.GetResult()`

If I run the solution out of docker, everything works.

I tried restarting Docker, rebooting PC, issue is persistent.

Source

iBoonz

👍2

All 20 comments

The authorization token is not valid at the current time. Please create another token and retry (token start time: Mon, 14 May 2018 12:48:25 GMT, token expiry time: Mon, 14 May 2018 13:03:25 GMT, current server time: Mon, 14 May 2018 05:03:36 GMT). ActivityId: 516356b5-09d5-47fd-b708-67ed1fdc8865, Microsoft.Azure.Documents.Common/1.22.0.0, Windows/10.0.16299 documentdb-netcore-sdk/1.9.1

This means there is a clock skew between the server and client. What's the local time when this error is thrown?

kirankumarkolli on 14 May 2018

👍1

6:AM, but how do you explain why this works if I just run out of a docker container. And if I run in Docker I get the error again

iBoonz on 14 May 2018

I got the same error with an Azure Function and CosmosDbTrigger:

Run: Microsoft.Azure.WebJobs.Host: Error indexing method 'TelemetryProcessingFunction.Run'. Microsoft.Azure.WebJobs.Extensions.DocumentDB: Cannot create Collection Information for telemetry in database xxx with lease leases in database xxx: The authorization token is not valid at the current time. Please create another token and retry (token start time: Tue, 08 May 2018 12:47:34 GMT, token expiry time: Tue, 08 May 2018 13:02:34 GMT, current server time: Tue, 08 May 2018 12:39:54 GMT).
ActivityId: a4bbfe36-40b7-48a2-b5bc-4cc9b4b9a916, Microsoft.Azure.Documents.Common/1.22.0.0. Microsoft.Azure.Documents.Client: The authorization token is not valid at the current time. Please create another token and retry (token start time: Tue, 08 May 2018 12:47:34 GMT, token expiry time: Tue, 08 May 2018 13:02:34 GMT, current server time: Tue, 08 May 2018 12:39:54 GMT).
ActivityId: a4bbfe36-40b7-48a2-b5bc-4cc9b4b9a916, Microsoft.Azure.Documents.Common/1.22.0.0.

marxxxx on 17 May 2018

The source machine where the code ran might have clock skew. What's the difference between 6AM local time and 'Mon, 14 May 2018 12:48:25 GMT'

kirankumarkolli on 19 May 2018

They are 2 different test sessions, not related

iBoonz on 29 May 2018

I'm running into this exact same issue right now. If I run outside of a Docker container locally, DocumentDB connection is happy. But the moment I run it in my local Docker container, I get this error. I've tried cleaning/rebuilding, restarting Docker and restarting my development machine with no luck in it working...

aladd04 on 17 Aug 2018

I'm seeing the same thing as @aladd04. When trying to use DocumentDB from a Service Fabric Mesh project (which runs using Docker), I get the DocumentClientException: The authorization token is not valid at the current time. Please create another token and retry error. When calling the exact same code from a unit test, it works as expected.

csdahlberg on 9 Sep 2018

I'm seeing the same thing as @aladd04. When trying to use DocumentDB from a Service Fabric Mesh project (which runs using Docker), I get the DocumentClientException: The authorization token is not valid at the current time. Please create another token and retry error. When calling the exact same code from a unit test, it works as expected.

same here. @csdahlberg did you find any solution to this?

sebader on 28 Sep 2018

I was able to get it working for my needs by switching from microsoft/dotnet:2.1-aspnetcore-runtime-nanoserver-1709 to microsoft/dotnet:2.1-aspnetcore-runtime-nanoserver-sac2016.

The underlying cause seems to be tracked at https://github.com/moby/moby/issues/37283, which has comments saying it has also been fixed in the latest 1803 image, too, but I have not tried that again.

csdahlberg on 28 Sep 2018

Thanks a lot! I just changed it to microsoft/dotnet:2.1-aspnetcore-runtime and now CosmosDB connection works!

sebader on 1 Oct 2018

👍1

Same here. Created an issue: Time sync differences between docker containers for Windows and the hosting machine

tmarinov on 10 Jan 2019

@csdahlberg @sebader how are you guys connecting to Cosmos DB from the container? I've been fighting with that trying to import certificates and what not.

julipur on 22 Jan 2019

I am having this issue. using aspnet:latest
It seems that sometimes if I change the image it works for awhile, but that same image will fail later. Pretty frustrating.

TimZander on 29 Jan 2019

@TimZander which image did it work for you for?

julipur on 29 Jan 2019

@julipur last time I got it working I went from a specific version to aspnet:latest but now that is not working.

`PS U:> [System.DateTime]::UtcNow

Tuesday, January 29, 2019 10:24:12 PM

PS U:> docker exec bgvapi powershell [System.DateTime]::UtcNow

Tuesday, January 29, 2019 10:55:09 PM`

TimZander on 29 Jan 2019

I created a service in .Net Core in a service fabric mesh. The service tryes to get data from Azure Cosmos Table. I have a scenario that sometimes I am getting the data, but most of the time I am getting this exception:
Microsoft.Azure.Cosmos.Table.StorageException: 'The authorization token is not valid at the current time. Please create another token and retry (token start time: Tue, 23 Apr 2019 22:06:54 GMT, token expiry time: Tue, 23 Apr 2019 22:21:54 GMT, current server time: Tue, 23 Apr 2019 14:49:52 GMT).
ActivityId: 5190e817-a5b5-4bbd-94d9-2fd2eb039859, Microsoft.Azure.Documents.Common/2.2.0.0, Windows/10.0.17134 documentdb-netcore-sdk/2.1.3'

Looks like some time discrepancy between my computer where I run the service and Azure server.
This is the code I am using to get data from the table:

var storageAccount = CreateStorageAccountFromConnectionString(connectionString);
var tableClient = storageAccount.CreateCloudTableClient();
var table = tableClient.GetTableReference(tableName);
var query = new TableQuery<TopicSubscriberDefinition>();
var entites = table.ExecuteQuery(query).ToList();

miljanrajkovic on 23 Apr 2019

Use this as the base image

FROM mcr.microsoft.com/dotnet/core/aspnet:2.1-nanoserver-sac2016 AS base

julipur on 23 Apr 2019

This is apparently a problem with in python (latest 3.8.2) using the the latest cosmosdb sdk (azure-cosmos==3.1.2). Im using as import azure.cosmos.cosmos_client as cosmos_client
and get that same time error on a token when running in a container.

Works great natively on the host the container runs on however.

And even in the container, using the SDK's keyvault client works great. Just the CosmosDB client is hosed.