Sqlclient: Globalization Invariant Mode is not supported

[WeihanLi]

when I use Microsoft.EntityFrameworkCore.SqlServer Version 3.0.0, got an error as follows:

Application startup exception: System.NotSupportedException: Globalization Invariant Mode is not supported.
   at Microsoft.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)
   at Microsoft.Data.SqlClient.SqlConnection.Open()
   at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenDbConnection(Boolean errorsExpected)
   at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.Open(Boolean errorsExpected)
   at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerDatabaseCreator.<>c__DisplayClass18_0.<Exists>b__0(DateTime giveUp)
   at Microsoft.EntityFrameworkCore.ExecutionStrategyExtensions.<>c__DisplayClass12_0`2.<Execute>b__0(DbContext c, TState s)
   at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.ExecutionStrategyExtensions.Execute[TState,TResult](IExecutionStrategy strategy, Func`2 operation, Func`2 verifySucceeded, TState state)
   at Microsoft.EntityFrameworkCore.ExecutionStrategyExtensions.Execute[TState,TResult](IExecutionStrategy strategy, TState state, Func`2 operation)
   at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerDatabaseCreator.Exists(Boolean retryOnNotExists)
   at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerDatabaseCreator.Exists()
   at Microsoft.EntityFrameworkCore.Storage.RelationalDatabaseCreator.EnsureCreated()
   at Microsoft.EntityFrameworkCore.Infrastructure.DatabaseFacade.EnsureCreated()

Additional context

Microsoft.EntityFrameworkCore.SqlServer Version 3.0.0
Microsoft.Data.SqlClient version: 1.0.19249.1
Target framework: (.NET Core 3.0)
Operating system: Docker Alpine

Solution

fixed with add the follows to the dockerfile

RUN apk add icu-libs
ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false

more details: https://www.abhith.net/blog/docker-sql-error-on-aspnet-core-alpine/
reference sample project: https://github.com/WeihanLi/ActivityReservation

Copied from https://github.com/aspnet/EntityFrameworkCore/issues/18025

[Wraith2]

This is a problem which has come up before. I believe the only way to fix the issue is to install the ICU libraries on the docker container and disable invariant mode.

Invariant mode is used to lower the size of the image and allows the native internationalization libraries to be omitted. Those libraries are a specific requirement for the sql server driver because it needs to be able to create encoders to translate strings from the protocol according to their applied collation sequence. Even logging in requires access to the libraries because that's when the database default collation is provided.

[80dB]

We ran into the same problem. And while I believe @Wraith2 :)

The problem we are facing is:
.NET Core 2.2 + EFCore 2.2.6 + Alpine works.

Same project:
.NET Core 3 + EFCore 3 + Alpine gives the above error.

[David-Engel]

This is by design.

.NET Core 2.2 + EFCore 2.2.6 + Alpine works because you happen to be using compatible collations between the client and server. If they had been different, you could hit #81. So we added the specific exception message pointing users to the problem and solution. It would take a lot of time to identify the scenarios where the new Globalization Invariant Mode can be supported by SqlClient and target code changes specific to those scenarios. Note, it's not just at connection Open() that exceptions could occur so identifying everywhere the code would have to essentially "defend" against Globalization Invariant Mode could be very extensive.

[80dB]

Interesting design choice. I'm guessing it worked on our images (and many others) just fine as long as we (/they) had an English installation of SQL Server.

I would have opted to detect (if possible) the InvalidOperationException from #81 and if caused by globalalization invariant mode, throw this new exception at that point. Instead of breaking it for everyone.

Anyhow, it's fine for me. I know it now and even though I don't like the idea of making my alpine-based images bigger (since most of them use SQL server to store data), I guess there's no other way to use EFCore 3 with .NET Core 3.

I would recommend putting up a blog post somewhere (although google now finds this issue, so at least that's something).

[Wraith2]

@divega is it possible a note could be made somewhere in EFCore that people might read that if they're using docker they'll need the ICU libraries installed? I know it isn't an EF dependency but if that's the route that most users are hitting this problem take then it might be the best way to get the information out to them.

Or is there scope to have an alpine+mssqldeps image for people to use do you think?

[divega]

cc @roji, @bricelam

[seroche]

Does this means that anyone using one of the docker images published by Microsoft will encounter this issue? That's a fairly big limitation. This flag has been set to true since .net 2.1...

Are we sure disabling DOTNET_SYSTEM_GLOBALIZATION_INVARIANT does not come with side effects?

[roji]

Thanks @Wraith2, we'll add a note about this in the docs (https://github.com/aspnet/EntityFramework.Docs/issues/1789)

[cheenamalhotra]

Closing issue as will be documented in EF Core Documentation.

[nxtx]

@cheenamalhotra the documentations are not updated, they merely refer to a blog post that does not answer this question. So, please re-open this issue so that it gets answered.

This is can be a blocking issue (and quite honestly, a very severe one) for many of us.

[Wraith2]

It isn't supported for a good technical reason. If you want to use sqlclient in docker you'll need to make sure you have the icu libraries installed. I don't know what you're expecting anyone here to do to change that.

[nxtx]

a workaround?

by workaround I don't mean description, rather I mean actionable settings or commands to run, and where to run them. (you have made a breaking change for "good" reasons, and not providing any workaround.)

[80dB]

It isn't supported for a good technical reason. If you want to use sqlclient in docker you'll need to make sure you have the icu libraries installed. I don't know what you're expecting anyone here to do to change that.

I think the major issue is this one:
https://github.com/dotnet/SqlClient/issues/220#issuecomment-535628814
(e.g. it worked for many people without the change in the docker file (e.g. without installing icu))

And that it's not listed as a breaking change on this page (yet):
https://docs.microsoft.com/en-us/ef/core/what-is-new/ef-core-3.0/breaking-changes

Just helping nxtx ;) And helping nxtx a bit more, the temporary solution isn't temporary. It's the (only) permanent solution. Add the 3 lines (or 2 if you remove the comment) just before the ENTRYPOINT part in your docker file and it'll work just fine.

[Wraith2]

The only change was to make the error specificy that the cause was globalization invariant mode rather than a confusing error about being unable to connect. The problem existed on all previous versions and if you managed to get it working it was purely by luck.

The "workaround" is the fix, install the required libraries on the container.

[cheenamalhotra]

@nxtx https://github.com/aspnet/EntityFramework.Docs/issues/1789 has been opened to address documentation change in above comment https://github.com/dotnet/SqlClient/issues/220#issuecomment-536541037 counter to this issue.

@roji do you have any update on when this documentation note is planned to be published?

[nxtx]

I am clueless on what you're talking about!
If there exist "A" solution, what is it?

install the required libraries on the container.

what are the required libraries? how should I install them?

[ErikEJ]

@nxtx The so called "Temporary solution" https://github.com/dotnet/SqlClient/issues/220#issue-498595465 (which is probably more like a permanent solution)

[CodeSwimBikeRunner]

RUN apk add icu-libs
ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false

Does this have any side effects for existing 2.1 runtime code?

[yukiwongky]

@CodeSwimBikeRunner you'll need these 2 lines to get the globalization library, which is needed by the SqlClient driver.

[roji]

@cheenamalhotra I'll try to update the docs on our side by the end of the week. However, that's only EF Core - this should ideally be documented in SqlClient docs (to which we'd link to from EF Core).

[nunorelvao]

Wow , i just lost 3 days over trying to undertand why my API sudenly started to hang just by moving to .net core 3.1 in DOCKER , just to discover that SQL Client for Entity Framework dos not play well with some Docker Images!

I just overcome the issu by trying almost all and finally i am using the Ubunty 18.04 (Bionic) 3.1-bionic and all is working now!!!

And with Alpine as example i get this error "Globalizations Invariant Mode is not supported" but eg with Debian i dont get that error and still hangs!

WEIRDO ALERT!!!!!! This need to be solved or Documented on how to overcome this issues!

Great Microsoft and no warning!!!

[Wraith2]

with Alpine as example i get this error "Globalizations Invariant Mode is not supported" but eg with Debian i dont get that error and still hangs!

The alpine image you used doesn't contain ICU libraries and the Debian one did. They're pretty large libraries so people like to try and remove them from docker images if possible.

just to discover that SQL Client for Entity Framework dos not play well with some Docker Images!

They play fine together. The problem is that you used an image that doesn't contain the dependency. That's a you problem.

The library tells you about the problem with the exception message that if you put into your favorite search engine will get you plenty of discussion and this rather good blog article https://andrewlock.net/dotnet-core-docker-and-cultures-solving-culture-issues-porting-a-net-core-app-from-windows-to-linux/ that will lead you through solving the problem. Kindly direct your anger at somewhere useful or appropriate.

[nunorelvao]

with Alpine as example i get this error "Globalizations Invariant Mode is not supported" but eg with Debian i dont get that error and still hangs!

The alpine image you used doesn't contain ICU libraries and the Debian one did. They're pretty large libraries so people like to try and remove them from docker images if possible.

just to discover that SQL Client for Entity Framework dos not play well with some Docker Images!

They play fine together. The problem is that you used an image that doesn't contain the dependency. That's a you problem.

The library tells you about the problem with the exception message that if you put into your favorite search engine will get you plenty of discussion and this rather good blog article https://andrewlock.net/dotnet-core-docker-and-cultures-solving-culture-issues-porting-a-net-core-app-from-windows-to-linux/ that will lead you through solving the problem. Kindly direct your anger at somewhere useful or appropriate.

First of All , Thank you for the reply, and sorry if you took this as "Anger", i am more kind of tired and not really happy with lack of documentation about this issues.

I am not fully sure on what is the problem exactly. I just know it was Entity Framework trying to connect, and become "Stuck" , i did not get any runtime error or even in docker shell ("I did not set verbose as i had already a lot to invetigate") but i started to test some images , and the ones i tried eg: latest 3.1 buster (debian) , Alpine, even Ubuntu 20.04 (Focal) ,all had same behaviour just stuck. The only working well for now is "Ubuntu 18.04" Bionic.

So again i just wish there were more clear and simpler documentation about this issues at least so that we could be warned.

[roji]

@cheenamalhotra @David-Engel looked at adding a warning about this in the EF docs for the SQL Server provider (https://github.com/dotnet/EntityFramework.Docs/issues/1789), and it occurred to me this would live better in the SqlClient docs.

Do you think it makes sense to add a "Limitations and gotchas" under your docs, and we'd link to it from ours?

[cheenamalhotra]

@roji

Yes, we're considering doing that, will update you when our doc is published.

[roji]

Thanks @cheenamalhotra. Another candidate may be to warn people of the async perf issue when reading large values (https://github.com/dotnet/EntityFramework.Docs/issues/2608).