Runtime: Add gss-ntlmssp to runtime-deps packages

[MichaelSimons]

This is related to dotnet/corefx#28961, dotnet/wcf#943, and dotnet/dotnet-docker#478

Without this NTLM authentication will not work. In the past that would impact only NegotiateStream but in 2.1 this also impacts HTTPClient so the exposure is bigger.

cc @wfurt, @rakeshsinghranchi

[rakeshsinghranchi]

cc @leecow , @Petermarcu , @drneve

Thanks @MichaelSimons for opening the issue here.
Looking at the bug description, fix is required for 2.1 release.

@leecow - PTAL at the issue and it's impact for 2.1 release.

@drneve can work on this.

[Petermarcu]

@stephentoub should this be a hard dependency or a doc'd dependency? ie, document "if you call this API, make sure you have XXX library installed". With a good error message?

[stephentoub]

I'm actually not sure if we've ever tested that case. It's possible things don't end well. I think it'd be reasonable to make the dependency optional as long as the failure mode is reasonable. This isn't something people opt-in to, e.g. things shouldn't blow up if you try to connect to a site that challenges negotiate or NTLM and you don't have this dependency available.

cc: @karelz

[Petermarcu]

I think we need a good failure mode regardless. People xcopying .NET Core onto a machine need to get good errors when things don't go well due to missing dependencies. This isn't sounding like a great option for "optional" based on what I'm hearing so I do think we should add this package dependency as well as a good error when the dependency is missing.

[drneve]

It seems there is no availability for that package in most of the RPM distros (the only one I saw that has gss-ntlmssp available is fedora). Is it right?

[leecow]

Looks like that's the case. I didn't find the lib on the default feeds for CentOS, OpenSUSE and SLES.

[MichaelSimons]

I would also like guidance on what package is needed on Alpine.

[wfurt]

The intention was to include it in our docker images so ntlm just works. I don’t think we need to necceasily claim it as dependency. There are other ways how to get it working.

[MichaelSimons]

@wfurt - The intention is for the Docker images to align with the product installers (e.g. the runtime-deps Docker images mirror the runtime-deps packages). Why shouldn't the installers/packages setup the environment so that ntlm authentication works with no changes?

[karelz]

We should not claim it as dependency in installers - the poor availability on distros is one reason.
The key thing is that we do not directly call into this package ever. We call GSSAPIs and this will lightup NTLM in them. Without the package installed (or without another alternative like that), NTLM will just not work.

If we feel strongly that Docket images should align with product install, then let's close this issue. We will doc it - "If you need NTLM, you need some GSSAPI plugin to enable it on the OS - e.g. gss-ntlmssp".
We treat NTLM as OS capability (potentially via OS "extensions").

[wfurt]

As @karelz said we make no direct calls or use of this. We depend on GSSAPI - authentication layer defined by rfc. It is up to OS or distribution to provide services. There can be more than one provider for same mechanism or they can decide not to support some.

Now in normal distribution .Net is just one package from many. In case of Docker we provide many packages beyond .Net. We inherit many parts from upstream image(s) but we put our name and tag on final set. From that prospective we behave as micro distribution.

As far as some other earlier questions:
I did no testing or research on Alpine. There may be alternative solutions (like Heindal) or somebody (us, maintainers or vommunity) would need to create package specification.

If given distribution has no binary package, one can use binary rpm from similar distribution or simply rebuild from SRPM.

I also did no testing on MacOS. However if we fix our Docker, it should just work as solution.

This is not attemp to argue anout dependencies. On Linux there are some times multiple ways how to solve some problem. We will need to choose for Docker image while leaving some freedom to distribution owners.

[Petermarcu]

@wfurt @karelz it sounds like we've concluded this is not an issue that is going to be addressed in the native installers. Is there a follow up for our containers or should this be closed?

[karelz]

It would be a nice thing to have in the Docker image for ease of use. However, it is not strictly needed. NTLM users can install it explicitly.
I am fine closing it as I suggested above.

[Petermarcu]

Do we need to document the dependency anywhere? Is there a good NTLM doc or the like?

[ayg-qliktech]

Hi @MichaelSimons, what is the status for NTLM support for the Alpine dotnetcore base images? I see you asked for guidance for Alpine distros, but didn't see a response or any further movement here or elsewhere. We are in a situation where we need NTLM support for work project and really don't want to move to Debian/Ubuntu (less secure/heavier images) for this support. We have been on Alpine for a while and really happy with the results thus far. Any help, guidance, or working examples of where this is done for Alpine is greatly appreciated!

[wfurt]

I did not see any pre-built Alpine package. You best bet is to build it your self @ayg-qliktech . I did it long time ago and the code is small and it should not give you any grief. https://github.com/simo5/gss-ntlmssp

[ayg-qliktech]

@wfurt: Will give this a try. Thank you!

[jruizx]

I did not see any pre-built Alpine package. You best bet is to build it your self @ayg-qliktech . I did it long time ago and the code is small and it should not give you any grief. https://github.com/simo5/gss-ntlmssp

Hi @wfurt, could you tell me how to build the package?

[wfurt]

https://github.com/simo5/gss-ntlmssp/blob/master/BUILD.txt
I think hardest part will be finding matching Alpine dependency packages.
Maybe @ayg-qliktech succeeded and may have list.
You can also use ./configure --without-wbclient --without-manpages to limit dependencies.
After that make install puts everything in place - without building package.

[mikeTWC1984]

Looks like building/installing it manually doesn't take any effect on alpine. I tried to enable ntlm for powershell's Invoke-WebRequest, and it didn't help (although installing gss-ntlmspp makes it work on ubuntu/centos).
Another strange thing - installing krb5 on alpine enables ntlm for curl (without building gss-ntlmspp). Shouldn't it be enough for .net too?

curl $myNtlmUrl --ntlm -u myUser:myPasswor  # that works after installing krb5
Invoke-WebRequest $myNtlmUrl -Credential (get-credential myUser) # powershell - doesn't work, even after building ntlmspp

below is a Dockerfile you can use to build/test. FYI - it only works on alpine 3.11 or higher

FROM mcr.microsoft.com/powershell:preview-alpine-3.11
RUN apk add --no-cache git curl tini util-linux bash neovim openssl
RUN apk add --no-cache make m4 autoconf automake gcc g++ krb5-dev openssl-dev gettext-dev  
RUN apk add --no-cache libtool libxml2 libxslt libunistring-dev zlib-dev samba-dev

RUN git clone https://github.com/gssapi/gss-ntlmssp
WORKDIR gss-ntlmssp
RUN autoreconf -f -i
RUN ./configure --without-manpages --disable-nls
RUN make 
RUN make install

I believe to set up plugin you need to copy mech.ntlmspp (from examples folder) to /etc/gss/mech.d/ and update it setting path to .so file generated during install (/usr/local/lib/gssntlmssp/gssntlmssp.so). In centos this file is called ntlmssp.conf (I tried it too, none of that make powershell work).

[wfurt]

AFAIK Curl has independent NTLN implementation and it does not need the GSSAPI. And yes, the plugin needs to register it self. .NET does not use KRB or NTLM directly. I would expect this would be done part oof the install but maybe not.
Can you build and run https://github.com/davidsh/NetworkingTests/tree/master/gssapitests ?

[mikeTWC1984]

@wfurt Just tried, ntlm test failed indeed. Ok, I guess something is missing here, I'll keep experimenting.

Installed GSSAPI mechanisms:
KRB5 - Kerberos 5 (1.2.840.113554.1.2.2)
Legacy Kerberos 5 (1.3.6.1.5.2.5)
SPNEGO - Simple Protected Negotiation (1.3.6.1.5.5.2)

NTLM installed: no

[mikeTWC1984]

I ran same steps on plain ubuntu image and it worked. So once /usr/local/lib/gssntlmssp/gssntlmssp.so is generated adding reference to it in /etc/gss/mech.d/mech.ntlmspp.conf makes it work. By some reason it's not happening on Alpine. When I build ntlmssp on alpine it also generates it's own tests, and those are passing unlike https://github.com/davidsh/NetworkingTests/tree/master/gssapitests (Is it possible it's not running correctly on Alpine?)

[wfurt]

That test uses same API as .NET. I don't know how gssntlmssp tests are structured and if they use gssapi or just call functions from the library directly. Is it possible to that location is different on Alpine?
You can check with strace to see if the mech.ntlmssp.conf is being used.
I can give it try later this week if I find some cycles.

[mikeTWC1984]

@wfurt Thanks! using strace I find out that proper config file path is
/usr/etc/gss/mech.d/mech.ntlmssp.conf
Once config is there your test is passing and Invoke-WebRequest is working

[wfurt]

good to know. That may depend on the --prefix when you build the packages. Can you leave some summary notes @mikeTWC1984 for anybody why may try to follow your steps?

[Gladskih]

@mikeTWC1984 Could you post resulting Dockerfile please?
I think here should be multistage one to prevent dev dependencies be baked in app image.
P.S. I'm glad not to be the only one who faced the problem right now.

[mikeTWC1984]

@Gladskih The above Dockerfile is good to generate library file (gssntlmssp.so) which you can copy to other containers/images (to avoid installing dev dependencies). After executing make, this file should be either in .libs or /usr/local/lib/gssntlmssp/ folders.
I put step by step instruction here:
https://github.com/mikeTWC1984/gssntlm
I prebuild this library file too, so you can use above repo to install ntlm on your test containers (until official alpine package is out there)

[GitMonkey007]

Sorry to comment on a closed issue, but could I suggest revisiting this for docker scenarios? That is, please could you add gss-ntlmssp to runtime-deps packages in docker images?

.Net Core supports a default proxy (based on the environment variables as defined here), but if you use the environment variables and then create an HttpClient that reaches outside a corporate network (where presumably the corporate proxy requires NTLM), then the code works on Windows, but as soon as you put the code into a container (note, this is a Linux container, with a Win32Exception raised), you immediately fail with:

System.ComponentModel.Win32Exception (0x80090020): GSSAPI operation failed with error - An invalid name was supplied (Configuration file does not specify default realm).

I can't find any documentation about solving this (maybe my google-foo is lacking, but I have looked) without resorting to adding the environment variable DOTNET_SYSTEM_NET_HTTP_USESOCKETSHTTPHANDLER=false, (or setting AppContext.SetSwitch("System.Net.Http.UseSocketsHttpHandler", false);, which seems to be the same as the environment variable) or add the package to a container, but I think this package should be part of the runtime dependencies such that it "just works" out of the box...

The docker image I'm using is the dotnet/core/runtime:3.1.8 image.

[wfurt]

For your case @GitMonkey007, you need to use dotnet/core/runtime:3.1.8 as base and extend it with the extra package instead of using the dotnet/core/runtime:3.1.8 image directly.

[GitMonkey007]

@wfurt Thanks - That's what I'm doing for now

...but surely this should be part of the runtime dependencies in the base image (this issue?), so that it works as expected in Docker scenarios... (and doesn't trip people like me up)?

I'd be fine with not including it in the alpine images (by default) where you want the smallest possible image, but shouldn't it be part of the standard image (ie, to provide widest compatibility?)?

I won't continue to argue the case, but if it is just a question of who will do it, I'll look into it and try to submit a PR (if you think it'll have any chance of being accepted?)

[MichaelSimons]

@GitMonkey007, We have discussed this in the past and are not willing to add gss-ntlmssp to the base .NET images. The package is not considered a core dependency which is why it isn't a dependency of the NET Debian packages. Adding all of the dependencies for all scenarios would add to the size of the base images and increase the attack surface area. Our general philosophy is to make the .NET images a foundation that users can add on the components/dependencies required for their scenarios.