Mailkit: NTLM authentication not working

[ravenpride]

Hello,

the issue occurs with version MailKit v1.18.1.1 (pulled via Nuget).

I've tried to get MailKit working with our Exchange Server. MailKit successfully connected to port 587 and established a secure channel using STARTTLS. After authenticating the Exchange server offered the mechanisms GSSAPI, NTLM and LOGIN.

I've removed GSSAPI and LOGIN from the authentication mechanisms MailKit may use:

client.AuthenticationMechanisms.Remove("GSSAPI");
client.AuthenticationMechanisms.Remove("LOGIN");

Only NTLM is left in the list of authentication mechanisms.

Trying to authenticate using...

client.Authenticate(CredentialCache.DefaultCredentials);

...throws the following exception

 --- Exception ---------------------------------------------------------------------------------------------
 --- Exception Type: System.NotSupportedException
 --- Message: No compatible authentication mechanisms found.
 --- Stacktrace:
    bei MailKit.Net.Smtp.SmtpClient.Authenticate(Encoding encoding, ICredentials credentials, CancellationToken cancellationToken)
    bei MailKit.MailService.Authenticate(ICredentials credentials, CancellationToken cancellationToken)
    ...

Here comes the log of the SMTP session:

S: 220 Elexis-Exch02.elexisad.net Microsoft ESMTP MAIL Service ready at Fri, 27 Oct 2017 13:08:11 +0200
C: EHLO [10.14.1.175]
S: 250-Elexis-Exch02.elexisad.net Hello [10.14.1.175]
S: 250-SIZE 52428800
S: 250-PIPELINING
S: 250-DSN
S: 250-ENHANCEDSTATUSCODES
S: 250-STARTTLS
S: 250-AUTH GSSAPI NTLM
S: 250-8BITMIME
S: 250-BINARYMIME
S: 250 CHUNKING
C: STARTTLS
S: 220 2.0.0 SMTP server ready
C: EHLO [10.14.1.175]
S: 250-Elexis-Exch02.elexisad.net Hello [10.14.1.175]
S: 250-SIZE 52428800
S: 250-PIPELINING
S: 250-DSN
S: 250-ENHANCEDSTATUSCODES
S: 250-AUTH GSSAPI NTLM LOGIN
S: 250-8BITMIME
S: 250-BINARYMIME
S: 250 CHUNKING

I hope this helps you to track the issue down.

Thank you in advance!

[jstedfast]

MailKit 1.18 disabled NTLM authentication because I was getting too many bug reports about it not working and no way to diagnose the problems (because it worked just fine for me).

This is why you are getting NotSupportedException.

Just use LOGIN and it should work fine.

[PeterHagen]

Does this mean that NTLM won't be supported anymore?

[jstedfast]

It's not in the list of SASL mechanisms that are tried by default, but starting with 2.0, you can use it manually:

var ntlm = new SaslMechanismNtlm ("username", "password");
client.Authenticate (ntlm);

[PeterHagen]

Great! Thats seems to work!

[alex-jitbit]

Just switched from .NET builtin SmtpClient (which was working fine) to MailKit and several customers are reporting this problem now. How can I make MailKit to also try using NTLM among other methods?

[jstedfast]

@alex-jitbit

if (client.AuthenticationMechanisms.Contains ("NTLM")) {
    var ntlm = new SaslMechanismNtlm ("username", "password");
    client.Authenticate (ntlm);
} else {
    // use the default supported mechanisms
    client.Authenticate ("username", "password");
}

[alex-jitbit]

@jstedfast thank you sir

[pawanbhabad]

Hello,
I have recently switched to Mailkit 2.5.0.0. Earlier I was using "System.net.mail.SmtpClient".
In case of NTLM authentication, only authentication supported by SMTP relay, client application is working with SmtpClient but with Mailkit it's not working (error: Authentication unsuccessful), even though the credentials provided are also same and same configuration.

I have tried using code mentioned above,

var ntlm = new SaslMechanismNtlm ("username", "password");
client.Authenticate (ntlm);

but same error.

i.e. username and password are windows login credentials.

Sharing Protocol logs,

Connected to smtp://pawan_dev.vxcountry.cinemax.com:25/?starttls=when-available
S: 220 Pawan_Dev.vxcountry.cinemax.com Microsoft ESMTP MAIL Service, Version: 8.5.9600.16384 ready at Thu, 5 Nov 2020 12:37:21 +0530
C: EHLO [10.217.79.102]
S: 250-Pawan_Dev.vxcountry.cinemax.com Hello [10.217.79.102]
S: 250-AUTH NTLM
S: 250-TURN
S: 250-SIZE 2097152
S: 250-ETRN
S: 250-PIPELINING
S: 250-DSN
S: 250-ENHANCEDSTATUSCODES
S: 250-8bitmime
S: 250-BINARYMIME
S: 250-CHUNKING
S: 250-VRFY
S: 250 OK
C: AUTH NTLM TlRMTVNTUAABAAAABxIAAAkACQAgAAAAAAAAACAAAABDT01NVU5JVFk=
S: 334 TlRMTVNTUAACAAAADgAOADgAAAAFAoECMR2TRr16f68AAAAAAAAAAMoAygBGAAAABgOAJQAAAA9WAFgASQBOAEQASQBBAAIADgBWAFgASQBOAEQASQBBAAEAEgBQAEEAVwBBAE4AXwBEAEUAVgAEACYAdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQADADoAUABhAHcAYQBuAF8ARABlAHYALgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAUAJgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAcACAAq475QQrPWAQAAAAA=
C: TlRMTVNTUAADAAAAAAAAAG4AAAD2APYAbgAAAA4ADgBIAAAAGAAYAFYAAAAAAAAAbgAAAAAAAABkAQAAAQIAAgoAWikAAAAPVgBYAEkATgBEAEkAQQBwAGEAdwBhAG4ALgBiAGgAYQBiAGEAZADczddHWVbSM1KTMjMJgPsWAQEAAAAAAADrAHxqcLPWAa0J73JjwXLDAAAAAAIADgBWAFgASQBOAEQASQBBAAEAEgBQAEEAVwBBAE4AXwBEAEUAVgAEACYAdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQADADoAUABhAHcAYQBuAF8ARABlAHYALgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAUAJgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAcACAAq475QQrPWAQAAAAA=
S: 535 5.7.3 Authentication unsuccessful

Does Mailkit supports NTLM authentication ?

Please suggest me some work-around for this issue.

Thanks!

[sauravcaptech]

@pawanbhabad I am also facing similar issue
I have 3 different SMTP supporting NTLM auth mechanism . It works in one case but fails in other two
Attaching the logs . It will be good if someone can see this and make some sense

Case 1 : Successful

Connection

Connected to smtp://mail.celebritygroup.com:125/
S: 220 CFLEXHC.celebritygroup.int Microsoft ESMTP MAIL Service ready at Thu, 5 Nov 2020 17:30:43 +0530
C: EHLO [10.0.2.15]
S: 250-CFLEXHC.celebritygroup.int Hello [223.187.190.219]
S: 250-SIZE 20971520
S: 250-PIPELINING
S: 250-DSN
S: 250-ENHANCEDSTATUSCODES
S: 250-X-ANONYMOUSTLS
S: 250-AUTH NTLM LOGIN
S: 250-X-EXPS GSSAPI NTLM
S: 250-8BITMIME
S: 250-BINARYMIME
S: 250-CHUNKING
S: 250-XEXCH50
S: 250-XRDST
S: 250 XSHADOW

Auth

C: AUTH NTLM TlRMTVNTUAABAAAABwIAAAAAAAAgAAAAAAAAACAAAAA=
S: 334 TlRMTVNTUAACAAAAHAAcADgAAAAFAoECu0dkY4fmB3sAAAAAAAAAAMoAygBUAAAABgGxHQAAAA9DAEUATABFAEIAUgBJAFQAWQBHAFIATwBVAFAAAgAcAEMARQBMAEUAQgBSAEkAVABZAEcAUgBPAFUAUAABAA4AQwBGAEwARQBYAEgAQwAEACQAYwBlAGwAZQBiAHIAaQB0AHkAZwByAG8AdQBwAC4AaQBuAHQAAwA0AEMARgBMAEUAWABIAEMALgBjAGUAbABlAGIAcgBpAHQAeQBnAHIAbwB1AHAALgBpAG4AdAAFACQAYwBlAGwAZQBiAHIAaQB0AHkAZwByAG8AdQBwAC4AaQBuAHQABwAIAHwg/n1rs9YBAAAAAA==
C: TlRMTVNTUAADAAAAAAAAAHIAAAD2APYAcgAAABwAHABIAAAADgAOAGQAAAAAAAAAcgAAAAAAAABoAQAAAQIAAgoAWikAAAAPQwBFAEwARQBCAFIASQBUAFkARwBSAE8AVQBQAHMAdQBwAHAAbwByAHQAVP3M3RD5KXArTOqiHfvIqwEBAAAAAAAAuZjI1Zmz1gGprENbDyd/EAAAAAACABwAQwBFAEwARQBCAFIASQBUAFkARwBSAE8AVQBQAAEADgBDAEYATABFAFgASABDAAQAJABjAGUAbABlAGIAcgBpAHQAeQBnAHIAbwB1AHAALgBpAG4AdAADADQAQwBGAEwARQBYAEgAQwAuAGMAZQBsAGUAYgByAGkAdAB5AGcAcgBvAHUAcAAuAGkAbgB0AAUAJABjAGUAbABlAGIAcgBpAHQAeQBnAHIAbwB1AHAALgBpAG4AdAAHAAgAfCD+fWuz1gEAAAAA
S: 235 2.7.0 Authentication successful

Case 2 : Failed

Connection

Connected to smtp://webmail.acilogistics.net:25/
S: 220 MAIL01.acilogistics.net Microsoft ESMTP MAIL Service ready at Thu, 5 Nov 2020 18:26:25 +0600
C: EHLO [10.0.2.15]
S: 250-MAIL01.acilogistics.net Hello [223.187.190.219]
S: 250-SIZE 26214400
S: 250-PIPELINING
S: 250-DSN
S: 250-ENHANCEDSTATUSCODES
S: 250-X-ANONYMOUSTLS
S: 250-AUTH NTLM
S: 250-X-EXPS GSSAPI NTLM
S: 250-8BITMIME
S: 250-BINARYMIME
S: 250-CHUNKING
S: 250 XRDST

Auth

C: AUTH NTLM TlRMTVNTUAABAAAABwIAAAAAAAAgAAAAAAAAACAAAAA=
S: 334 TlRMTVNTUAACAAAAGAAYADgAAAAFAoECePSzTnORDAIAAAAAAAAAALYAtgBQAAAABgOAJQAAAA9BAEMASQBMAE8ARwBJAFMAVABJAEMAUwACABgAQQBDAEkATABPAEcASQBTAFQASQBDAFMAAQAMAE0AQQBJAEwAMAAxAAQAIABhAGMAaQBsAG8AZwBpAHMAdABpAGMAcwAuAG4AZQB0AAMALgBNAEEASQBMADAAMQAuAGEAYwBpAGwAbwBnAGkAcwB0AGkAYwBzAC4AbgBlAHQABQAgAGEAYwBpAGwAbwBnAGkAcwB0AGkAYwBzAC4AbgBlAHQABwAIAN1owiVvs9YBAAAAAA==
C: TlRMTVNTUAADAAAAAAAAAIwAAADiAOIAjAAAABgAGABIAAAALAAsAGAAAAAAAAAAjAAAAAAAAABuAQAAAQIAAgoAWikAAAAPQQBDAEkATABPAEcASQBTAFQASQBDAFMAaABlAGwAbABvAEAAYQBjAGkAbABvAGcAaQBzAHQAaQBjAHMALgBuAGUAdACanU+zhIL8LmEFAG6pwJjKAQEAAAAAAAA/LPYum7PWAXjXpkDHlwTBAAAAAAIAGABBAEMASQBMAE8ARwBJAFMAVABJAEMAUwABAAwATQBBAEkATAAwADEABAAgAGEAYwBpAGwAbwBnAGkAcwB0AGkAYwBzAC4AbgBlAHQAAwAuAE0AQQBJAEwAMAAxAC4AYQBjAGkAbABvAGcAaQBzAHQAaQBjAHMALgBuAGUAdAAFACAAYQBjAGkAbABvAGcAaQBzAHQAaQBjAHMALgBuAGUAdAAHAAgA3WjCJW+z1gEAAAAA
S: 535 5.7.3 Authentication unsuccessful

Case 3 : Failed

Connection

Connected to smtp://mail.ajlanbros.com:587/?starttls=always
S: 220 AJL-EXCH-MBX-01.ajlanbros.com Microsoft ESMTP MAIL Service ready at Thu, 5 Nov 2020 16:20:48 +0300
C: EHLO [10.0.2.15]
S: 250-AJL-EXCH-MBX-01.ajlanbros.com Hello [223.187.190.219]
S: 250-SIZE 37748736
S: 250-PIPELINING
S: 250-DSN
S: 250-ENHANCEDSTATUSCODES
S: 250-STARTTLS
S: 250-AUTH GSSAPI NTLM
S: 250-8BITMIME
S: 250-BINARYMIME
S: 250 CHUNKING
C: STARTTLS
S: 220 2.0.0 SMTP server ready
C: EHLO [10.0.2.15]
S: 250-AJL-EXCH-MBX-01.ajlanbros.com Hello [223.187.190.219]
S: 250-SIZE 37748736
S: 250-PIPELINING
S: 250-DSN
S: 250-ENHANCEDSTATUSCODES
S: 250-AUTH GSSAPI NTLM LOGIN
S: 250-8BITMIME
S: 250-BINARYMIME
S: 250 CHUNKING

Auth

C: AUTH NTLM TlRMTVNTUAABAAAABwIAAAAAAAAgAAAAAAAAACAAAAA=
S: 334 TlRMTVNTUAACAAAAFAAUADgAAAAFAoECOJhxa2h9hIkAAAAAAAAAAMQAxABMAAAACgA5OAAAAA9BAEoATABBAE4AQgBSAE8AUwAxAAIAFABBAEoATABBAE4AQgBSAE8AUwAxAAEAHgBBAEoATAAtAEUAWABDAEgALQBNAEIAWAAtADAAMQAEABoAYQBqAGwAYQBuAGIAcgBvAHMALgBjAG8AbQADADoAQQBKAEwALQBFAFgAQwBIAC0ATQBCAFgALQAwADEALgBhAGoAbABhAG4AYgByAG8AcwAuAGMAbwBtAAUAGgBhAGoAbABhAG4AYgByAG8AcwAuAGMAbwBtAAcACADNUKindrPWAQAAAAA=
C: TlRMTVNTUAADAAAAAAAAAIgAAADwAPAAiAAAABQAFABIAAAALAAsAFwAAAAAAAAAiAAAAAAAAAB4AQAAAQIAAgoAWikAAAAPQQBKAEwAQQBOAEIAUgBPAFMAMQBzAGgAbwBwAHAAaQBuAGcAQABhAGoAbABhAG4AYgByAG8AcwAuAGMAbwBtAI/CxjSX3C49u/3j2ovZNhkBAQAAAAAAABV7VcKks9YB+IhndHHCeSoAAAAAAgAUAEEASgBMAEEATgBCAFIATwBTADEAAQAeAEEASgBMAC0ARQBYAEMASAAtAE0AQgBYAC0AMAAxAAQAGgBhAGoAbABhAG4AYgByAG8AcwAuAGMAbwBtAAMAOgBBAEoATAAtAEUAWABDAEgALQBNAEIAWAAtADAAMQAuAGEAagBsAGEAbgBiAHIAbwBzAC4AYwBvAG0ABQAaAGEAagBsAGEAbgBiAHIAbwBzAC4AYwBvAG0ABwAIAM1QqKd2s9YBAAAAAA==
S: 535 5.7.3 Authentication unsuccessful

[jstedfast]

I don't suppose @sauravcaptech or @pawanbhabad can get me a log of some other SMTP client/library successfully authenticating via NTLM so that I can compare what MailKit is sending to what a successful authentication is sending?

I'm stepping thru the debugger comparing @sauravcaptech's successful vs failed NTLM exchanges and I can't tell why one works and the others don't. In all cases, the exact same NTLM flags are used by the server (and MailKit). The only differences (other than usernames and passwords) are the domain names provided by the server (and used in the final authentication request to the server). That's it.

[pawanbhabad]

@sauravcaptech As per your logs, it seems, the failure is definitely because of NTLM AUTH, i.e. case 2 & 3.
First case is successful due to LOGIN AUTH mechanism supported by server. Though there is NTLM present but not used by server.
In later cases, AUTH mechanism supported is only NTLM, so it's failing (AUTH GSSAPI also fails). So you are landing into same issue what I am facing.

[jstedfast]

No, all of @sauravcaptech's authentication examples use NTLM for authenticating (even the successful one).

As you noted, however, in the case where the NTLM authentication was successful, the server also supported the LOGIN SASL mechanism.

That should be irrelevant, though, because these authentication mechanisms do not inter-relate.

[pawanbhabad]

Hello @jstedfast
As you have asked for the working API logs for comparison. Hope, this may help!
I am using same credentials with Mailkit.

Please have a look into the following logs, using "System.net.mail.SmtpClient" API, email is sent successfully.

S: 220 PNV86SMTP01.PNE.VEN.CINEMAX.COM ESMTP Sendmail 8.14.4/8.14.4; Mon, 9 Nov 2020 14:30:43 +0530
C: EHLO Pawan_Dev.vxcountry.cinemax.com
S: 250-PNV86SMTP01.PNE.VEN.CINEMAX.COM Hello [10.217.79.102], pleased to meet you | ENHANCEDSTATUSCODES | PIPELINING | 8BITMIME | SIZE | DSN | ETRN | AUTH GSSAPI DIGEST-MD5 CRAM-MD5 | DELIVERBY | HELP
C: MAIL FROM:[email protected]
S: 250 2.1.0 pawan.[email protected]... Sender ok
C: RCPT TO:pawan.[email protected]
S: 250 2.1.5 pawan.[email protected]... Recipient ok
C: DATA
S: 354 Enter mail, end with "." on a line by itself

S: 250 2.0.0 0A990hmD059588 Message accepted for delivery
C: QUIT
S: 221 2.0.0 PNV86SMTP01.PNE.VEN.CINEMAX.COM closing connection

Logs using Mailkit;

Connected to smtp://pawan_dev.vxcountry.cinemax.com:25/?starttls=when-available
S: 220 Pawan_Dev.vxcountry.cinemax.com Microsoft ESMTP MAIL Service, Version: 8.5.9600.16384 ready at Thu, 5 Nov 2020 12:37:21 +0530
C: EHLO [10.217.79.102]
S: 250-Pawan_Dev.vxcountry.cinemax.com Hello [10.217.79.102]
S: 250-AUTH NTLM
S: 250-TURN
S: 250-SIZE 2097152
S: 250-ETRN
S: 250-PIPELINING
S: 250-DSN
S: 250-ENHANCEDSTATUSCODES
S: 250-8bitmime
S: 250-BINARYMIME
S: 250-CHUNKING
S: 250-VRFY
S: 250 OK
C: AUTH NTLM TlRMTVNTUAABAAAABxIAAAkACQAgAAAAAAAAACAAAABDT01NVU5JVFk=
S: 334 TlRMTVNTUAACAAAADgAOADgAAAAFAoECMR2TRr16f68AAAAAAAAAAMoAygBGAAAABgOAJQAAAA9WAFgASQBOAEQASQBBAAIADgBWAFgASQBOAEQASQBBAAEAEgBQAEEAVwBBAE4AXwBEAEUAVgAEACYAdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQADADoAUABhAHcAYQBuAF8ARABlAHYALgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAUAJgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAcACAAqrPWAQAAAA=
C: TlRMTVNTUAADAAAAAAAAAG4AAAD2APYAbgAAAA4ADgBIAAAAGAAYAFYAAAAAAAAAbgAAAAAAAABkAQAAAQIAAgoAWikAAAAPVgBYAEkATgBEAEkAQQBwAGEAdwBhAG4ALgBiAGgAYQBiAGEAZADczddHWVbSM1KTMjMJgPsWAQEAAAAAAADrAHxqcLPWAa0J73JjwXLDAAAAAAIADgBWAFgASQBOAEQASQBBAAEAEgBQAEEAVwBBAE4AXwBEAEUAVgAEACYAdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQADADoAUABhAHcAYQBuAF8ARABlAHYALgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAUAJgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAcACAAqrPWAQAAAAA=
S: 535 5.7.3 Authentication unsuccessful

[jstedfast]

@pawanbhabad

It looks like you connected to 2 different SMTP servers.

In the case of System.Net.Mail, you used a Sendmail 8.14 server while in MailKit's case, you used a Microsoft Exchange ESMTP server. In the System.Net.Mail case, you also did not authenticate (well, the System.Net.Mail SmtpClient did not authenticate even if you gave it credentials to use).

[pawanbhabad]

@jstedfast
I am using Windows Server 2012 R2 SMTP relay, where, only NTLM AUTH is enabled (in Access control authentication) and then it is connected to Office 365 as "Anonymous access" AUTH (in Outbound security).
I am trying to send email to this SMTP relay via both the APIs (i.e. SMTPClient and Mailkit) with same authentication credentials.
Corresponding logs are generated.

Also, in context to your second point, when I provide incorrect credentials to SMTPClient API OR no credentials, then it does not work. So I think SMTPClient authenticates with an appropriate credentials.

[jstedfast]

What are you using to get the logs for System.Net.Mail's SmtpClient? Perhaps it stripped out the authentication commands? In any event, the Sendmail 8.14 server does not support NTLM, so System.Net.Mail probably used GSSAPI or CRAM-MD5 (I don't think System.Net.Mail supports DIGEST-MD5).

[pawanbhabad]

Using WireShark to get logs.
Initially, the client should connect "Pawan_Dev.vxcountry.cinemax.com", as it is a SMTP relay host then it should try for "PNV86SMTP01.PNE.VEN.CINEMAX.COM" (Office 365).
Server "PNV86SMTP01.PNE.VEN.CINEMAX.COM" does not support NTLM, as you said. Only SMTP relay "Pawan_Dev.vxcountry.cinemax.com" supports NTLM AUTH.
So I think the problem lies with authenticating "Pawan_Dev.vxcountry.cinemax.com" SMTP relay server.

While capturing logs, the authentication commands are not captured. Shared logs, what all got generated from Wireshark.

[jstedfast]

Unfortunately, the bit of the logs that I need are the authentication commands

[jstedfast]

@pawanbhabad and @sauravcaptech if you guys can find a way to get me the logs of a successful NTLM authentication to a server where MailKit fails (via NTLM), please open a new bug.

I'm going to close this one because the original issue is resolved (and was really a question rather than a bug).

I'm definitely interested in fixing bugs in NTLM if I can, I just need info that unfortunately I don't have.

Basically, what I want to do, is to compare what MailKit is sending with what a working implementation is sending in your cases, this way I can hopefully figure out what MailKit is doing wrong.

[pawanbhabad]

Hello @jstedfast
I was trying to create logs of successful NTLM authentication to a server by using System.Net.Mail.SMTPClient, as it was not available earlier. Now here I am able to create it. Hope, this may be useful for comparing NTML authentications in both the cases, Mailkit and SMTPClient.

SMTP Relay server : "Pawan_Dev.vxcountry.cinemax.com".
SMTP Server : "PNV86SMTP01.PNE.VEN.CINEMAX.COM".

I am using same credentials in both the cases.

Below are the logs shared for both APIs;

Mailkit logs

Connected to smtp://10.217.79.102:25/?starttls=when-available
S: 220 Pawan_Dev.vxcountry.cinemax.com Microsoft ESMTP MAIL Service, Version: 8.5.9600.16384 ready at Mon, 23 Nov 2020 16:02:54 +0530
C: EHLO [10.217.79.102]
S: 250-Pawan_Dev.vxcountry.cinemax.com Hello [10.217.79.102]
S: 250-AUTH NTLM
S: 250-TURN
S: 250-SIZE 2097152
S: 250-ETRN
S: 250-PIPELINING
S: 250-DSN
S: 250-ENHANCEDSTATUSCODES
S: 250-8bitmime
S: 250-BINARYMIME
S: 250-CHUNKING
S: 250-VRFY
S: 250 OK
C: AUTH NTLM TlRMTVNTUAABAAAABwIAAAAAAAAgAAAAAAAAACAAAAA=
S: 334 TlRMTVNTUAACAAAADgAOADgAAAAFAoECONBRvRlwC2wAAAAAAAAAAMoAygBGAAAABgOAJQAAAA9WAFgASQBOAEQASQBBAAIADgBWAFgASQBOAEQASQBBAAEAEgBQAEEAVwBBAE4AXwBEAEUAVgAEACYAdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQADADoAUABhAHcAYQBuAF8ARABlAHYALgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAUAJgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAcACACfXLgGhMHWAQAAAAA=
C: TlRMTVNTUAADAAAAAAAAAG4AAAD2APYAbgAAAA4ADgBIAAAAGAAYAFYAAAAAAAAAbgAAAAAAAABkAQAAAQIAAgoAWikAAAAPVgBYAEkATgBEAEkAQQBwAGEAdwBhAG4ALgBiAGgAYQBiAGEAZAASQHh9Lw9LhW2jR1P7QTfiAQEAAAAAAACsfnUgssHWAQ87rKz/7CSbAAAAAAIADgBWAFgASQBOAEQASQBBAAEAEgBQAEEAVwBBAE4AXwBEAEUAVgAEACYAdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQADADoAUABhAHcAYQBuAF8ARABlAHYALgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAUAJgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAcACACfXLgGhMHWAQAAAAA=
S: 535 5.7.3 Authentication unsuccessful
C: MAIL FROM:[e-mail] SIZE=367
C: RCPT TO:[e-mail]
S: 530 5.7.3 Client was not authenticated
S: 530 5.7.3 Client was not authenticated
C: QUIT
S: 221 2.0.0 Pawan_Dev.vxcountry.cinemax.com Service closing transmission channel

SMTPClient logs

S: 220 Pawan_Dev.vxcountry.cinemax.com Microsoft ESMTP MAIL Service, Version: 8.5.9600.16384 ready at Mon, 23 Nov 2020 15:20:40 +0530
C: EHLO W3115570Q81SG
S: 250-Pawan_Dev.vxcountry.cinemax.com Hello [10.217.217.53] | AUTH NTLM | TURN | SIZE 2097152 | ETRN | PIPELINING | DSN | ENHANCEDSTATUSCODES | 8bitmime | BINARYMIME | CHUNKING | VRFY | OK
C: AUTH ntlm TlRMTVNTUAABAAAAB4IIogAAAAAAAAAAAAAAAAAAAAAKAO5CAAAADw==
S: 334 TlRMTVNTUAACAAAADgAOADgAAAAFgomi18THmUUjMM4AAAAAAAAAAMoAygBGAAAABgOAJQAAAA9WAFgASQBOAEQASQBBAAIADgBWAFgASQBOAEQASQBBAAEAEgBQAEEAVwBBAE4AXwBEAEUAVgAEACYAdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQADADoAUABhAHcAYQBuAF8ARABlAHYALgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAUAJgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAcACABL1oYafsHWAQAAAAA=

C: TlRMTVNTUAADAAAAGAAYAIoAAAB4AXgBogAAAAAAAABYAAAAGAAYAFgAAAAaABoAcAAAAAAAAAAaAgAABYKIogoA7kIAAAAP1t31DYB66IvmtXjlDBp3jXAAYQB3AGEAbgAuAGIAaABhAGIAYQBkAFcAMwAxADEANQA1ADcAMABRADgAMQBTAEcAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABo9OFnsse8fOpnPtN+EODgEBAAAAAAAAS9aGGn7B1gENlOJzBTEX/AAAAAACAA4AVgBYAEkATgBEAEkAQQABABIAUABBAFcAQQBOAF8ARABFAFYABAAmAHYAeABpAG4AZABpAGEALgB2AGUAcgBpAHQAYQBzAC4AYwBvAG0AAwA6AFAAYQB3AGEAbgBfAEQAZQB2AC4AdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQAFACYAdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQAHAAgAS9aGGn7B1gEGAAQAAgAAAAgAMAAwAAAAAAAAAAEAAAAAIAAAQb6NkFRsPXqeapbOErEdQ9DC3EFPhyIR29iV0T05QSIKABAAAAAAAAAAAAAAAAAAAAAAAAkAKgBTAE0AVABQAFMAVgBDAC8AMQAwAC4AMgAxADcALgA3ADkALgAxADAAMgAAAAAAAAAAAA==

S: 235 2.7.0 Authentication successful
C: MAIL FROM:[e-mail]
S: 250 2.1.0 pawan.[email protected] OK
C: RCPT TO:[e-mail]
S: 250 2.1.5 pawan.[email protected]
C: DATA
S: 354 Start mail input; end with .
C: DATA fragment, 235 bytes
C: DATA fragment, 41 bytes
from: pawan.[email protected], subject: Trial email subject, (text/html)
S: 250 2.6.0 PAWAN_DEVFRaqbC8wSA00000027@Pawan_Dev.vxcountry.cinemax.com Queued mail for delivery
S: 220 PNV86SMTP01.PNE.VEN.CINEMAX.COM ESMTP Sendmail 8.14.4/8.14.4; Mon, 23 Nov 2020 15:20:41 +0530
C: EHLO Pawan_Dev.vxcountry.cinemax.com
S: 250-PNV86SMTP01.PNE.VEN.CINEMAX.COM Hello [10.217.79.102], pleased to meet you | ENHANCEDSTATUSCODES | PIPELINING | 8BITMIME | SIZE | DSN | ETRN | AUTH GSSAPI DIGEST-MD5 CRAM-MD5 | DELIVERBY | HELP
C: MAIL FROM:[e-mail]
S: 250 2.1.0 [e-mail]... Sender ok
C: RCPT TO:[e-mail]
S: 250 2.1.5 [e-mail]... Recipient ok
C: DATA
S: 354 Enter mail, end with "." on a line by itself
from: pawan.[email protected], subject: Trial email subject, (text/html)
S: 250 2.0.0 0AN9ofCe026795 Message accepted for delivery
C: QUIT
S: 221 2.0.0 PNV86SMTP01.PNE.VEN.CINEMAX.COM closing connection
C: QUIT
S: 221 2.0.0 Pawan_Dev.vxcountry.cinemax.com Service closing transmission channel

[jstedfast]

@pawanbhabad it looks like it will help, thanks.

[jstedfast]

@pawanbhabad

I sent you an email earlier detailing the differences that I found between MailKit and System.Net.Mail.

I did a bit of reading of the NTLM docs and have spotted 2 potential problems in MailKit's code that I have "fixed", but I am not 100% confident in either of the fixes.

What I need you to do is to test the NuGet package located at https://www.myget.org/feed/mimekit/package/nuget/MailKit/2.10.0.4

Once you've switched to using the above NuGet package, what I want you to do is to test the following cases for me:

1. According to the docs, that 24-byte array of null bytes is supposed to be sent if the server's challenge includes a TargetInfo with a timestamp. In your case, the server is sending a timestamp and as I mentioned in the email, System.Net.Mail is sending this 24-byte array of null bytes.

To enable it, do the following:

var ntlm = new SaslMechanismNtlm (username, password) {
    NtlmFixes = NtlmFixes.NTLMv2IncludeZ24
};

// The following code enables the NTLMSSP_NEGOTIATE_VERSION debugging info and
// System.Net.Mail seems to include it, so I've made it possible to include as well.
if (Environment.OSVersion.Platform == PlatformID.Win32NT)
    ntlm.OSVersion = Environment.OSVersion.Version;

1. According to the docs, it sounds like the NTLMv2 password hashing/encryption logic is supposed to use the timestamp included in the server's TargetInfo (if it is included; otherwise it should use the current time). Currently, MailKit always uses the current time, ignoring the timestamp sent by the server.

To enable this fix, do the following:

var ntlm = new SaslMechanismNtlm (username, password) {
    NtlmFixes = NtlmFixes.NTLMv2UseTargetInfoTimestamp
};

// The following code enables the NTLMSSP_NEGOTIATE_VERSION debugging info and
// System.Net.Mail seems to include it, so I've made it possible to include as well.
if (Environment.OSVersion.Platform == PlatformID.Win32NT)
    ntlm.OSVersion = Environment.OSVersion.Version;

1. Here's where I want you to test both fixes combined.

var ntlm = new SaslMechanismNtlm (username, password) {
    NtlmFixes = NtlmFixes.NTLMv2IncludeZ24 | NtlmFixes.NTLMv2UseTargetInfoTimestamp
};

// The following code enables the NTLMSSP_NEGOTIATE_VERSION debugging info and
// System.Net.Mail seems to include it, so I've made it possible to include as well.
if (Environment.OSVersion.Platform == PlatformID.Win32NT)
    ntlm.OSVersion = Environment.OSVersion.Version;

Let me know if any of the above solutions work.

[jstedfast]

Official docs for NTLM can be found here: https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-nlmp/b38c36ed-2804-4868-a9ff-8dd3182128e4

Another useful resource can be found here: http://davenport.sourceforge.net/ntlm.html

In fact, that second resource was the basis for the current NTLM implementation in MailKit (which was borrowed from Mono's NTLM code).

[pawanbhabad]

Yes sure @jstedfast
Thank you!

[pawanbhabad]

Hi @jstedfast I have tried all the above three cases but it does not seem to be working.

Following are the test case logs,

Case 1: 24 Byte array buffer case.

S: 220 Pawan_Dev.vxcountry.cinemax.com Microsoft ESMTP MAIL Service, Version: 8.5.9600.16384 ready at Mon, 30 Nov 2020 10:45:06 +0530
C: EHLO [10.217.216.0]
S: 250-Pawan_Dev.vxcountry.cinemax.com Hello [10.217.216.0] | AUTH NTLM | TURN | SIZE 2097152 | ETRN | PIPELINING | DSN | ENHANCEDSTATUSCODES | 8bitmime | BINARYMIME | CHUNKING | VRFY | OK
C: AUTH NTLM TlRMTVNTUAABAAAABxIAAgkACQAoAAAAAAAAACgAAAAGAvAjAAAAD0NPTU1VTklUWQ==
S: 334 TlRMTVNTUAACAAAADgAOADgAAAAFAoECnCuy2+TYfW0AAAAAAAAAAMoAygBGAAAABgOAJQAAAA9WAFgASQBOAEQASQBBAAIADgBWAFgASQBOAEQASQBBAAEAEgBQAEEAVwBBAE4AXwBEAEUAVgAEACYAdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQADADoAUABhAHcAYQBuAF8ARABlAHYALgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAUAJgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAcACACdpyDE18bWAQAAAAA=
C: TlRMTVNTUAADAAAAGAAYAG4AAAD2APYAhgAAAA4ADgBIAAAAGAAYAFYAAAAAAAAAbgAAAAAAAAB8AQAAAQIAAgYC8CMAAAAPVgBYAEkATgBEAEkAQQBwAGEAdwBhAG4ALgBiAGgAYQBiAGEAZAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2pZmJb2q4+VplLLGJKuEJAQEAAAAAAAD+/vPdBcfWAUI7z8zYkmdHAAAAAAIADgBWAFgASQBOAEQASQBBAAEAEgBQAEEAVwBBAE4AXwBEAEUAVgAEACYAdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQADADoAUABhAHcAYQBuAF8ARABlAHYALgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAUAJgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAcACACdpyDE18bWAQAAAAA=
S: 535 5.7.3 Authentication unsuccessful
C: MAIL FROM:[e-mail] SIZE=319 | RCPT TO:[e-mail]
S: 530 5.7.3 Client was not authenticated
S: 530 5.7.3 Client was not authenticated
C: QUIT
S: 221 2.0.0 Pawan_Dev.vxcountry.cinemax.com Service closing transmission channel

Case 2: Timestamp included in server's TargetInfo;

S: 220 Pawan_Dev.vxcountry.cinemax.com Microsoft ESMTP MAIL Service, Version: 8.5.9600.16384 ready at Mon, 30 Nov 2020 10:51:59 +0530
C: EHLO [10.217.216.0]
S: 250-Pawan_Dev.vxcountry.cinemax.com Hello [10.217.216.0] | AUTH NTLM | TURN | SIZE 2097152 | ETRN | PIPELINING | DSN | ENHANCEDSTATUSCODES | 8bitmime | BINARYMIME | CHUNKING | VRFY | OK
C: AUTH NTLM TlRMTVNTUAABAAAABxIAAgkACQAoAAAAAAAAACgAAAAGAvAjAAAAD0NPTU1VTklUWQ==
S: 334 TlRMTVNTUAACAAAADgAOADgAAAAFAoECUkHVuQ8pqJEAAAAAAAAAAMoAygBGAAAABgOAJQAAAA9WAFgASQBOAEQASQBBAAIADgBWAFgASQBOAEQASQBBAAEAEgBQAEEAVwBBAE4AXwBEAEUAVgAEACYAdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQADADoAUABhAHcAYQBuAF8ARABlAHYALgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAUAJgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAcACAArFJy62MbWAQAAAAA=
C: TlRMTVNTUAADAAAAAAAAAG4AAAD2APYAbgAAAA4ADgBIAAAAGAAYAFYAAAAAAAAAbgAAAAAAAABkAQAAAQIAAgYC8CMAAAAPVgBYAEkATgBEAEkAQQBwAGEAdwBhAG4ALgBiAGgAYQBiAGEAZAAJ49UrdjmCx7XlF4V3As8ZAQEAAAAAAAArFJy62MbWAVbHJrGGAy8YAAAAAAIADgBWAFgASQBOAEQASQBBAAEAEgBQAEEAVwBBAE4AXwBEAEUAVgAEACYAdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQADADoAUABhAHcAYQBuAF8ARABlAHYALgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAUAJgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAcACAArFJy62MbWAQAAAAA=
S: 535 5.7.3 Authentication unsuccessful
C: MAIL FROM:[e-mail] SIZE=320 | RCPT TO:[e-mail]
S: 530 5.7.3 Client was not authenticated
S: 530 5.7.3 Client was not authenticated
C: QUIT
S: 221 2.0.0 Pawan_Dev.vxcountry.cinemax.com Service closing transmission channel

Case 3: Both fixes combined;

S: 220 Pawan_Dev.vxcountry.cinemax.com Microsoft ESMTP MAIL Service, Version: 8.5.9600.16384 ready at Mon, 30 Nov 2020 10:19:18 +0530
C: EHLO [10.217.216.0]
S: 250-Pawan_Dev.vxcountry.cinemax.com Hello [10.217.216.0] | AUTH NTLM | TURN | SIZE 2097152 | ETRN | PIPELINING | DSN | ENHANCEDSTATUSCODES | 8bitmime | BINARYMIME | CHUNKING | VRFY | OK
C: AUTH NTLM TlRMTVNTUAABAAAABxIAAgkACQAoAAAAAAAAACgAAAAGAvAjAAAAD0NPTU1VTklUWQ==
S: 334 TlRMTVNTUAACAAAADgAOADgAAAAFAoECX0mzDfWJGtMAAAAAAAAAAMoAygBGAAAABgOAJQAAAA9WAFgASQBOAEQASQBBAAIADgBWAFgASQBOAEQASQBBAAEAEgBQAEEAVwBBAE4AXwBEAEUAVgAEACYAdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQADADoAUABhAHcAYQBuAF8ARABlAHYALgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAUAJgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAcACADDOM4p1MbWAQAAAAA=
C: TlRMTVNTUAADAAAAGAAYAG4AAAD2APYAhgAAAA4ADgBIAAAAGAAYAFYAAAAAAAAAbgAAAAAAAAB8AQAAAQIAAgYC8CMAAAAPVgBYAEkATgBEAEkAQQBwAGEAdwBhAG4ALgBiAGgAYQBiAGEAZAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACxl6cmbx9v3GgM0EFxkEfjAQEAAAAAAADDOM4p1MbWAWaJNx3w3+yyAAAAAAIADgBWAFgASQBOAEQASQBBAAEAEgBQAEEAVwBBAE4AXwBEAEUAVgAEACYAdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQADADoAUABhAHcAYQBuAF8ARABlAHYALgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAUAJgB2AHgAaQBuAGQAaQBhAC4AdgBlAHIAaQB0AGEAcwAuAGMAbwBtAAcACADDOM4p1MbWAQAAAAA=
S: 535 5.7.3 Authentication unsuccessful
C: MAIL FROM:[e-mail] SIZE=320 | RCPT TO:[e-mail]
S: 530 5.7.3 Client was not authenticated
S: 530 5.7.3 Client was not authenticated
C: QUIT
S: 221 2.0.0 Pawan_Dev.vxcountry.cinemax.com Service closing transmission channel

[jstedfast]

@pawanbhabad Thanks for testing out these fixes... I'll see if I can think of some other ideas to try.

[jstedfast]

@pawanbhabad can you try the same with https://www.myget.org/feed/mimekit/package/nuget/MailKit/2.10.0.5 ?

This sets more flags the way System.Net.Mail does in the 3rd message (RequestTarget and NegotiateTargetInfo) that MailKit wasn't including before. I doubt those will make a difference, but it's something we can at least try.

The docs mention a MIC, so I may need to try and calculate that... at this point it's just guessing :-\

[pawanbhabad]

Ok, I will try with https://www.myget.org/feed/mimekit/package/nuget/MailKit/2.10.0.5

[pawanbhabad]

Hi @jstedfast I tried with above updated Mailkit 2.10.0.5 but same result.

[jstedfast]

Thanks @pawanbhabad - I'll see if I can poke at it some more this weekend.

[pawanbhabad]

Hi @jstedfast Any further updates ?

[jstedfast]

@pawanbhabad sorry I haven't had a chance to look at this as much as I had hoped. I've been spending a lot of time with my girlfriend and her family and just haven't had any time to work on stuff.

[pawanbhabad]

@jstedfast Oh ok 😊. Please let me know once you get chance to look into that. Thanks!