Containers-roadmap: Add Windows Credentialspec Support

[Cgettys]

Summary

Add Windows Credentialspec support to enable Windows Auth in containers.

Description

Although Docker for Windows supports Windows Auth through the use of credentialspec files, Amazon ECS does not support this capability.

Retaining the ability to use Windows Authentication within the docker ecosystem eliminates the need to refactor applications to use other forms of authentication. Given how ubiquitous Windows Auth is within many legacy dotnet applications, this feature would greatly streamline the migration process.

[mattshep]

It would be great if there were a way to just pass through the "credentialspec=file://gmsa.json" security-opt blindly and let me worry about whether the not the task is able to run successfully. I've already scripted the installation of the AD service account and creation of the credentialspec file I require when my ASG instances launch prior to initializing the ECS agent, this is literally the only thing preventing me from being able to use integrated windows authentication in my .NET Framework services. Thanks!

[Cgettys]

@eisware I have used the Docker labels option and a modified version of the agent that parses the labels and injects the right label into the right place in the past. You can basically put credentialspec=file://gmsa.json into the labels, match that, and then put that where it belongs in the agent, where it's past the validation. I've done this successfully before with one or two lines of code, though unfortunately I don't have up to date source code to share, and it's not officially supported. I also recommend you reach out to your Amazon rep to get yourself added to the list of people who really want this feature, to increase the likelihood of it actually happening.
Disclaimer: I am not affiliated with Amazon. I cannot recommend you take this route. This is not officially supported and you're on your own if you do it.

[mattshep]

@Cgettys Thanks for the ideas :)

[Cgettys]

@eisware No problem. I hit the exact same problem, went down the road of figuring out how to work around it, and thought I'd save a fellow developer the trouble since there's no downside to me doing so. Besides, the more people (especially enterprise customers) who tell them that they want this feature, the more likely they'll move it up their priority list, and that would benefit both of us.

[Cgettys]

Hit the wrong button there, oops.

[bitshop]

It seems like a few things don't have solutions discussed as @samuelkarp mentioned in #1543

• The spec file shouldn't be in plain text - This could be put in S3 encrypted and in the definition have the S3 file spec - This would allow other specific options to be listed in the S3 file if those come about in the future. The S3 file could be protected such that only the role the ECS host runs could access it, plus the key, allowing it to be protected.

• The host must be joined to the proper domain, so in ECS there should be extra checks before this option is used. This is more documentation / FAQ than software.

• The host requires having the gMSA rights, this should be checked before the host is allowed to be used - This could use a friendly check so that the error message can be reported nicely upon failure to launch.

I definitely agree this is a VERY useful feature (critical really) to rolling out a PoC for Step Functions and ECS replacing an in house solution baked in Jenkins and Agents specifying where code can run.

[thelok]

@bitshop 100% agree. We're investigating using gMSA in ECS and have run into roadblocks with the issues you discussed. Documentation would be greatly beneficial for us on how to accomplish your points.

[thelok]

Are there any workarounds to this other than modifying the ECS agent source code?

[GaryAult]

About a year ago this pretty much killed our plans to move legacy Windows applications from servers to ECS. We got it to work with some hacking, but decided it wasn't production worthy.

Hacks in addition to the gMSA setup and joining the hosts to the domain on spin up.

1) Label then Windows environment as SELINUX in userdata.
2) Insert “label:” into this line of object oci_windows.go inside the dockerd daemon and rebuild.
if splitsOpt[0] != "label:credentialspec". You'll then need to use your custom dockerd and use label:credentialspec vs. just credential spec.

This gets you around the AWS parameter limits, but do so at your own risk.

I would also recommend double checking load balancer and application configuration around connection pooling. Having the application maintain the security at the connection level and connection pooling from the LB isn't a good combination. This can be fixed with config changes.

[Cgettys]

You can also use another field instead of SELINUX, to do the same thing. See my above messages. This should avoid the SELINUX hack at least. But I don't know of any workaround that doesn't involve modifying the ECS agent. One step might be to try to solve the issues @bitshop brought up with my attempt #1543.

I'm no expert on Windows Active Directory, and it obviously might vary based on application whether this is really the case, but when I took a look at it, the credential spec looked to me to be non-sensitive metadata (though if I'm wrong, please correct me on this). For instance, here's a snippet of the code that generates it:

    # Start hash table for output
    $output = @{}

    # Create ActiveDirectoryConfig Object
    $output.ActiveDirectoryConfig = @{}
    $output.ActiveDirectoryConfig.GroupManagedServiceAccounts = @( @{"Name" = $AccountName; "Scope" = $ADDomain.DNSRoot } )
    $output.ActiveDirectoryConfig.GroupManagedServiceAccounts += @{"Name" = $AccountName; "Scope" = $ADDomain.NetBIOSName }
    if ($AdditionalAccounts) {
        $AdditionalAccounts | ForEach-Object {
            $output.ActiveDirectoryConfig.GroupManagedServiceAccounts += @{"Name" = $_.AccountName; "Scope" = $_.Domain }
        }
    }

    # Create CmsPlugins Object
    $output.CmsPlugins = @("ActiveDirectory")

    # Create DomainJoinConfig Object
    $output.DomainJoinConfig = @{}
    $output.DomainJoinConfig.DnsName = $ADDomain.DNSRoot
    $output.DomainJoinConfig.Guid = $ADDomain.ObjectGUID
    $output.DomainJoinConfig.DnsTreeName = $ADDomain.Forest
    $output.DomainJoinConfig.NetBiosName = $ADDomain.NetBIOSName
    $output.DomainJoinConfig.Sid = $ADDomain.DomainSID.Value
    $output.DomainJoinConfig.MachineAccountName = $AccountName

    $output | ConvertTo-Json -Depth 5 | Out-File -FilePath $FullPath -Encoding ascii -NoClobber:$NoClobber

Official Credentialspec Powershell module.

I didn't ever follow up on my pull request because I had other priorities at the time & am busy with college. I was working on a project targeting the ECS platform as part of a summer internship, so while I find this stuff interesting, I'm not personally looking to pour more time into it at present. I'm happy to share what I know and wish you the best of luck though!

[pavneeta]

277 Same request.

[coultn]

This shipped: https://aws.amazon.com/about-aws/whats-new/2019/12/amazon-elastic-container-service-now-supports-active-directory-authentication-using-windows-group-managed-service-accounts/

[Cgettys]

Thanks! I have passed this on to my former colleagues, and I'm sure they will be happy to hear that this is now supported by default!

[pmscad]

Guys,

Does anyone know how to check this credential spec file on a Domain Controller? I have 2 DC's and few worker nodes synced with these Domain controllers but the gMSA account when used for a particular service is not working.

Any thoughts anyone can share would be a great help..