Dotnet-docker: dotnet build causes exiting of docker container unexpectedly when using a bind mount to host machine

[lurock]

Steps to reproduce the issue

1. mkdir C:ConsoleTestApp
2. cd C:ConsoleTestApp
3. dotnet new console
4. Remove-Item -Recurse bin
5. Remove-Item -Recurse obj
6. docker run -it -v "$pwdmd5-13b44dfe71d0c7f74c0f70c1449f5afc:C:build" microsoft/dotnet:sdk cmd
7. cd C:build (now inside the docker container)
8. dotnet build

Expected behavior

Project to build bin directory on host machine.

Actual behavior

Build never completes and your console just exits from the docker container and drops back to your host machine console. Output looks like this:

Microsoft (R) Build Engine version 15.7.179.6572 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

Restoring packages for C:buildTestConsoleApp.csproj...

Then it just drops back into your host machines console and the docker container exits unexpectedly.

Additional information (e.g. issue happens only occasionally)

Output of docker version

Client:
 Version:       17.06.2-ee-10
 API version:   1.30
 Go version:    go1.8.7
 Git commit:    66261a0
 Built: Fri Apr 27 00:42:30 2018
 OS/Arch:       windows/amd64

Server:
 Engine:
  Version:      17.06.2-ee-10
  API version:  1.30 (minimum version 1.24)
  Go version:   go1.8.7
  Git commit:   66261a0
  Built:        Fri Apr 27 00:54:58 2018
  OS/Arch:      windows/amd64
  Experimental: false

Output of docker info

Containers: 3
 Running: 3
 Paused: 0
 Stopped: 0
Images: 4
Server Version: 17.06.2-ee-10
Storage Driver: windowsfilter
 Windows:
Logging Driver: json-file
Plugins:
 Volume: local
 Network: l2bridge l2tunnel nat null overlay transparent
 Log: awslogs etwlogs fluentd json-file logentries splunk syslog
Swarm: inactive
Default Isolation: process
Kernel Version: 10.0 17134 (17134.1.amd64fre.rs4_release.180410-1804)
Operating System: Windows Server Datacenter
OSType: windows
Architecture: x86_64
CPUs: 2
Total Memory: 7GiB
Name: backend000003
ID: VBG4:B2BQ:H4VR:NVKP:5KQ6:MLHC:WYE2:53PL:SZCD:TVWN:XJRN:4DEM
Docker Root Dir: D:\docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

[MichaelSimons]

What version of dotnet do you have installed on your Docker host? Curious about step 4, there isn't a bin directory created as part of creating a new console project with the latest version of the dotnet cli. Can you list the output of c:build within the containter? Does this scenario work if you create the project within the Docker container rather than using volume mounting?

[lurock]

I have dotnet version 2.1.300. You are right when you first create a project you do not get a bin directory. I just want to make sure when you start the container you do not have a "bin" or "obj" directory so that you start a clean restore and build from inside the container. Yes I can list the output of C:build within the container. It works fine when I create a project within the Docker container rather than using volume mounting.

[vyadh]

I get this crash/exit for 'dotnet restore' as well as 'dotnet build'. I tried with the 2.0.9-sdk-2.1.202-nanoserver-1803 image and it works fine, it's just the 2.1 version (2.1.302-sdk-nanoserver-1803) that doesn't work.

[henryrt]

+1
image: microsoft/dotnet 2.1-sdk 9406fea8e123 7 days ago 1.71GB

I do not have dotnet core installed on the docker host, these commands are in an interactive container.

I can reproduce @vyadh issue. 'dotnet build' and 'dotnet restore' crash the container, but only when mounts exist. Using the default console app, and comparing the -v d log between the mount and no-mount situation:

WITHOUT mounts:

   Using "RestoreTask" task from assembly "C:\Program Files\dotnet\sdk\2.1.302\NuGet.Build.Tasks.dll".
   Task "RestoreTask"
     (in) RestoreGraphItems Count '4'
     (in) RestoreDisableParallel 'False'
     (in) RestoreNoCache 'False'
     (in) RestoreIgnoreFailedSources 'False'
     (in) RestoreRecursive 'True'
     (in) RestoreForce 'False'
     (in) HideWarningsAndErrors 'False'
     Running restore with 2 concurrent jobs.
     Reading project file C:\ds\test4\test4.csproj.
     Restoring packages for C:\ds\test4\test4.csproj...
     Restoring packages for .NETCoreApp,Version=v2.1...
     Resolving conflicts for .NETCoreApp,Version=v2.1...
     Checking compatibility of packages on .NETCoreApp,Version=v2.1.
     Checking compatibility for test4 1.0.0 with .NETCoreApp,Version=v2.1.
     Checking compatibility for Microsoft.NETCore.App 2.1.0 with .NETCoreApp,Version=v2.1.
     Checking compatibility for Microsoft.NETCore.DotNetHostPolicy 2.1.0 with .NETCoreApp,Version=v2.1.
     Checking compatibility for Microsoft.NETCore.Platforms 2.1.0 with .NETCoreApp,Version=v2.1.

(etc. it completes successfully)

WITH mounts:

   Using "RestoreTask" task from assembly "C:\Program Files\dotnet\sdk\2.1.302\NuGet.Build.Tasks.dll".
   Task "RestoreTask"
     (in) RestoreGraphItems Count '4'
     (in) RestoreDisableParallel 'False'
     (in) RestoreNoCache 'False'
     (in) RestoreIgnoreFailedSources 'False'
     (in) RestoreRecursive 'True'
     (in) RestoreForce 'False'
     (in) HideWarningsAndErrors 'False'
     Running restore with 2 concurrent jobs.
     Reading project file c:\ds\test4\test4.csproj.
     Restoring packages for c:\ds\test4\test4.csproj...
     Restoring packages for .NETCoreApp,Version=v2.1...
     Resolving conflicts for .NETCoreApp,Version=v2.1...
     **********Crashes here **********

c:ds>dotnet --info
.NET Core SDK (reflecting any global.json):
Version: 2.1.302
Commit: 9048955601

Runtime Environment:
OS Name: Windows
OS Version: 10.0.17134
OS Platform: Windows
RID: win10-x64
Base Path: C:Program Filesdotnetsdk2.1.302

Host (useful for support):
Version: 2.1.2
Commit: 811c3ce6c0

.NET Core SDKs installed:
2.1.302 [C:Program Filesdotnetsdk]

.NET Core runtimes installed:
Microsoft.AspNetCore.All 2.1.2 [C:Program FilesdotnetsharedMicrosoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.2 [C:Program FilesdotnetsharedMicrosoft.AspNetCore.App]
Microsoft.NETCore.App 2.1.2 [C:Program FilesdotnetsharedMicrosoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
https://aka.ms/dotnet-download

PS C:ds> docker version
Client:
Version: 18.03.1-ce
API version: 1.37
Go version: go1.9.5
Git commit: 9ee9f40
Built: Thu Apr 26 07:12:48 2018
OS/Arch: windows/amd64
Experimental: false
Orchestrator: swarm

Server:
Engine:
Version: 18.03.1-ce
API version: 1.37 (minimum version 1.24)
Go version: go1.9.5
Git commit: 9ee9f40
Built: Thu Apr 26 07:21:42 2018
OS/Arch: windows/amd64
Experimental: false

PS C:ds> docker info
Containers: 3
Running: 0
Paused: 0
Stopped: 3
Images: 8
Server Version: 18.03.1-ce
Storage Driver: windowsfilter
Windows:
Logging Driver: json-file
Plugins:
Volume: local
Network: ics l2bridge l2tunnel nat null overlay transparent
Log: awslogs etwlogs fluentd gelf json-file logentries splunk syslog
Swarm: inactive
Default Isolation: hyperv
Kernel Version: 10.0 17134 (17134.1.amd64fre.rs4_release.180410-1804)
Operating System: Windows 10 Pro
OSType: windows
Architecture: x86_64
CPUs: 8
Total Memory: 15.91GiB
Name: XXXXXXX
ID: XXXXXXX
Docker Root Dir: C:ProgramDataDocker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: -1
Goroutines: 21
System Time: 2018-07-17T23:43:32.4515469-04:00
EventsListeners: 0
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false

[vyadh]

To clarify, the issue I saw was also using mounts. In my case these were added automatically by TeamCity as part of a containerised compile process. I also tried the commands interactively, and like @henryrt I do not have dotnet installed on the host.

[henryrt]

From the Event Log:

[09:14:22.249][WindowsDaemon ][Info ] debug: HCSShim::Process::Wait succeeded processid=1420
[09:14:22.249][WindowsDaemon ][Info ] debug: HCSShim::Process::ExitCode processid=1420
[09:14:22.249][WindowsDaemon ][Info ] debug: HCSShim::Process::properties processid=1420
[09:14:22.250][WindowsDaemon ][Info ] debug: HCSShim::Process::properties succeeded processid=1420, properties={"ProcessId":1420,"Exited":true,"ExitCode":0,"LastWaitResult":-1070137082}
[09:14:22.251][WindowsDaemon ][Warning] unable to get exit code for process [container=bd84132fbbb98dca3c5510248177cf1b4132af58690a37abc7517f2508386885 error=process 1420 in container bd84132fbbb98dca3c5510248177cf1b4132af58690a37abc7517f2508386885 encountered an error during ExitCode: failure in a Windows system call: The compute system exited unexpectedly. (0xc0370106) process=init module=libcontainerd namespace=moby]
[09:14:22.252][WindowsDaemon ][Info ] debug: HCSShim::Process::Close processid=1420
[09:14:22.252][WindowsDaemon ][Info ] debug: HCSShim::Process::Close succeeded processid=1420
[09:14:22.252][WindowsDaemon ][Info ] debug: HCSShim::Container::Shutdown id=bd84132fbbb98dca3c5510248177cf1b4132af58690a37abc7517f2508386885
[09:14:22.253][WindowsDaemon ][Info ] debug: Result: {"Error":-1070137078,"ErrorMessage":"The connection with the Virtual Machine hosting the container was closed."}
[09:14:22.253][WindowsDaemon ][Info ] debug: failed to shutdown container, terminating it [namespace=moby error=container bd84132fbbb98dca3c5510248177cf1b4132af58690a37abc7517f2508386885 encountered an error during Shutdown: failure in a Windows system call: The connection with the Virtual Machine hosting the container was closed. (0xc037010a) container=bd84132fbbb98dca3c5510248177cf1b4132af58690a37abc7517f2508386885 module=libcontainerd]

[MichaelSimons]

Out of curiosity has anyone experiencing this problem tried running as ContainerAdministrator? With 1709 and 1803 images, the default user is ContainerUser and you could be encountering permissions issues with the mounted volumes.

[henryrt]

I removed the line in the Dockerfile: USER ContainerUser and rebuilt the image. When I ran the image, I was ContainerAdministrator. Unfortunately, dotnet restore crashed the container anyway.

[MichaelSimons]

I was able to reproduce this but only with the 1803 images (e.g. 2.1-sdk-nanoserver-1803). I wasn't able to reproduce the issue with 2.1-sdk-nanoserver-1709 or 2.1-sdk-nanoserver-sac2016.

[henryrt]

Interesting. I also tried 2.1-sdk-nanoserver-1709 and could not reproduce it. I guess that's the image to use for now.

[MichaelSimons]

The other thing I noticed is that I couldn't reproduce the problem on an Windows Server 1803 Docker Host. I could only reproduce on my Win10 environment.

[henryrt]

Agreed, it seems to be a Windows 10 issue. It works on my Azure Windows Server 1803 Docker Host.

[JensNordenbro]

We have the same on windowsservercore 1803-based .net core 2.1 sdk images on Windows 10 Pro. Removing VOLUME and it works again.

[MichaelSimons]

I logged https://github.com/docker/for-win/issues/2540 for this issue as the problem seems specific to Docker for Windows with the use of Volumes.

[shueybubbles]

I see what appears to be a related issue on my Server Core LCOW running the dotnet:2.1-sdk image we use for running our tests. We use this same dockerfile on a non-LCOW machine and on an LCOW machine. On the LCOW machine the tests don't have access to write to the mounted volume anymore. This was working in the 2.0 SDK image we used previously. We had to upgrade to 2.1 because our NetCore implementation moved up to 2.1.

FROM microsoft/dotnet:2.1-sdk
WORKDIR /test
COPY ./ ./
EXPOSE 1450
ENTRYPOINT ["dotnet", "vstest", "./Microsoft.SqlServer.Test.NetCoreSmo.dll", "/logger:console;verbosity=Normal"]

the powershell snippet that runs the tests
docker run -v $localFolder`:$resultsFolder -e "RESULTS_FOLDER=$resultsFolder" -e "TEST_DATABASE=$testDatabase" -e "TEST_HOSTNAME=$endpoint" -e "TEST_PASSWORD=$pwd" -e "TEST_USERNAME=sa" -h testhost --rm $testContainer /settings:$testSettings /logger:trx --ResultsDirectory:$resultsFolder $testParams

And this is the error
Test Run Failed.
Test execution time: 1.6764 Minutes
Access to the path '/results/_testhost_2018-09-23_01_02_02.trx' is denied.

Is there a better method to get test results files off the container?

[MichaelSimons]

Worked with @shueybubbles offline and his specific issue appears to be something different, possibly LCOW related.

[MichaelSimons]

Closing as external - please follow docker/for-win#2540.