Dotnet-docker: unable to find a project to restore during interactive session

[dotChris90]

Hello, dotnet developer,

I was playing around with docker and .NET Core on my Windows 7, Windows 10 and Fedora Systems and at moment detected a strange behaviour on my Fedora System. The System is 64 Bit, Fedora 27 (upgraded some days ago) and with docker from fedora.repo. I tried it on Fedora, Windows and compared the csproj files. Slowly I run out of ideas. ^^ Honestly I was surprised that it worked on first try on the Windows systems (which create a virtual linux before able to run docker) and Fedora failed.
I am open to any idea.

Steps to reproduce the issue

1. sudo docker pull microsoft/dotnet
2. sudo docker run -i -t microsoft/dotnet:latest
3. mkdir app
4. cd app
5. dotnet new console

Expected behavior

There should be app.csproj file (this actual really exist after step 5) and restore command (which is done automaticly since SDK 2 after creation with new command) should restore all packages. With "dotnet run" application should start.

Actual behavior

• /usr/share/dotnet/sdk/2.0.3/NuGet.targets(102,5): warning : Unable to find a project to restore! [/app/app.csproj] (after dotnet new console )
• when execute "dotnet run" inside interactive docker container it says "error - out of memory"

Additional information

• On my fedora 27 exist already a .NET Core with SDK 2.0 (outside docker container). This .NET Core works fine outside the docker container.
• docker interactive works fine for other images e.g. python 3 or 2.
• dotnet works fine on Docker of Windows 7 (Docker Toolbox ) and Windows 10
• according to dnf package manager docker comes from fedora.repo
• CPU is on < 40% during execution, disk storage 40GB left, RAM still have 2GB left after inside container, (CPU is 64 Bit Arch - was my first guess ... but i Think if you just pull microsoft/dotnet it should pull 64 Bit version)

Output of docker version

Client:
Version: 1.13.1
API version: 1.26
Package version: docker-1.13.1-26.gitb5e3294.fc27.x86_64
Go version: go1.9beta2
Git commit: b5e3294/1.13.1
Built: Tue Aug 15 14:39:35 2017
OS/Arch: linux/amd64

Server:
Version: 1.13.1
API version: 1.26 (minimum version 1.12)
Package version: docker-1.13.1-26.gitb5e3294.fc27.x86_64
Go version: go1.9beta2
Git commit: b5e3294/1.13.1
Built: Tue Aug 15 14:39:35 2017
OS/Arch: linux/amd64
Experimental: false

Output of docker info

Containers: 10
Running: 0
Paused: 0
Stopped: 10
Images: 5
Server Version: 1.13.1
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: journald
Cgroup Driver: systemd
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Authorization: rhel-push-plugin
Swarm: inactive
Runtimes: oci runc
Default Runtime: oci
Init Binary: /usr/libexec/docker/docker-init-current
containerd version: (expected: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1)
runc version: N/A (expected: 9df8b306d01f59d3a8029be411de015b7304dd8f)
init version: N/A (expected: 949e6facb77383876aeff8a6944dde66b3089574)
Security Options:
seccomp
Profile: default
selinux
Kernel Version: 4.13.13-300.fc27.x86_64
Operating System: Fedora 27 (Workstation Edition)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 3
CPUs: 4
Total Memory: 3.731 GiB
Name: localhost.localdomain
ID: EOHD:JPXG:V23L:KEXL:4AYZ:TLD5:UTWL:GKSD:FVYP:VY4T:Z7PB:C4WE
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://registry.fedoraproject.org/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Registries: registry.fedoraproject.org (secure), registry.access.redhat.com (secure), docker.io (secure)

[ravimeda]

Thanks very much for reporting this issue. I can reproduce it locally on my machine. Let me investigate to find the root cause.

[root@localhost ~]# docker run --rm -it microsoft/dotnet:latest
root@3aa6823641f5:/# mkdir app
root@3aa6823641f5:/# cd app
root@3aa6823641f5:/app# dotnet new console
The template "Console Application" was created successfully.
Processing post-creation actions...
Running 'dotnet restore' on /app/app.csproj...
/usr/share/dotnet/sdk/2.0.3/NuGet.targets(102,5): warning : Unable to find a project to restore! [/app/app.csproj]
Restore succeeded.
root@3aa6823641f5:/app# ls -lR
.:
total 8
-rw-r--r--. 1 root root 185 Nov 23 04:39 Program.cs
-rw-r--r--. 1 root root 178 Nov 23 04:39 app.csproj
root@3aa6823641f5:/app# dotnet run
/usr/share/dotnet/sdk/2.0.3/NuGet.targets(102,5): warning : Unable to find a project to restore! [/app/app.csproj]
MSBUILD : error MSB1025: An internal failure occurred while running MSBuild.
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.Runtime.Loader.AssemblyLoadContext.LoadFromPath(IntPtr ptrNativeAssemblyLoadContext, String ilPath, String niPath, ObjectHandleOnStack retAssembly)
   at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath)
   at Microsoft.Build.Shared.CoreClrAssemblyLoader.LoadAndCache(String fullPath)
   at Microsoft.Build.Shared.CoreClrAssemblyLoader.LoadFromPath(String fullPath)
   at Microsoft.Build.Shared.TypeLoader.LoadAssembly(AssemblyLoadInfo assemblyLoadInfo)
   at Microsoft.Build.Shared.TypeLoader.AssemblyInfoToLoadedTypes.ScanAssemblyForPublicTypes()
   at Microsoft.Build.Shared.TypeLoader.AssemblyInfoToLoadedTypes.GetLoadedTypeByTypeName(String typeName)
   at Microsoft.Build.Shared.TypeLoader.GetLoadedType(Object cacheLock, Object loadInfoToTypeLock, ConcurrentDictionary`2 cache, String typeName, AssemblyLoadInfo assembly)
   at Microsoft.Build.BackEnd.AssemblyTaskFactory.InitializeFactory(AssemblyLoadInfo loadInfo, String taskName, IDictionary`2 taskParameters, String taskElementContents, IDictionary`2 taskFactoryIdentityParameters, Boolean taskHostFactoryExplicitlyRequested, TargetLoggingContext targetLoggingContext, ElementLocation elementLocation, String taskProjectFile)
   at Microsoft.Build.Execution.TaskRegistry.RegisteredTaskRecord.GetTaskFactory(TargetLoggingContext targetLoggingContext, ElementLocation elementLocation, String taskProjectFile)
   at Microsoft.Build.Execution.TaskRegistry.RegisteredTaskRecord.CanTaskBeCreatedByFactory(String taskName, String taskProjectFile, IDictionary`2 taskIdentityParameters, TargetLoggingContext targetLoggingContext, ElementLocation elementLocation)
   at Microsoft.Build.Execution.TaskRegistry.GetMatchingRegistration(String taskName, List`1 taskRecords, String taskProjectFile, IDictionary`2 taskIdentityParameters, TargetLoggingContext targetLoggingContext, ElementLocation elementLocation)
   at Microsoft.Build.Execution.TaskRegistry.GetTaskRegistrationRecord(String taskName, String taskProjectFile, IDictionary`2 taskIdentityParameters, Boolean exactMatchRequired, TargetLoggingContext targetLoggingContext, ElementLocation elementLocation, Boolean& retrievedFromCache)
   at Microsoft.Build.Execution.TaskRegistry.GetRegisteredTask(String taskName, String taskProjectFile, IDictionary`2 taskIdentityParameters, Boolean exactMatchRequired, TargetLoggingContext targetLoggingContext, ElementLocation elementLocation)
   at Microsoft.Build.BackEnd.TaskExecutionHost.FindTaskInRegistry(IDictionary`2 taskIdentityParameters)
   at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.FindTask(IDictionary`2 taskIdentityParameters)
   at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteBucket>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteTask>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteTask>d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.Build.BackEnd.TargetEntry.<ProcessBucket>d__49.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.Build.BackEnd.TargetEntry.<ExecuteTarget>d__42.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Build.BackEnd.TargetBuilder.<ProcessTargetStack>d__22.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Build.BackEnd.TargetBuilder.<BuildTargets>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.Build.BackEnd.RequestBuilder.<BuildProject>d__57.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.Build.BackEnd.RequestBuilder.<BuildAndReport>d__49.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Build.BackEnd.RequestBuilder.<RequestThreadProc>d__48.MoveNext()
MSBUILD : error MSB1025: An internal failure occurred while running MSBuild.
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at Microsoft.Build.CommandLine.MSBuildApp.BuildProject(String projectFile, String[] targets, String toolsVersion, Dictionary`2 globalProperties, ILogger[] loggers, LoggerVerbosity verbosity, DistributedLoggerRecord[] distributedLoggerRecords, Int32 cpuCount, Boolean enableNodeReuse, TextWriter preprocessWriter, Boolean debugger, Boolean detailedSummary, ISet`1 warningsAsErrors, ISet`1 warningsAsMessages)
   at Microsoft.Build.CommandLine.MSBuildApp.Execute(String[] commandLine)
Unhandled Exception: OutOfMemoryException.
The build failed. Please fix the build errors and run again.
root@3aa6823641f5:/app# 

[abn]

The root cause here looks to be SELinux. An upstream issue as be filed for Fedora 27 at RHBZ#1517739. Not really sure where the fix should lie.

Journal Snippets

SELinux is preventing dotnet from map access on the chr_file /dev/zero.

type=AVC msg=audit(1511779226.350:5495): avc:  denied  { map } for  pid=22295 comm="dotnet" path="/dev/zero" dev="tmpfs" ino=1556922 scontext=system_u:system_r:container_t:s0:c362,c894 tcontext=system_u:object_r:container_file_t:s0:c362,c894 tclass=chr_file permissive=0

Coredump

Stack trace of thread 103:
#0  0x00007fd4db95dfcf n/a (/lib/x86_64-linux-gnu/libc-2.24.so)
#1  0x00007fd4db17cbcb n/a (/usr/share/dotnet/shared/Microsoft.NETCore.App/2.0.3/libcoreclr.so)
#2  0x00007fd4daee58a8 n/a (/usr/share/dotnet/shared/Microsoft.NETCore.App/2.0.3/libcoreclr.so)
#3  0x00007fd4daee5959 n/a (/usr/share/dotnet/shared/Microsoft.NETCore.App/2.0.3/libcoreclr.so)
#4  0x00007fd4dae562d9 n/a (/usr/share/dotnet/shared/Microsoft.NETCore.App/2.0.3/libcoreclr.so)
#5  0x00007fd461d33d3a n/a (n/a)
#6  0x00007fd461d32eb8 n/a (n/a)
#7  0x00007fd4daef2067 n/a (/usr/share/dotnet/shared/Microsoft.NETCore.App/2.0.3/libcoreclr.so)
#8  0x00007fd4dae02e40 n/a (/usr/share/dotnet/shared/Microsoft.NETCore.App/2.0.3/libcoreclr.so)
#9  0x00007fd4daf13db4 n/a (/usr/share/dotnet/shared/Microsoft.NETCore.App/2.0.3/libcoreclr.so)
#10 0x00007fd4daf14033 n/a (/usr/share/dotnet/shared/Microsoft.NETCore.App/2.0.3/libcoreclr.so)
#11 0x00007fd4dad4550b n/a (/usr/share/dotnet/shared/Microsoft.NETCore.App/2.0.3/libcoreclr.so)
#12 0x00007fd4dad1fe86 n/a (/usr/share/dotnet/shared/Microsoft.NETCore.App/2.0.3/libcoreclr.so)
#13 0x00007fd4db43e433 n/a (/usr/share/dotnet/shared/Microsoft.NETCore.App/2.0.3/libhostpolicy.so)
#14 0x00007fd4db4330d8 n/a (/usr/share/dotnet/shared/Microsoft.NETCore.App/2.0.3/libhostpolicy.so)
#15 0x00007fd4db433772 n/a (/usr/share/dotnet/shared/Microsoft.NETCore.App/2.0.3/libhostpolicy.so)
#16 0x00007fd4db6f38f4 n/a (/usr/share/dotnet/host/fxr/2.0.3/libhostfxr.so)
#17 0x00007fd4db6fd978 n/a (/usr/share/dotnet/host/fxr/2.0.3/libhostfxr.so)
#18 0x00007fd4db6fc8f7 n/a (/usr/share/dotnet/host/fxr/2.0.3/libhostfxr.so)
#19 0x00007fd4db6fdfac n/a (/usr/share/dotnet/host/fxr/2.0.3/libhostfxr.so)
#20 0x00007fd4db6f3975 n/a (/usr/share/dotnet/host/fxr/2.0.3/libhostfxr.so)
#21 0x000000000040c42a _Z3runiPPKc (dotnet)
#22 0x000000000040c597 main (dotnet)
#23 0x00007fd4db94b2b1 n/a (/lib/x86_64-linux-gnu/libc-2.24.so)

[ravimeda]

@abn thanks for finding the root cause.

A workaround in this situation is to run the container with --privileged (https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities). This means

sudo docker run --privileged -i -t microsoft/dotnet:latest

[abn]

@ravimeda an alternative approach might be to do the following as root to install a custom module.

ausearch -c 'dotnet' --raw | audit2allow -M my-dotnet
semodule -X 300 -i my-dotnet.pp

Once the issue is resolved, either in upstream policy or within the dotnet context, you can remove it by executing the following command.

semodule --remove=my-dotnet

[abn]

As per this comment the issue should be resolved with container-selinux-2.36 (see commit).

[ravimeda]

Thanks Arun.

[richlander]

Is there a reason why you are running docker run with sudo?

[dotChris90]

@ravimeda : "sudo docker run --privileged -i -t microsoft/dotnet:latest" this works fine for me! Thanks a lot. I could start a container with dotnet CLI and everything just works like on Windows. :) thanks a lot. I am not sure if you want to keep this ticket or close it. For me this ticket is closed. 👍

[dotChris90]

@richlander : honestly I am not sure if this is the official way or not. I just used Fedoras "dnf" (the new package manager ) to install docker (or was it on Fedora by default? I am not sure I have to admit) and when I tried docker without sudo - Fedora keeps tell me "you can not start the docker daemon!" so I though "ok maybe a linux security strategy so just Root users can Start Dockers". But just for curiousity "Sudo is not the official way to use docker on Linux system or is it the official way?".

[ravimeda]

Glad to to know! @dotChrisSharp I think if you make sure Docker service is started, then you can skip sudo before Docker commands. Please refer (https://docs.docker.com/engine/installation/linux/docker-ce/fedora/#install-docker-ce-1)

$ sudo systemctl start docker

[MichaelSimons]

As noted earlier this is being addressed in container-selinux-2.36. Closing.

[omajid]

Sudo is not the official way to use docker on Linux system or is it the official way?

You need to be either root itself, use sudo (to effectively become root) or be part of the docker group. If you can run docker commands, you are effectively root on the machine, since the containers are not locked down.

https://docs.docker.com/engine/installation/linux/linux-postinstall/ says:

The docker daemon binds to a Unix socket instead of a TCP port. By default that Unix socket is owned by the user root and other users can only access it using sudo. The docker daemon always runs as the root user.

If you don’t want to use sudo when you use the docker command, create a Unix group called docker and add users to it. When the docker daemon starts, it makes the ownership of the Unix socket read/writable by the docker group.

Warning: The docker group grants privileges equivalent to the root user. For details on how this impacts security in your system, see Docker Daemon Attack Surface.

[ravimeda]

Thanks for the clarification, Omair.