Runtime: Bundle Extraction Directory: Default to a users-specific tmp directory on Unix systems

[swaroop-sridhar]

In .net core 3, single file apps run by extracting the bundled contents to a temp directory.
The extraction directory is machine specific, and can be set through DOTNET_BUNDLE_EXTRACT_BASE_DIR environment variable.

When this setting is not configured, the host tries to use certain default directories.
On windows, extraction is within %TMPDIR%, which is user specific.
On Unix systems (like MAC) where $TMPDIR is set, extraction happens there -- and this is also user specific.
Otherwise, the extraction directory is within /var/tmp/ or /tmp/ which is common to all users, and may be locked by a specific user on first creation. Therefore, fix this issue by choosing something like /var/tmp/<user-id>/.net/... or $HOME/.net/....

This fix ameliorates the current problem, but is still robust -- especially within the TMP directory because any user can create a similar directory structure inaccessible to others. So the best way to configure the extraction directory is by setting BUNDLE_EXTRACTION_BASE_DIR.

[am11]

Isn't the plan to forgo the entire bundle extraction mechanism in future, and read all bundle constituents from mmap?

[swaroop-sridhar]

@am11 in .net 5, we'll reduce the reliance on extraction for single-file apps. However, extraction may still be supported for including custom native DLLs within the single-file app.

We also want to fix the extraction directory permission issue for a servicing fix to 3.1.

[swaroop-sridhar]

In order to fix this problem, a few options are:

1. Set permissions of /var/tmp/.net/... as 777
   
   
   
   • Pros: Share extraction across all users
   
   
   • Cons: Shares the extraction across users
   
   
2. Choose /var/tmp//.net as the default extraction location
   
   
   
   • Pros: Extraction cleaned up routinely
   
   
   • Cons: Anybody can create the /var/tmp// directory and lock out other users from executing single-file apps
   
   
3. Choose $HOME/.net as the default extraction location
   
   
   
   • Pros: The directory is completely under user control
   
   
   • Cons: Not cleaned up by the system.
   
   
4. Use a scheme similar to mkdtemp logic, as explained by @lpereira in this PR: https://github.com/dotnet/designs/pull/88
   
   
   
   • Pros: More robust tmp-directory creation logic, better guard against name-collisions
   
   
   • Cons: Maintains non-temp state, more complex logic.
   
   

If single-file bundles use extraction in .net 5 (for certain file types), we should use solution (4) above.

In .net 3.1 servicing fix, we’ll go after a simpler solution, by extracting to a user-specific directory:
/var/tmp/.net/<user-ID>/<app>/…, where /var/tmp/.net has permission 0777, and /var/tmp/.net/<user-ID>/… 0700.

[JustArchi]

Hey @swaroop-sridhar, could you let me know if this fix is expected to land in 3.1.4, or maybe has already landed in 3.1.x series? Thank you in advance for answering.

[swaroop-sridhar]

This fix is expected in 3.1.4.

[elxbarbosa]

Same here!

[xPaw]

I just tried the 3.1.4 build, and it hasn't really improved the bundle extraction. It still requires the user to set DOTNET_BUNDLE_EXTRACT_BASE_DIR.

Example:

$ ./SteamTokenDumper
realpath(): Permission denied
Failure processing application bundle.
Failed to determine location for extracting embedded files
DOTNET_BUNDLE_EXTRACT_BASE_DIR is not set, and a read-write temp-directory couldn't be created.
A fatal error was encountered. Could not extract contents of the bundle

And if you set DOTNET_BUNDLE_EXTRACT_BASE_DIR to a folder that does not exist, it simply segfaults, without any useful message:

$ export DOTNET_BUNDLE_EXTRACT_BASE_DIR="test"
$ ./SteamTokenDumper
[1]    1203 segmentation fault (core dumped)  ./SteamTokenDumper

After creating the test folder, it will extract and run correctly.

EDIT: After reading through the PR carefully, this change will only work if the tmp/.net currently does not exist on the machine.

[swaroop-sridhar]

@xPaw can you please add a few more details:

• Which OS are you running the app on?
• The single-file app was rebuilt with 3.1.4 SDK, right?
• Have you deleted the tmp-directory (/vat/tmp/.net or /tmp/.net) created by apps built with previous SDKs.
• What is the permissions on /var/tmp/.net/ and /var/tmp/.net/<username>/? I checked on a Ubuntu machine that the permissions are set correctly /var/tmp/.net/ 0x777 and /var/tmp/.net/<username>/ 0x700.

[xPaw]

I built on Windows 10, and then tried to run it on Debian 10 (both a separate server, and WSL). It was built with 3.1.4.

The problem is that /var/tmp/.net already existed from previous apps extracting there, so the permissions were set incorrectly. This is kind of a poor user experience especially when you run apps that were built against earlier SDK version.

Besides, as you can see changing DOTNET_BUNDLE_EXTRACT_BASE_DIR to a folder that does not exist produces a segfault instead of a useful message, or just creating the folder.

[swaroop-sridhar]

Thanks for confirming @xPaw. Any colliding directory structures that mask the extraction directory (whether created by an earlier .net host, or via any other process will need to be cleaned up) for the apps to work correctly. Otherwise, a non-colliding path should be specified using DOTNET_BUNDLE_EXTRACT_BASE_DIR.

Did you provide an absolute path in DOTNET_BUNDLE_EXTRACT_BASE_DIR ?
I think the problem is not whether the directory exists or not. But there seems to be a bug when the path doesn't contain a '/' in it. I have filed https://github.com/dotnet/runtime/issues/36804 for this.