Powershell: Misuse of `SHEmptyRecycleBin` in `Clear-RecycleBin` cmdlet

[GeeLaw]

Overview

Clear-RecycleBin produces spurious error records when used in Windows PowerShell. (Yes I know this repo is dedicated to PowerShell Core, but since it contains the code, and UserVoice isn't quite effective, I'll just put the analysis here.)

In a freshly launched PowerShell instance, running Clear-RecycleBin -Force will produce the following error (whether or not the recycle bin is empty does not matter at all):

Clear-RecycleBin : The system cannot find the path specified
At line:1 char:1
+ Clear-RecycleBin -Force
+ ~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (RecycleBin:String) [Clear-RecycleBin], Win32Exception
    + FullyQualifiedErrorId : FailedToClearRecycleBin,Microsoft.PowerShell.Commands.ClearRecycleBinCommand

However, subsequent invocations to the same command (with -Force) will not produce such error. Moreover, in a newly launched PowerShell instance, running Clear-RecycleBin (without -Force) and manually confirming the operation) will not write error.

Root Cause

Upon close investigation, the root cause starts on line 226 of ClearRecycleBinCommand.cs. Relevant excerpt:

uint result = NativeMethod.SHEmptyRecycleBin(IntPtr.Zero, drivePath,
    NativeMethod.RecycleFlags.SHERB_NOCONFIRMATION |
    NativeMethod.RecycleFlags.SHERB_NOPROGRESSUI |
    NativeMethod.RecycleFlags.SHERB_NOSOUND);

int lastError = Marshal.GetLastWin32Error();
progress.PercentComplete = 100;
progress.RecordType = ProgressRecordType.Completed;
WriteProgress(progress);
// 0 is for a successful operation
// 203 comes up when trying to empty an already emptied recyclebin
// 18 comes up when there are no more files in the given recyclebin
if (!(lastError == 0 || lastError == 203 || lastError == 18))
{
    Win32Exception exception = new Win32Exception(lastError);
    WriteError(new ErrorRecord(exception, "FailedToClearRecycleBin", ErrorCategory.InvalidOperation, "RecycleBin"));
}

The mistake here is to use GetLastWin32Error -- according to SHEmptyRecycleBin function (Windows) on MSDN, the function uses HRESULT to indicate error status. The documentation does not mention GetLastError, and it is true that the function does not call SetLastError on the caller thread before returning.

The fact that GetLastError returns 3 on the first invocation of Clear-RecycleBin -Force is purely accidental.

Playing around with SHEmptyRecycleBin indicates that it returns S_OK (0) for success (also documented), and E_UNEXPECTED (0x8000FFFF, catastrophic/unexpected failure) when the recycle bin is already empty. At this time, I would suggest remove error detection in the cmdlet, since SHEmptyRecycleBin doesn't know how to handle the case when the recycle bin is already empty at all -- we can't expect it to do any reasonable failure indication.

There could have been a time when SHEmptyRecycleBin uses SetLastError to indicate failure, and the code could have been written when the documentation wasn't updated.

---

I wrote a blog entry on this issue, and I found that the code is wrong in every way. If you're interested, read https://geelaw.blog/entries/clear-recyclebin-shemptyrecyclebin/

Mistakes made summarised into a list:

• not declaring [DllImport(SetLastError = true)] and using Marshal.GetLastWin32Error to retrieve the possible error code set by SHEmptyRecycleBin, which won't work and will get the leftover of previous marshalled calls.
• documentation for SHEmptyRecycleBin doesn't indicate that it will call SetLastError, and recommends that one use the return value to determine error status.
• SHEmptyRecycleBin doesn't expect to deal with already emptied recycle bin.

I would recommend anyone using this cmdlet use Clear-RecycleBin -ErrorAction SilentlyContinue to prevent irrelevant ErrorRecord.

[iSazonov]

/cc @SteveL-MSFT Have we plan to add the cmdlet to PowerShell Core?

[SteveL-MSFT]

@iSazonov no plan to add it in the near future, but certainly desirable but we should have a cross-platform solution

[iSazonov]

What is RecycleBin on Unix? Is there a standard for Trash?

/cc @mklement0 Could you please comment too?

[mklement0]

@iSazonov:

I don't know much about this, but here's what I can offer:

• There's no standard, because there's no standardized desktop environment across Linux distros.

• Individual desktop environments and their file managers may have a recycle-bin concept, such as the Nautilus file manager on Ubuntu.

• On macOS, the Finder file manager has a special, virtual folder named "Trash", showing "recycled" items across volumes, based on hidden, volume-specific folders.

• In all cases, the standard rm utility doesn't use the recycle bin (and neither does Powershell's Remove-Item).

[iSazonov]

@mklement0 Thanks! You confirm my thoughts that we can implement the cmdlet on Windows and MacOs but not on Linux.

[jeffbi]

On my Linux box (Mint 18.2, based on Ubuntu 16.04, using Xfce), deleting a file using VSCode puts the deleted file into the "trash", which is ~/.local/share/Trash/. If I use the Thunar file explorer to look at the "Trash" folder, I see the deleted file. This suggests to me that these is at least some level of standardization on Linux to implement recycle-bin operations, although it may require the presence of a desktop environment such as GNOME, KDE, or Xfce.

I haven't looked into exactly how VSCode implements trash, but I believe VSCode is fairly desktop-environment agnostic.

[iSazonov]

@jeffbi Thanks! I agree that the Trash comes from desktop applications.

I found Freedesktop.org Trash spec

Ubuntu has it https://help.ubuntu.com/stable/ubuntu-help/files-recover.html
https://askubuntu.com/questions/327943/how-to-open-trash-through-terminal

What other Linux distributives implement this standard? MacOs?

[iSazonov]

On MacOs http://kb.mit.edu/confluence/display/istcontrib/View+Trash+directory+in+Mac+OS+X+via+Terminal
cd ~/.Trash

Not clear about $XDG_DATA_HOME/Trash

[mklement0]

Good point about the freedesktop.org specification, which applies to X11 (X Window System) desktops, namely the following: https://www.freedesktop.org/wiki/Desktops/

Sounds like that may cover the major Linux distros.

The freedesktop.org doesn't apply to macOS, however, which uses Quartz.

Also, in addition to the user's trash folder at ~/.Trash, macOS has volume-specific trash folders at {mountpoint}/.Trashes/._{UID} , where {UID} is the user's UID as reported by id -u.

From glancing at https://specifications.freedesktop.org/trash-spec/trashspec-latest.html, it seems that the freedesktop.org similarly, but _optionally_ supports volume-specific trash folders via a single $topdir/.Trash, with user-specific UID-named subfolders.

[iSazonov]

I hope we don't need to implement this standard from scratch. It is better to use existing API.

[jeffbi]

After poking around a little, it looks to me as if VSCode (via Electron) implements move-to-trash via one of several command-line programs. https://github.com/electron/electron/pull/7178

[mklement0]

For macOS it's implemented in Objective-C++ (I didn't even know that existed) with direct calls to the Cocoa Foundation framework:

https://github.com/aichingm/electron/blob/893fc2cd539cb4ccef01833a1209696e448df8d9/atom/common/platform_util_mac.mm#L155

Not sure how easy it is to call Cocoa from .NET Core, but it seems to be possible from Mono.

[iSazonov]

Guys, this is more suitable for Remove-Item enhancement.

[jeffbi]

Sorry. My point was that Clear-RecycleBin should be able to use a command-line tool such as gvfs-rm to clear the trash (recycle bin) on Linux.

[mklement0]

Similarly, on macOS you can do it via the standard osascript utility:

_Update_: Based on the discussion below, application "Finder" was replaced with application id "com.apple.finder"

osascript -e 'tell application id "com.apple.finder" to if (count of items in trash) > 0 then empty trash'

_Update: A more robust way to target the macOS Finder is to use_ application id "com.apple.Finder"

You can use the same technique to trash a file or folder:

osascript -e 'tell application id "com.apple.finder" to delete POSIX file "/path/to/some/file-or-folder"'

Note that if you want to try these commands from pwsh, you need to \-escape the " instances (sigh).

[GeeLaw]

Sadly no one's caring about Windows :-(

@mklement0 (last reply) I searched on the Internet for a while and couldn't find any documentation on how application resolving is done, so I'm not sure if the script works on macOS in another language (where Finder is not translated as "Finder"). -- Please do tell me whether it does or not (I don't have a Mac so cannot test it out). A better solution is to use a path to specify the app.

@iSazonov Did you mean to allow recycling items instead of deleting items in Remove-Item? This doesn't seem to be very useful as the recycle bin is often not organised, and it would be hard to choose which files to recover in the bin. Instead, one can prevent catastrophic removal by Move-Items into a temporary folder (in the same volume) dedicated for that operation.

[mklement0]

@GeeLaw:

I'm not sure if the script works on macOS in another language

The language of the -e argument passed to osascript is AppleScript.
There is no need for another language if you invoke the osascript CLI, i.e., an external executable.

Or am I missing something?

how application resolving is done

macOS has a central register of applications called the Launch Services database.

A named passed to application can be the display name (CFBundleDisplayName) or the short name (CFDisplayName).

Alternatively, you can pass either the full path or the bundle ID, which indeed is the more robust choice (though it's unlikely that a 3rd-party application will attempt to use the name "Finder"):

The following two AppleScript commands both unambiguously target the Finder and return its name:

tell application id "com.apple.finder" to name   
tell application "/System/Library/CoreServices/Finder.app" to name

[GeeLaw]

@mklement0 I was talking about natural language. Finder is 访达 in Chinese (a recent update changed its name, was “Finder”). Not sure how the system deals with multilingual display name — I’m not familiar with macOS. And, yeah, it would be better to use id "com.apple.Finder".

[mklement0]

@GeeLaw:

I see. The English name should work even with a different UI language in effect, but I agree that application id "com.apple.finder" is better and I've updated the code samples above.

(I've lowercased the .Finder component, because I've since realized that that is the original case - however, it also works with the case variant.)

Sadly no one's caring about Windows :-(

@SteveL-MSFT: Perhaps this issue needs the Consider-WindowsPowerShell51 label?

[iSazonov]

I believe we need new Issue for porting the cmdlet.

Seems we could use mentioned above Electron algorithm as reference for Linux. What API the Electron use for MacOs?

[mklement0]

@iSazonov

Agreed re new issue.

While we're at it:

• Something like Move-ItemToTrash to send filesystem items to the trash instead of deleting them instantly might be handy (though only the FileSystem provider would support it).

  • (I don't think there's a need for a restore-from-trash cmdlet, as that should be a deliberate, interactive user action, though perhaps a Show-Trash cmdlet might be useful).

• I'm deliberately using Trash rather than RecycleBin as the noun, because _trash_ is the term used on macOS and in the freedesktop.org spec. on Linux; we could provide aliases, though.

---

What API the Electron use for MacOs?

See my previous comment.

However, the Cocoa Foundation framework only supports _moving an item_ to the trash, it doesn't support _emptying_ the trash.

Hence my suggestion to use command-line utility osascript with a piece of AppleScript for simplicity.

We _could_ do it in-process, because you can call AppleScript from Cocoa too, but I'm not sure it's worth the effort - unless it's trivial to call Cocoa from .NET Core - I have not looked into it.

[GeeLaw]

@mklement0

On introducing the new functionality:

• I would suggest using Remove-Item with FileSystemProvider giving dynamic switch parameter ToRecycleBin. FileSystemProvider already uses dynamic switch parameters to provide functionality specific to itself.
• I would suggest RecycleBin for consistency with current cmdlets. Instead, I'd consider using Trash as the aliased noun. Plus, it doesn't make sense to stick to the name with more specs using it. I'd like it when PowerShell keeps more pure Windows genes and that it spreads them to the other world. (I'm a Microsoft fanboy, and you could ignore this suggestion, and I will understand.)
• Restoring from Recycle Bin could be very hard to use -- it would be hard to locate an item. One possible way for this is to let the cmdlet return a set of recycled items, where each item has an identifier (consistent across invocations but platform-dependent), date of removal, restoration path (sometimes might be unavailable) etc., and another cmdlet would restore items. I would suggest not having such cmdlets.
• However, it's rare that PowerShell provides something dedicated to opening GUI. The only two examples I've found are Show-WindowsDeveloperLicenseRegistration, which opens up Settings app on Windows 10, and Out-GridView, which outputs the objects into an interactive window for inspection. Plus, such functionality isn't very useful in scripting automation. (The last sentence is arguable because PowerShell already provides some cmdlets with GUI. I would say Show-WindowsDeveloperLicenseRegistration isn't very useful, which can be replaced by Start-Process ms-settings:developers, which doesn't do the unnecessary administrative privilege check before it opens something that cannot be administrative-privileged.)

[mklement0]

@GeeLaw:

• > I would suggest using Remove-Item with FileSystemProvider giving dynamic switch parameter ToRecycleBin.

Good idea.

• Regarding the terminology:

Making Trash the alias sounds good to me.

Plus, it doesn't make sense to stick to the name with more specs using it.

What do you mean?

My objection to _recycle bin_ is more on _logical_ grounds:
Recovering something from the trash is not an act of _recycling_.

• Re not providing a way to open the trash/recycle-bin GUI:

I agree that we can do without it, but regarding your general point:

such functionality isn't very useful in scripting automation

Yes, but PowerShell is also a shell for _interactive_ convenience. For instance, Out-GridView is probably rarely used in scripting, but it's a great tool for experiments / analysis of outputs from the command line.

[GeeLaw]

@mklement0 (Didn't know one could add more than one reactions to one reply!)

What I meant by "to stick to the name with more specs using it" should have been phrased as "to stick to a name just because two specs (macOS + freedesktop.org, more than one, namely Windows) use it", as it read to me that you initially suggested using Trash as the (main?) noun because of the number of specs using that name.

The logical ground is interesting and more sound. Now I think Trash should be paired up with Restore verb if we indeed used it.

[iSazonov]

Seems we could get the cmdlet in 6.1.

@mklement0 Could you please open new Issue to discuss new cmdlets?

[mklement0]

@iSazonov: Please see #6801.

[msftbot[bot]]

:tada:This issue was addressed in #10909, which has now been successfully released as v7.0.0-preview.6.:tada:

Handy links:

• Release Notes