Powershell: Move-Item does not work across mounts on Linux

Created on 26 Jun 2020 · 7Comments · Source: PowerShell/PowerShell

Summary

Just started using PowerShell core on Linux and its fantastic so far! I can across an interesting bug regarding the Move-Item cmdlet. If I'm on the same mount point, it works as expected but if I'm moving items from one Mount to another it errors with a cross device link error.

Copy-Item is able to copy across devices as a work around. See below for details.

Hopefully this can get fixed in the 7.1 release.

Steps to reproduce

# Note: Source and destinations must be on different mounts shown by the "df" command.
$Source=/tmp/files/*
$Destination=/etc/files/
Move-Item $Source $Destination

Expected behavior

Files are removed from the source and exist at the destiation.

Actual behavior

Red Error Message:
    Move-Item: Invalid cross-device link

Environment data

$PSVersionTable:

Name                           Value
----                           -----
PSVersion                      7.0.2
PSEdition                      Core
GitCommitId                    7.0.2
OS                             Linux 3.10.0-1127.13.1.el7.x86_64 #1 SMP Tue Jun 23 15:46:38 UTC 2020
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Other System Information:
Operating System: CentOS Linux release 7.8.2003 (Core)
UNAME: Linux op2-snf-ldev4.cgifederal.com 3.10.0-1127.13.1.el7.x86_64 #1 SMP Tue Jun 23 15:46:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux


System Mounts:
Filesystem                                Type     1K-blocks     Used Available Use% Mounted on
devtmpfs                                 devtmpfs   3987736        0   3987736   0% /dev
tmpfs                                       tmpfs      4004528        0   4004528   0% /dev/shm
tmpfs                                       tmpfs      4004528     9768   3994760   1% /run
tmpfs                                       tmpfs      4004528        0   4004528   0% /sys/fs/cgroup
/dev/mapper/centos_root       xfs        5232640   181832   5050808   4% /
/dev/mapper/centos_usr         xfs        8378368  4752188   3626180  57% /usr
/dev/sda1                                xfs        1038336   298172    740164  29% /boot
/dev/mapper/centos_tmp        xfs        8378368   183096   8195272   3% /tmp
/dev/mapper/centos_home     xfs      157209600   185644 157023956   1% /home
/dev/mapper/centos_var         xfs        8378368  3279240   5099128  40% /var
/dev/mapper/centos_opt         xfs      209612800 16759668 192853132   8% /opt
/dev/mapper/centos_var--lib  xfs        8378368   428752   7949616   6% /var/lib
tmpfs                                       tmpfs       800908       12    800896   1% /run/user/42
tmpfs                                       tmpfs       800908        0    800908   0% /run/user/1391052081

Note that LVM is used on this system, not sure if that's causing cross-device linking issues or not.

Current Work Around

I am able to Copy-Item and then Remove-Item. This only impacts Move-Item for some reason.

Copy-Item $Source $Destination -Recurse
Remove-Item $Source

Area-Cmdlets-Utility Issue-Bug

Source

madisonjsmith

Most helpful comment

Looks like this happens here, and the cause for the error is in the Unix implementation here, which uses the rename syscall.

This occurs because rename does not support renaming across filesystems.

The mv coreutil handles this by trying rename first but if not, copying the file and deleting it later.

Unsure if .NET would consider this a bug, but the solution for us is probably to do the same as mv. It turns out we already try to do this:

https://github.com/PowerShell/PowerShell/blob/9212aac0fa020b657454ef4f9ff2fdd6fc6d759c/src/System.Management.Automation/namespaces/FileSystemProvider.cs#L5991-L5999

But the test we use is primitive and Windows-specific:

https://github.com/PowerShell/PowerShell/blob/9212aac0fa020b657454ef4f9ff2fdd6fc6d759c/src/System.Management.Automation/namespaces/FileSystemProvider.cs#L6097-L6103

We should improve this test to distinguish Unix mounts.

rjmholt on 27 Jun 2020

🎉1 👍1

All 7 comments

Red Error Message:
Move-Item: Invalid cross-device link

The full error message would help a lot more here. You can get this with Get-Error

rjmholt on 26 Jun 2020

I have a repro:

PS /home/sambauser/share> df
Filesystem      1K-blocks       Used  Available Use% Mounted on
udev              8141164          0    8141164   0% /dev
tmpfs             1631196      21620    1609576   2% /run
/dev/sda2       228167084    6241540  210265616   3% /
tmpfs             8155972         20    8155952   1% /dev/shm
tmpfs                5120          0       5120   0% /run/lock
tmpfs             8155972          0    8155972   0% /sys/fs/cgroup
/dev/sda1          523248      13152     510096   3% /boot/efi
/dev/sdc1      3844641568 1158717308 2490556952  32% /home/sambauser/share/4tb
/dev/sdb1      1921803492 1556889988  267221396  86% /home/sambauser/share/newhd
tmpfs             1631192          0    1631192   0% /run/user/1000

PS /home/sambauser/share> move-item /home/rob/Library_Copy/* /home/sambauser/share/4tb/Library_Destination/
Move-Item: Invalid cross-device link
Move-Item: Invalid cross-device link

PS /home/sambauser/share> get-error

Exception             :
    Type       : System.IO.IOException
    TargetSite :
        Name          : MoveDirectory
        DeclaringType : System.IO.FileSystem, System.IO.FileSystem, Version=4.1.2.0,
Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
        MemberType    : Method
        Module        : System.IO.FileSystem.dll
    StackTrace :
   at System.IO.FileSystem.MoveDirectory(String sourceFullPath, String destFullPath)
   at System.IO.DirectoryInfo.MoveTo(String destDirName)
   at Microsoft.PowerShell.Commands.FileSystemProvider.MoveDirectoryInfoItem(DirectoryInfo
directory, String destination, Boolean force)
    Message    : Invalid cross-device link
    Source     : System.IO.FileSystem
    HResult    : 18
TargetObject          : /home/rob/Library_Copy/Logs
CategoryInfo          : WriteError: (/home/rob/Library_Copy/Logs:DirectoryInfo) [Move-Item],
IOException
FullyQualifiedErrorId : MoveDirectoryItemIOError,Microsoft.PowerShell.Commands.MoveItemCommand
InvocationInfo        :
    MyCommand        : Move-Item
    ScriptLineNumber : 1
    OffsetInLine     : 1
    HistoryId        : 9
    Line             : move-item /home/rob/Library_Copy/*
/home/sambauser/share/4tb/Library_Destination/
    PositionMessage  : At line:1 char:1
                       + move-item /home/rob/Library_Copy/* /home/sambauser/share/4tb/Library_ …
                       + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    InvocationName   : move-item
    CommandOrigin    : Internal
ScriptStackTrace      : at <ScriptBlock>, <No file>: line 1
PipelineIterationInfo :

rjmholt on 27 Jun 2020

Looks like this happens here, and the cause for the error is in the Unix implementation here, which uses the rename syscall.

This occurs because rename does not support renaming across filesystems.

The mv coreutil handles this by trying rename first but if not, copying the file and deleting it later.

Unsure if .NET would consider this a bug, but the solution for us is probably to do the same as mv. It turns out we already try to do this:

https://github.com/PowerShell/PowerShell/blob/9212aac0fa020b657454ef4f9ff2fdd6fc6d759c/src/System.Management.Automation/namespaces/FileSystemProvider.cs#L5991-L5999

But the test we use is primitive and Windows-specific:

https://github.com/PowerShell/PowerShell/blob/9212aac0fa020b657454ef4f9ff2fdd6fc6d759c/src/System.Management.Automation/namespaces/FileSystemProvider.cs#L6097-L6103

We should improve this test to distinguish Unix mounts.

rjmholt on 27 Jun 2020

🎉1 👍1

.Net uses MOVEFILE_COPY_ALLOWED flag for MoveFileEx() and this works cross volumes.

For Unixes it should be fixed in .Net. Please vote in https://github.com/dotnet/runtime/issues/31149