Pnp-powershell: Copy-PnPFile : The remote server returned an error: (500) Internal Server Error.

[gpdhillon]

I'm trying to copy files from document library to other on SharePoint Online using Copy-PnPFile. All the files are getting copied except for .json file. For JSON file throws "Copy-PnPFile : The remote server returned an error: (500) Internal Server Error."

Steps to reproduce.
1) open connection

$connProjSite = Connect-PnPOnline -Url https://**dev.sharepoint.com/sites/TestSIte

2) Get files from the folder
[Microsoft.SharePoint.Client.Folder]$spFolder = Get-PnPFolder -Url "/sites/TestSIte/Shared Documents/Test" -Includes Files -Connection $connProjSite -ErrorAction Continue

3) Loop thru to copy files
$spFolder.Files | ForEach-Object {
[Microsoft.SharePoint.Client.File]$spFile = $_

Write-Verbose -Verbose -Message "Moving file $($spFile.Name) to $($clientSiteRelativeURL)/$($projectLegalFolderPath)"
Copy-PnPFile -SourceUrl $spFile.ServerRelativeUrl -TargetUrl "/sites/TestSite/Shared Documents/Test2/$($spFile.Name)" -Connection $connProjSite -OverwriteIfAlreadyExists -Force

}

This works fine on most documents, but gives the following error on JSON files.

Copy-PnPFile : The remote server returned an error: (500) Internal Server Error.
At C:\Development\SiteProvisioning\PowerShell\JSON Test.ps1:9 char:5

• Copy-PnPFile -SourceUrl $spFile.ServerRelativeUrl -TargetUrl "/si ...
• ~~~~~~~~~~~~~
  
  
  
  • CategoryInfo : WriteError: (:) [Copy-PnPFile], WebException
  
  
  • FullyQualifiedErrorId : EXCEPTION,SharePointPnP.PowerShell.Commands.Files.CopyFile
  
  

Which version of the PnP-PowerShell Cmdlets are you using?

• [ ] PnP PowerShell for SharePoint 2013
• [ ] PnP PowerShell for SharePoint 2016
• [ ] PnP PowerShell for SharePoint 2019
• [x ] PnP PowerShell for SharePoint Online

What is the version of the Cmdlet module you are running?

3.22

How did you install the PnP-PowerShell Cmdlets?

• [ ] MSI Installed downloaded from GitHub
• [x ] Installed through the PowerShell Gallery with Install-Module
• [ ] Other means

[msftbot[bot]]

Thank you for reporting this issue. We will be triaging your incoming issue as soon as possible.

[KoenZomers]

I can reproduce this. When doing the copy, it will request the URL to the SharePoint site where the file resides. This triggers a call in CSOM to:

POST /sites/\

This call fails if it's done on a .json file. All other file types seem to work fine, so it looks like an issue on the SharePoint Server side. Trying to fetch ULS logs to get a deeper insight.

[gpdhillon]

Thanks for following up. So this something that cannot be fixed easily?

[kadu-jr]

The issue happens in this exact line of the CopyFile command. As Koen pointed out, a call to figure the web folder URL fails when sending the URL of a JSON file. With a simple fix to the code I was able to make it work for JSON files. I will try to test it in a new branch and if it works for other file extensions I will be submitting a pull request.

[KoenZomers]

I had a look at the ULS logs and they weren't really helpful. Tried to find out in the source code of SharePoint Online why it failed but got stranded and ran out of time to spend on the issue. The call is only used to determine the URL of the site anyway, so we should be able to find another way to do so.

I'll have a look at @kadu-jr his proposed solution, but it may not be that easy as we also need to think about different subsites and such. Have you tested your PR with all possible thinkable scenarios?

[kadu-jr]

You're right, @KoenZomers. I had forgotten this command can also copy a folder instead of a single file, among with other examples. I looked through the command examples and I believe the solution would be to check if the SourceUrl parameter ends in ".json" and treat it to generate the sourceWebUri, which may be an ugly workaround.
If you'd like this solution to be used, while the bug still exists server-side, I can submit a new PR. Otherwise we'll just have to wait until you guys can fix it.