Sp-dev-docs: Documentation describes REST API max file size as 2gb, but it is actually 250mb.

[TuomasPakkanen]

Category

• [ ] Question
• [ ] Typo
• [X] Bug
• [ ] Additional article idea

Expected or Desired Behavior

We are trying to upload large files via REST API and have also tried with PnP PowerShell, but are getting error message about file size (roughly 250mb), even though documentation states 2gb as the max file size.

Premier support pointed to this article:
https://blogs.technet.microsoft.com/sharepointdevelopersupport/2016/11/23/always-use-file-chunking-to-upload-files-250-mb-to-sharepoint-online/

Observed Behavior

See above.

Steps to Reproduce

See above.

Submission Guidelines

Please update the documentation or preferrably increase the file size from 250mb to at least 1gb to support more scenarios for simplicity. The bulk approach is more difficult to implement with integration tools.

[msft-github-bot]

Thank you for reporting this issue. We will be triaging your incoming issue as soon as possible.

[Dennis-LZ]

To upload large file(more than 250mb) to SharePoint Online library, we can use CSOM with PowerShell to achieve it.

In my test, I upload the file size 500mb to the SharePoint Online library using the PowerShell script below.

$SiteURL = "https://tenant.sharepoint.com/sites/team"
$libraryName="DocumentLibrary"
$SPOnlineUserName="[email protected]"
$SPOnlinePassword="xxx"
$fileDirectory="C:\temp"
Try{
    Add-Type -Path "c:\Program Files\Common Files\microsoft shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll" 
    Add-Type -Path "c:\Program Files\Common Files\microsoft shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"
}
catch {
    Write-host "No dlls found"
}
Function UploadFileInSlice ($ctx, $libraryName, $fileName, $fileChunkSizeInMB) {

    $fileChunkSizeInMB = 9

    # Each sliced upload requires a unique ID.
    $UploadId = [GUID]::NewGuid()

    # Get the name of the file.
    $UniqueFileName = [System.IO.Path]::GetFileName($fileName)


    # Get the folder to upload into. 
    $Docs = $ctx.Web.Lists.GetByTitle($libraryName)
    $ctx.Load($Docs)
    $ctx.Load($Docs.RootFolder)
    $ctx.ExecuteQuery()

    # Get the information about the folder that will hold the file.
    $ServerRelativeUrlOfRootFolder = $Docs.RootFolder.ServerRelativeUrl

    # File object.
    [Microsoft.SharePoint.Client.File] $upload


    # Calculate block size in bytes.
    $BlockSize = $fileChunkSizeInMB * 1024 * 1024


    # Get the size of the file.
    $FileSize = (Get-Item $fileName).length

    if ($FileSize -le $BlockSize)
    {
        # Use regular approach.
        $FileStream = New-Object IO.FileStream($fileName,[System.IO.FileMode]::Open)
        $FileCreationInfo = New-Object Microsoft.SharePoint.Client.FileCreationInformation
        $FileCreationInfo.Overwrite = $true
        $FileCreationInfo.ContentStream = $FileStream
        $FileCreationInfo.URL = $UniqueFileName
        $Upload = $Docs.RootFolder.Files.Add($FileCreationInfo)
        $ctx.Load($Upload)
        $ctx.ExecuteQuery()
        return $Upload
    }
    else
    {

        # Use large file upload approach.
        $BytesUploaded = $null

        $Fs = $null

        Try {

            $Fs = [System.IO.File]::Open($fileName, [System.IO.FileMode]::Open, [System.IO.FileAccess]::Read, [System.IO.FileShare]::ReadWrite)

            $br = New-Object System.IO.BinaryReader($Fs)

            $buffer = New-Object System.Byte[]($BlockSize)
            $lastBuffer = $null
            $fileoffset = 0
            $totalBytesRead = 0
            $bytesRead
            $first = $true
            $last = $false

            # Read data from file system in blocks. 
            while(($bytesRead = $br.Read($buffer, 0, $buffer.Length)) -gt 0) {

                $totalBytesRead = $totalBytesRead + $bytesRead

                # You've reached the end of the file.
                if($totalBytesRead -eq $FileSize) {
                    $last = $true
                    # Copy to a new buffer that has the correct size.
                    $lastBuffer = New-Object System.Byte[]($bytesRead)
                    [array]::Copy($buffer, 0, $lastBuffer, 0, $bytesRead)
                }

                If($first)
                {
                    $ContentStream = New-Object System.IO.MemoryStream

                    # Add an empty file.
                    $fileInfo = New-Object Microsoft.SharePoint.Client.FileCreationInformation
                    $fileInfo.ContentStream = $ContentStream
                    $fileInfo.Url = $UniqueFileName
                    $fileInfo.Overwrite = $true
                    $Upload = $Docs.RootFolder.Files.Add($fileInfo)
                    $ctx.Load($Upload)

                    # Start upload by uploading the first slice.

                    $s = New-Object System.IO.MemoryStream(,$buffer) 

                    # Call the start upload method on the first slice.
                    $BytesUploaded = $Upload.StartUpload($UploadId, $s)
                    $ctx.ExecuteQuery()

                    # fileoffset is the pointer where the next slice will be added.
                    $fileoffset = $BytesUploaded.Value

                    # You can only start the upload once.
                    $first = $false

                }
                Else
                {
                    # Get a reference to your file.
                    $Upload = $ctx.Web.GetFileByServerRelativeUrl($Docs.RootFolder.ServerRelativeUrl + [System.IO.Path]::AltDirectorySeparatorChar + $UniqueFileName);

                    If($last) {

                        # Is this the last slice of data?
                        $s = New-Object System.IO.MemoryStream(,$lastBuffer)


                        # End sliced upload by calling FinishUpload.
                        $Upload = $Upload.FinishUpload($UploadId, $fileoffset, $s)
                        $ctx.ExecuteQuery()


                        Write-Host "File upload complete"
                        # Return the file object for the uploaded file.
                        return $Upload

                    }
                    else {

                        $s = New-Object System.IO.MemoryStream(,$buffer)

                        # Continue sliced upload.
                        $BytesUploaded = $Upload.ContinueUpload($UploadId, $fileoffset, $s)
                        $ctx.ExecuteQuery()

                        # Update fileoffset for the next slice.
                        $fileoffset = $BytesUploaded.Value

                    }
                }

            }  #// while ((bytesRead = br.Read(buffer, 0, buffer.Length)) > 0)
        }
        Catch {
            Write-Host $_.Exception.Message -ForegroundColor Red
        }
        Finally {

            if ($Fs -ne $null)
            {
                $Fs.Dispose()
            }

        }
    }
    return $null

} 
$Context = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL)
$securePassword=ConvertTo-SecureString $SPOnlinePassword -AsPlainText -Force
$Context.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($SPOnlineUserName, $securePassword)
foreach($fileName in Get-ChildItem $fileDirectory)
{
    $UpFile = UploadFileInSlice -ctx $Context -libraryName $libraryName -fileName $fileName.FullName
    $Context.Dispose();
}

Dennis Guo
Microsoft SharePoint Community Support

[TuomasPakkanen]

Hi. Any idea how this could be accomplished in HTTP REST? Seems a rather long script for a simple thing, but I do see the benefits of chunking. I have a use case where CSOM may not be usable, so REST API way of uploading large files would be nice! Thanks for the comment anyway!

[tavikukko]

Hi Tuomas,
Have you tried with PnPjs? It's fluent api for SP REST APIs?
Here's a reference for adding files larger than 10mb using .addChunked method:
https://pnp.github.io/pnpjs/sp/docs/files/#adding-files

And here is sample from @koltyakov using node&PnPjs for large file uploads:
https://github.com/koltyakov/pnp-upload

I hope this helps!
cheers,
Tomi

[TuomasPakkanen]

Thanks a lot. These examples are helpful. I'm not directly responsible for this scenario, but I'll report them to the responsible people. Thanks again!

[msft-github-bot]

Closing issue as "answered". If you encounter similar issue(s), please open up a NEW issue. Thank you.

[msft-github-bot]

Issues that have been closed & had no follow-up activity for at least 7 days are automatically locked. Please refer to our wiki for more details, including how to remediate this action if you feel this was done prematurely or in error: Issue List: Our approach to locked issues