Modernization: Cannot convert a page if source (or possibly target) list has more than 5000 items

[rudfoss]

Category

[X] Bug
[ ] Enhancement

Problem Area

[ ] Page Transformation: Error during the setup/use of the Page Transformation UI solution (did you check our troubleshooting guide?)
[X] Page Transformation: Error during the use of page transformation from PnP PowerShell
[ ] Page Transformation: Error during the use of page transformation from .Net
[ ] Page Transformation: Page is not looking correct after transformation
[ ] Modernization Scanner: something went wrong...

Expected or Desired Behavior

Running ConvertTo-PnPClientSidePages for a file on a source or target library with more than 5000 items works without problems.

Observed Behavior

Running ConvertTo-PnPClientSidePages for a file on a source library with more than 5000 items results in an error message because the list view threshold is exceeded.

ConvertTo-PnPClientSidePage : Attempted operation is not allowed because it exceeds the list view threshold enabled by the system administrator.
At line:1 char:1
+ ConvertTo-PnPClientSidePage -Identity "70-people-detained-on-suspecti ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : WriteError: (:) [ConvertTo-PnPClientSidePage], ServerException
    + FullyQualifiedErrorId : EXCEPTION,SharePointPnP.PowerShell.Commands.ClientSidePages.ConvertToClientSidePage

Get-PnPException output

CorrelationId    : 5a83279f-4049-2000-2897-cac0e77ed06a
TimeStampUtc     : 02.01.2020 11:42
Message          : Attempted operation is not allowed because it exceeds the list view threshold enabled by the system administrator.
Stacktrace       :    at Microsoft.SharePoint.Client.ClientRequest.ProcessResponseStream(Stream responseStream)
                      at Microsoft.SharePoint.Client.ClientRequest.ProcessResponse()
                      at Microsoft.SharePoint.Client.ClientRequest.<ExecuteQueryToServerAsync>d__6.MoveNext()
                   --- End of stack trace from previous location where exception was thrown ---
                      at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                      at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                      at Microsoft.SharePoint.Client.ClientRequest.<ExecuteQueryAsync>d__0.MoveNext()
                   --- End of stack trace from previous location where exception was thrown ---
                      at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                      at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                      at Microsoft.SharePoint.Client.ClientRuntimeContext.<ExecuteQueryAsync>d__0.MoveNext()
                   --- End of stack trace from previous location where exception was thrown ---
                      at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                      at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                      at Microsoft.SharePoint.Client.ClientContext.<ExecuteQueryAsync>d__4.MoveNext()
                   --- End of stack trace from previous location where exception was thrown ---
                      at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                      at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                      at Microsoft.SharePoint.Client.ClientContextExtensions.<ExecuteQueryImplementation>d__7.MoveNext()
                   --- End of stack trace from previous location where exception was thrown ---
                      at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                      at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                      at Microsoft.SharePoint.Client.ClientContextExtensions.ExecuteQueryRetry(ClientRuntimeContext clientContext, Int32 retryCount, Int32 delay, String userAgent)
                      at SharePointPnP.PowerShell.Commands.Base.PipeBinds.PagePipeBind.GetPage(Web web, String listToLoad)
                      at SharePointPnP.PowerShell.Commands.ClientSidePages.ConvertToClientSidePage.ExecuteCmdlet()
                      at SharePointPnP.PowerShell.Commands.PnPCmdlet.ProcessRecord()
ScriptLineNumber : 1

Steps to Reproduce

Add more than 5000 items to an old publising "Pages" library and try to convert one page from classic to modern using ConvertTo-PnPClientSidePages.

PnP Version: 3.16.1912.0

Edit: Updated with English error message.

[jansenbe]

Translated error message: "Attempted operation is not allowed because it exceeds the list view threshold enabled by the system administrator."

[jansenbe]

Hey @rudfoss ,

Can you try with specifying the ID of the source page as Identity parameter when you call ConvertTo-PnPClientSidePage (https://docs.microsoft.com/en-us/sharepoint/dev/transform/modernize-userinterface-site-pages-powershell#options-for-the-convertto-pnpclientsidepage-cmdlet)? Using this approach the public ListItem GetItemById(int id); CSOM method is used instead of doing a caml query.

You however also will need to ensure that the initial query that you use to obtain the pages works in safe way: using paging should help as shown in this example: Get-PnPListItem -List sitepages -PageSize 10

Would be good if you could check if this unblocks your scenario today, we'll be shipping the January release soon and if there are changes needed I would like to get them still included in that release.

[rudfoss]

@jansenbe Using the ID works! Just verified it with the same file both with file name and ID. Should've thought of this myself though since the parameter is -Identity. Thanks for the quick reply and work-around. I have a way to get the IDs without bumping into the limit so this should be good.

For anyone curious I'm using this query:

<View Scope="RecursiveAll">
    <Query>
        <OrderBy>
            <FieldRef Name="ID" Ascending="TRUE"/>
        </OrderBy>
    </Query>
    <RowLimit Paged="TRUE">5000</RowLimit>
</View>

Along with these calls:

$sourceWebConnection = Connect-PnPOnline -Url "[absolute url to web]" -ReturnConnection
$sourceList = Get-PnPList "Pages" -Connection $sourceWebConnection
$items = Get-PnpListItem -List $sourceList -Query $query -Connection $sourceWebConnection

To get all list items beyond the 5k limit. Which I can then use IDs from to perform the convertion.

[jansenbe]

Good to hear it works, I've updated the sample scripts to use ID by default in combination with a page query.

You're the first one with 5000+ publishing pages in a single library, quite a big intranet :-)

[jansenbe]

Closing as issue is resolved.

[rudfoss]

@jansenbe it's one single sub site in about 120 with a Pages library with just above 5000 items. I guess I just got lucky ;)