Pnp-modern-search: Upgrade Logistics for v3 -> v4

[sympmarc]

Is your feature request related to a problem? Please describe.
I'd like a good way to upgrade from v3 to v4 without breaking anything. Because these Web Parts are so useful, we usually deploy tenant-wide.

Describe the solution you'd like
A way to identify where v3 Web Parts are in use so we can upgrade to v4 and eventually remove v3 altogether.

Describe alternatives you've considered
I'm probably going to write some PowerShell for this. It would need to loop through all the sites AND the pages to see if the Web Parts are in use - but there's no pages API. I just wondered if you already had something I could start with.

[wobba]

See https://www.techmikael.com/2019/02/locate-pages-where-particular-web-part.html

Pick the result wp manifest id as the query guid 🙂

[sympmarc]

Nice! I didn't think of using search. I'll post back with whatever I come up with.

[wobba]

Awesome!

[sympmarc]

Here's what I've come up with. It seems to work well in my test tenant. Larger tenants may be less pleasant to deal with.

Connect-PnPOnline -Url https://sympmarc-admin.sharepoint.com -UseWebLogin

# You can get the Ids for the v3 Web Parts by adding them to a page and running:
# Get-PnPClientSideComponent -Page "page-name" | Where-Object { Title -ne $null } | Select-Object Title, WebPartId

# Title            WebPartId
# -----            ---------
# Search Filters   e899ac12-9256-4c8d-a8ad-dbd20fc459c3
# Search Box       096b96cc-8a44-41fa-9b4d-c0ab2ab2a779
# Search Verticals 9d441773-f735-46a3-9ca0-9e2eb4bef203
# Search Results   42ad2740-3c60-49cf-971a-c44e33511b93

$webPartIds = @(
    "e899ac12-9256-4c8d-a8ad-dbd20fc459c3",
    "096b96cc-8a44-41fa-9b4d-c0ab2ab2a779",
    "9d441773-f735-46a3-9ca0-9e2eb4bef203",
    "42ad2740-3c60-49cf-971a-c44e33511b9"
    )

# Get all the sites
$sites = Get-PnPTenantSite

# You may choose to exclude some subsites of sites
$filteredSites = $sites #| Where-Object { $_.Url -eq 'https://sympmarc.sharepoint.com/sites/DemoSite' } 

# Loop through te sites
foreach ($site in $filteredSites) {

    # Build the query
    $query = "Path:$($site.Url) AND FileExtension:aspx AND ($($webPartIds -join " OR "))"
    Write-Host -BackgroundColor White -ForegroundColor Black "Looking in $($site.Url)" # $($query)"

    # Submit the query
    $pages = Submit-PnPSearchQuery -Query $query -All -RelevantResults -ErrorAction SilentlyContinue  | Select-Object OriginalPath 
    # Idf there are results, display them
    if ($pages) {
        foreach ($page in $pages) {
            Write-Host -BackgroundColor Green -ForegroundColor Black "Found Web Parts in $($page.OriginalPath)"                
        }
    }

}

[sympmarc]

I've run the script above on several client tenants, and it works a charm. Obviously, it doesn't make any changes, but it shows me exactly where we have work to do (or postpone).

[sympmarc]

I spotted one issue: when searching for the root site, the PS above returns all the pages in the tenant with the Web Parts. So it's a "double count". I'll come up with a fix and at that point, do a blog post.

[wobba]

@sympmarc Curious why you fetch all sites first. You can do with one search query and exclude sites by path/id if needed. Adding the filetype filter might make the query perform worse as well, but might be needed?

[sympmarc]

I figured it might be helpful to "slice" the results by site, since different people may need to work on each. But, yes, I could just search for the GUIDs across all objects.

I also was thinking about making this more useful for more common Web Parts and not worrying about paging if there are a lot of results.

Maybe we need a little library of possibilities.

[sympmarc]

Also, I did a blog post about this today: Upgrading the PnP Modern Search Web Parts from v3 to v4: Where are they?

[wobba]

@sympmarc the -All param on the search cmdlet does paging for you already 🙂 I don't have time until Friday most likely to address it, but I would pass filteredSites as a refinement filter. Thus making less API calls with the same result and less resources spent 😉

[sympmarc]

I'll experiment with that. I'm always happy to do things in a more efficient - better - way.

[wobba]

@sympmarc Something like this. The order might not be per site if you include more due to optimized paging internally in search when using -All, but you could add a sort on the $pages response.

Connect-PnPOnline -Url https://sympmarc-admin.sharepoint.com -UseWebLogin

# You can get the Ids for the v3 Web Parts by adding them to a page and running:
# Get-PnPClientSideComponent -Page "page-name" | Where-Object { Title -ne $null } | Select-Object Title, WebPartId

# Title            WebPartId
# -----            ---------
# Search Filters   e899ac12-9256-4c8d-a8ad-dbd20fc459c3
# Search Box       096b96cc-8a44-41fa-9b4d-c0ab2ab2a779
# Search Verticals 9d441773-f735-46a3-9ca0-9e2eb4bef203
# Search Results   42ad2740-3c60-49cf-971a-c44e33511b93

$webPartIds = @(
    "e899ac12-9256-4c8d-a8ad-dbd20fc459c3",
    "096b96cc-8a44-41fa-9b4d-c0ab2ab2a779",
    "9d441773-f735-46a3-9ca0-9e2eb4bef203",
    "42ad2740-3c60-49cf-971a-c44e33511b9"
    )

# Get all the sites
$sites = Get-PnPTenantSite

# You may choose to exclude some subsites of sites
$filteredSites = @('https://sympmarc.sharepoint.com/sites/DemoSite')


# Build the query
$query = "FileExtension:aspx AND ($($webPartIds -join " OR "))"

# Build filter for sites to include
if( $filteredSites.Length -gt 0 ) {
    # Adding dummy item in case we only have one site
    $filteredSites += "x"
    $refinementFilter = 'SPSiteURL:or("' + ($filteredSites -join '","') + '")'
    $pages = Submit-PnPSearchQuery -Query $query -RefinementFilters $refinementFilter -All -RelevantResults -ErrorAction SilentlyContinue  | Select-Object OriginalPath 
} else {
    $pages = Submit-PnPSearchQuery -Query -All -RelevantResults -ErrorAction SilentlyContinue  | Select-Object OriginalPath 
}

# If there are results, display them
if ($pages) {
    foreach ($page in $pages) {
        Write-Host -BackgroundColor Green -ForegroundColor Black "Found Web Parts in $($page.OriginalPath)"                
    }
} 

[sympmarc]

Thanks! I was having a devil of a time finding info about -RefinementFilters.

[wobba]

It's FQL - mentioned way back https://www.techmikael.com/2013/07/limiting-search-results-by-exact-time.html

Docs: https://docs.microsoft.com/en-us/sharepoint/dev/general-development/fast-query-language-fql-syntax-reference

[tomresing]

You should turn this into a cmdlet, @wobba! That would codify the performance tweaks