Modernization: URL Rewrite enhancements

[jansenbe]

Some ideas for future url rewrite improvements:

• Build up URL mapping table by:
  
  
  
  • Read URL's from managed navigation term set to allow the rewrite of URL's constructed using '_layouts/15/FIXUPREDIRECT.ASPX'
  
  
  • Read URL's from a customer provided url mapping file
  
  
  • Adding known mappings (pages -> site pages, documents -> shared documents,...)
  
  
• Verify content to be rewritten against this list of url mappings

[pkbullock]

Some thoughts as well:

• Link validation e.g. check the target exists, possibly just report on its validity
• External links filters to ensure they are not processed
• Exclude assets that would be processed by asset transfer
• Thinking about a selective transform where we re-run a transform on a file for part previous transform, giving the ability for an SME to quickly re-process that aspect of the transform quickly, especially if there are changes to the mapping file and all you want is to refresh the links. Although this would lean towards a post processing feature after a large batch.
• Prescan or listing of a source page URLs for the mapping

[jansenbe]

Thanks for the feedback @pkbullock!

Some comments:

• External links are currently filtered out and that should indeed stay as is
• I've moved url rewrite to a latter phase as I indeed got into trouble with asset transfer...currently links are renamed after asset transfer was done as we need to the original url's as source for an asset
• Selective transform will be a nightmare to implement...would rather favor the "overwrite" flag and do a full transform in case anything changed
• Scanning source files to generate a url mapping: nice idea :-)

[jansenbe]

Move to July release, to be handled together with user mapping (#161).

[jansenbe]

Maybe align with https://docs.microsoft.com/en-us/sharepointmigration/create-a-user-mapping-file-for-data-content-migration as a model

[pkbullock]

Could we use their code? Shame that tool isn't open source, would be easier to include/extend.

[jansenbe]

@pkbullock / @MartinHatch / others: I've been running over the issue list and cleaning up some of the issues. When it comes to URL mapping improvements I would like to understand what your key requirements are here...the current mapping will work as described below and I'm wondering what we really need vs what woudl be nice to have:

• Works for text and summarylinks transformations
• Assuming the source site is the following http://contoso.com/sites/portal/hr and the target communication site lives in https://contoso.sharepoint.com/sites/hr then the following URL rewrites are applied:
  
  
  
  • http://contoso.com/sites/portal -> https://contoso.sharepoint.com/sites/hr
  
  
  • http://contoso.com/sites/portal/pages -> https://contoso.sharepoint.com/sites/hr/sitepages
  
  
  • /sites/portal -> /sites/hr
  
  
  • /sites/portal/pages -> /sites/hr/sitepages
  
  
  • http://contoso.com/sites/portal/hr -> https://contoso.sharepoint.com/sites/hr
  
  
  • http://contoso.com/sites/portal/hr/pages -> https://contoso.sharepoint.com/sites/hr/sitepages
  
  
  • /sites/portal/hr -> /sites/hr
  
  
  • /sites/portal/hr/pages -> /sites/hr/sitepages
  
  

[pkbullock]

@jansenbe so for me the ability replace an existing URL with a new one via a mapping e.g. where client has used summary links web parts to refer to other pages inside the intranet.

Whilst it wouldn't necessary for the tool to detect the new location of the page, if I had a (another) mapping file, or even part of the web part mapping file, a list of Old Locations and New Locations (added by me/SME), that if matched would replace during transform.

The pages I wish to transform is highly selective, so I will know ahead of time what the old/new locations are for pages (some will be new e.g. not transformed).

[jansenbe]

Simple URL mapping configuration committed: provide a CSV file without header listing source and target values and these will be handled. The custom URL mapping happens before the default, so if the default mapping is not desired one could add custom mapping for those as well. Closing this issue, let's open new ones to further define improvements/fixes that are still needed.

[pkbullock]

Awesome stuff!

[jansenbe]

@pkbullock : PowerShell usage is like this (also added verbose logging so you can see in logs which url's are rewritten)

ConvertTo-PnPClientSidePage -PublishingPage -Identity url_2 -Overwrite -TargetWebUrl https://bertonline.sharepoint.com/sites/ModernizationTarget -LogType File -LogVerbose -LogFolder "c:\temp" -UrlMappingFile "C:\github\sp-dev-modernization\Tools\SharePoint.Modernization\SharePointPnP.Modernization.Framework\Samples\urlmapping_sample.csv"