Sp-dev-docs: SPFx Extension app package periodically fails to load JavaScript files

[sionjlewis]

Category

• [] Question
• [x] Bug
• [] Enhancement

Expected or Desired Behavior

SPFx Extension to load when page is requested.
Steps:

• Upload SPFx Extension to the App Catalog and select Tenant wide deployment
• Run the Add-PnPCustomAction PowerShell CommandLet (for a specific site)
• Load a modern site page ALL OK

Observed Behavior

The SPFx Extension will load OK for weeks and then "out of the blue" if fails with errors like:

GET https://[tenent].sharepoint.com/sites/apps/ClientSideAssets/[guid]/extention-name-application-customizer_[guid].js net: ERR_ABORTED 404
HEAD https://[tenent].sharepoint.com/sites/apps/ClientSideAssets/[guid]/extention-name-sp-page-assembly_[guid].js 404

Steps to Reproduce

Not sure how we can reproduce this problem?

• The problem:
• 
  
  • Occur generally on or just after a Tuesday (New Zealand time)
  
  
• 
  
  • Occurs for multiple tenants, but generally not all at the same time
  
  
• 
  
  • Affect all SPFx Extensions (when it happens)
  
  
• 
  
  • Does not affect SPFx Webparts
  
  
• The JavaScript files appears to be from an older versions of the App Package (we have started a log to confirm this assumption)
• When an error occurs, the SPFx Extension does not load
• Opening the Browser Console locate the error:
• 
  
  • Copy and past the JavaScript's URL into a browser window will also show a 404 error
  
  
• 
  
  • Remove the File name & extension from the URL will open the container folder (no errors)
  
  
• This may not be an SPFx Extension bug but rather an Office 365 update issue?

Possible solution:

I assume the GUIDs used within the file names as for cache-busting purposes. Why not remove the GUID from file name and apply it to the requesting code as a query string parameter?

Ideas and suggestions are welcome :)

sp-dev-fx-extensions

[msft-github-bot]

Thank you for reporting this issue. We will be triaging your incoming issue as soon as possible.

[nateforsyth]

Can confirm have seen this multiple times.

[patmill]

If you browse the directory when you get a 404, can you find the file? If you can, can you select it from the library? If so, does it load, and if so, does the URL match the original request URL?

Also - can you look at the response headers and response body of the original 404? Does anything look out of whack?

[sionjlewis]

If you browse the directory when you get a 404, can you find the file?
Yes there are files, however their names end with different GUIDs
~/ClientSideAssets/[EXT-GUID]/extension-name-application-customizer_[FILE-GUID].js

If you can, can you select it from the library?
Yes, the files within the library load without errors (OK)

If so, does it load, and if so, does the URL match the original request URL?
A. Yes the file Loads.
B. No, the files has a different GUID appended to the file name
~/ClientSideAssets/[EXT-GUID]/extension-name-application-customizer_[FILE-GUID].js

Also - can you look at the response headers and response body of the original 404?
I will have to wait for the next incident and get back to you.

Does anything look out of whack?
TBC: assuming not as the request is for a different file name.

[patmill]

That's super weird. Are you updating your solution in any way? Are you uploading code separate from the solution?

The path should be in the solution manifest, so it's really strange that it should not match. Let's try this:
1 - Go to the component manifests list in your app catalog. It can be reached at yourtenant.sharepoint.com/sites/yourappcatalog/Lists/ComponentManifests/AllItems.aspx
2 - Find the manifest of the component that is misbehaving
3 - Look at the references in that file. Do they match with the file names that are in the code assets library?
4 - if not, can you see what the last modified time is on the original solution, the entry in the manifest list, and the entry in the assets library?

Those should all be in sync for the most part, and (unless there is some strange caching at play) the only reason the code filename should change is because the solution was updated, it had a manifest change, and a new bit of code.

[nateforsyth]

Following on from what @sionjlewis has said above, it's worth noting a few other things; every time this occurs:

• there has been an update to our extensions project 1-2 days before.
• the updates are done within the SPFx project.
• the project is built and bundled using the documented gulp based processes.
• the package-solution.json version is incremented.

These next points may be a red herring, however the timing seems consistent.

• this always seems to occur between the Tuesday - Wednesday window.
• this feels like the server is performing a post-update rollback to a predefined state.
• the filename it's attempting to load appears to be old (unconfirmed).
• the filename being loaded is different to what is within the ClientSideAssets folder for the extension (as previously mentioned).

Hope this helps.

[sionjlewis]

Are you updating your solution in any way?
From our recollection, this only seems to happen on environments where the package has been updated.

Are you uploading code separate from the solution?
No, all the code is deployed using an App Package

The path should be in the solution manifest, so it's really strange that it should not match.
Agree; it's like the "system" has be partially rolled back (back-up & restore)...

Let's try this:

1 - Go to the component manifests list in your app catalog. It can be reached at yourtenant.sharepoint.com/sites/yourappcatalog/Lists/ComponentManifests/AllItems.aspx
2 - Find the manifest of the component that is misbehaving
3 - Look at the references in that file. Do they match with the file names that are in the code assets library?
4 - if not, can you see what the last modified time is on the original solution, the entry in the manifest list, and the entry in the assets library?

Thanks for the URL. Unfortunately I'm unable to reproduce the error right now, as we had to apply the a fix ASAP. If/when the error occurs in our Test environment then we will be able to follow your steps above and get back to you.

Currently our fix is to replace the existing package (i.e. re-upload it). It does not require a rebuild (or changing package version).

Those should all be in sync for the most part, and (unless there is some strange caching at play) the only reason the code filename should change is because the solution was updated, it had a manifest change, and a new bit of code.

Make sense and logically agree :)

I don't want to muddy the water; however, I have wondered if this issue is related to the SharePoint Online Update process?

Remembering back to SharePoint 2007 & 2010 On-Premises days, we use to back up a site collection's database and file system separately. If these back-ups were slightly out of sync then we would see similar issues on a restore...

In the meantime here's a screen shot:

[patmill]

I don't think SPO updates are the cause here. The GUID in that file name is generated during the build process of the solution, and a reference is added to the manifest during gulp bundle and gulp package-solution --ship . Any change to the path would come from an upload to the library, either because of an update to the solution (which cracks the solution file and copies the files) or because someone updated the files directly. My only guess is that there is a cache issue, where the solution is updated, the manifest is changed, the new files are uploaded, but the manifest is cached somewhere and an old reference is getting used.

[sionjlewis]

The GUID in that file name is generated during the build process of the solution, and a reference is added to the manifest during gulp bundle and gulp package-solution --ship .

Yes, 100% agree that is what the SPFx build process does.

Any change to the path would come from an upload to the library, either because of an update to the solution (which cracks the solution file and copies the files) or because someone updated the files directly.

Yes, also agree.
NOTE: we are not uploading the files directly :)

My only guess is that there is a cache issue, where the solution is updated, the manifest is changed, the new files are uploaded, but the manifest is cached somewhere and an old reference is getting used.

A "Cache issue" would seem a plausible explanation. However the error does not occur immediately after and update. If it did then replicating the error would be much easier...

• We have previously tried deleting the App Package and JavaScript folder (located in ClientSideAssets) before re-uploading the Package to the App Catalog. This has fixed the immediate problem, but in some cases the issue has reoccurred some time later (days or weeks).

• What we haven't checked is that the "~/Lists/ComponentManifests" items have been deletes too.

If it's a "Cache issue", any ideas on how can we fix it?

[semopz]

I've had issues with extensions before the latest SPO updates and the following steps helped (may not be the case with OP's issue).

1. As mentioned, always increment the version number of your solution before deployment.
2. Don't let SharePoint update the extension for you. Use Uninstall-PnPApp to remove the old version and then use Install-PnPApp to install the new version. I found that doing these steps through the GUI would cause issues as sometimes, the uninstall stage would still be in progress even though the GUI would indicate otherwise. The PnP Install would fail if other operations are still in progress (might be worth just waiting 30 mins after an uninstall just to be safe).
3. Whenever you install an extension "App" you also get a CustomAction added to your SPWeb. Keep track of those and make sure that when you update an extension, 1 CustomAction is deleted after uninstall and that an install adds 1 CustomAction. You can keep an eye on those with Get-PnPCustomAction

[sionjlewis]

Hi @semopz,
Great to hear your experience with this issue.

1. As mentioned, always increment the version number of your solution before deployment.

Yes / thumbs-up

1. Don't let SharePoint update the extension for you. Use Uninstall-PnPApp to remove the old version and then use Install-PnPApp to install the new version. I found that doing these steps through the GUI would cause issues as sometimes, the uninstall stage would still be in progress even though the GUI would indicate otherwise. The PnP Install would fail if other operations are still in progress (might be worth just waiting 30 mins after an uninstall just to be safe).

Interesting; we have previously had issues with the Install-PnPApp cmdlet timing out, so started using a manual process (the UI) instead. Will try PowerShell next time, to see if that resolves the issue.

I have a concern regarding "... waiting 30 mins after an uninstall ...", as this often not possible on 'live' systems. So some more thought will be required...

1. Whenever you install an extension "App" you also get a CustomAction added to your SPWeb. Keep track of those and make sure that when you update an extension, 1 CustomAction is deleted after uninstall and that an install adds 1 CustomAction. You can keep an eye on those with Get-PnPCustomAction

Excellent point, we will include that in our monitoring of the issue/bug

Thank you

Updated to checklist:

• https://[tenent].sharepoint.com/sites/[app-catalog]/Lists/ComponentManifests/
• https://[tenent].sharepoint.com/sites/[app-catalog]/ClientSideAssets/[SPFx-Ext-GUID]/
• Get-PnPCustomAction

[andrewconnell]

@sionjlewis - Seems based on your last reply this issue can be closed as resolved... correct?

[sionjlewis]

Hi @andrewconnell,
I think you may have misunderstood my last comment, the issue/bug is not resolved.

I suspect the issue/bug lies with the underlying SharePoint infrastructure and not necessarily with the SPFx framework. @semopz may back this assumption up in his/her post, point 2 and 3. However we are still trying to gather evidence so that others may replicate.

At the end of my last post I created a checklist, of things we will log next time a SPFx Extension fails to load.

Notes:

• We have not had any issues with SPFx Webparts loading
• SPFx Extensions loads OK when deployed
• SPFx Extensions have a pattern of failing on Tuesdays intermittently

[nateforsyth]

Hi all and @sionjlewis

I apologise for the length of this reply in advance. I was able to capture this issue occurring on 3 tenants yesterday. I've managed to get screenshots as well as an extract of the Client Side Component Manifest list items when it occurred on all of them, though I'll only show you the results from one of them.

A successful update

It started last Friday, had to update our extension package on a client environment.

1a Extension state - Apps - the update from last week

Here's the Apps for SharePoint interface at the point of updating the extension package on 17th May 2019, note the version number.

1b Extension state - Client Side Assets - the update from last week

Here's the Client Side Assets folder on the same day, 17th May 2019.

1c Extension state - Client Side Component Manifests - the update from last week

And here's the Client Side Component Manifests List on 17th May 2019.

At this point everything was going well. Although the screenshot doesn't show it, the 5x entries for each Extension were pointing to the same .js file within the ClientSideAssets folder.

Github, we have a problem

Yesterday (Monday) it was noticed that the Manifests didn't look the way they did on Friday.

2a Extension state - Client Side Assets - the failure

Here's the Client Side Assets folder, 20th May 2019 (note the date).

2b Extension state - Client Side Component Manifests - the failure

Here's the Client Side Component Manifests list beside the console logging when the failure occurred.

Note where:

• the first manifest .js file has a different random string applied, this references the correct version of the file (confirmed).
• the 2nd through 5th entries now reference the previous .js files

This is what was replaced on 17th May 2019

Note the date and random string matches the 4 items that have reverted in the previous screenshot.

The fix

Fixing this appears to be permanent, until the next time we update our extension project/s.

I remove the client side assets for the package, as well as the app itself, before clearing out the recycle bin.

3a Extension state - Apps for SharePoint - new package

Here's where I have uploaded a freshly packaged .sppkg file to the Apps for SharePoint interface.

3b Extension state - Client Side Assets - new package

This is the state of the Client Side Assets folder after uploading the new package.

5b Extension state - Client Side Component Manifests - new package

And this is the way I expect to see the Client Side Component Manifests List when a new package is uploaded.

This is the state it's currently in.

6 Extension state - Client Side Component Manifests - an extract

And finally, here's an extract of the manifests from the list that has all of the differing .js filenames.

extension state - Client Side Component Manifests - 20190520.txt

Notes and caveats

• this occurs when a new version of an Extension package is uploaded.
  
  
  
  • when this happens, it's always a Monday or Tuesday.
  
  
  • this may still be circumstantial, but 3 tenants did this between 3pm and 5pm yesterday (local time)).
  
  
  • only one of the 3 tenants were updated last week; the other two were updated around 1pm yesterday.
  
  
• repairing this necessitates the following steps:
  
  
  
  • fresh build of the .sppkg file.
  
  
  • removal of the applicable Client Side Assets folder.
  
  
  • deletion of the existing .sppkg file from the Apps for SharePoint interface.
  
  
  • clear out the first stage recycle bin on the App Catalog site.
  
  
  • clear out the second stage recycle bin on the App Catalog site.
  
  
  • upload the new .sppkg, set to tenant wide deployment (automatically publishes).
  
  

The repair steps are a work-around at best due to this behaviour being seemingly random.

Thanks for reading, looking forward to some sort of action or guidance on this issue.

[nateforsyth]

In follow up. I left another tenant with an update yesterday to see whether it'd fail today. It has. I've just replicated the issue again. Note here that 4/5 of the entries are referencing the old .js file again (all 5 were the same yesterday).

[sionjlewis]

Thanks @nateforsyth,
We had been waiting sometime to be able to reproduce the problem, let's hope this enough to find a solution...?

[cwdata]

I can add that this has nothing to do with the storage location of the Javascript files.
We are also sometings seeing old files being loaded but we upload the assets to our own CDN. As we so far never deleted anything from our CDN we don't get 404s, but instead we get the old versions of the files. Sometimes they are several days old. It's not a browser cache issue either because this also happens after clearing the browser cache.

[sionjlewis]

Thanks @cwdata,
Even though you don't get 404 errors, responding with the wrong files is still an error...

[nateforsyth]

Thanks @cwdata that's definitely helpful.

Have you looked at your Client Side Component Manifests List when this occurs? Would be interested to see whether or not you're seeing the discrepancies within the List items for the manifest like we were.

https://[tenaneName].sharepoint.com/sites/[appCatalogName]/Lists/ComponentManifests/

Using the App Catalogue, updates to our packages overwrite the Client Side Assets folder, meaning that the old .js files are removed. Upon this failure they obviously cannot be resolved anymore.

Yours sounds like the exact same issue to me, where you have the old versions it makes perfect sense that you're seeing old versions of your components render unexpectedly.

I'd bet money on a 404 like we get if you were to temporarily remove the old versions.

This has been tried on another tenant that failed today and doesn't work:

~We've also realised that the Client Side Component Manifests List items are editable. We're going to try editing the manifests directly to resolve the ambiguity next time it happens - a hot fix if you will (we're not sure if it'll resolve the issue). We consider this a hacky work-around though and definitely something that should be fixed.~

Thanks again for confirming :)

[nateforsyth]

Ignore the above reference, I posted a reply to an incorrect thread.

[nateforsyth]

Hi all, I've just had this happen again, and on a Tuesday (I'm going back to this being related to updates).

[MikeMorawski]

Similar issues on our end, had it happen yesterday and today (Mon/Tues) as well. It might have been due to updating the app without incrementing the version as our CI/CD process never appears to have this issue (it auto updates the version), only manual deployments seemed to be affected.

[nateforsyth]

Cheers for the input @migee

This is quite an obscure issue. Did you by chance have a look at the your Client Side Component Manifests List?

https://[tenantName].sharepoint.com/sites/[appCatalogName]/Lists/ComponentManifests/

I've been religiously updating all versions every time I update a package since I started to encounter this issue and it still occurs, in these places:

• ./config/package-solution.json
  
  
  
  • _solution_
  
  
  • _features_
  
  
• ./src/extensions/[extensionName]/[extensionName]ApplicationCustomizer.manifest.json

We don't think this is occurring within the SharePoint Framework itself, but it is related.

We suspect it's server side state that isn't being correctly re-applied after updates to SharePoint Online/Office 365.

[sionjlewis]

Hi @VesaJuvonen,
Do you or your Team have any thought as to what is going on?

As my colleague @nateforsyth said

We suspect it's server side state that isn't being correctly re-applied after updates to SharePoint Online/Office 365

However, without full access to the Office 365 environment and update processes, we have no way of confirming this either way...

[nateforsyth]

I had this occur yesterday on a different clients' tenant (again, on a Tuesday).

This is becoming quite a liability for us, some sort of indication as to whether this is being looked at would be very useful at this point.

[patmill]

So I suspect that what is happening here is that the client assets are getting updated, but the manifest pointing to them is caching the previous version.

We have a work item in progress to keep the client assets for version n-1.

Two things to try in the meantime.
1 - If the bundle name doesn't change, does the problem go away?
2 - If you enable public CDN for the tenant, does the problem go away?

[sionjlewis]

Thanks @patmill,
Please keep us in the loop as to when your 'work item' is being rolled out so that we can keep an eye things.

[nateforsyth]

So I suspect that what is happening here is that the client assets are getting updated, but the manifest pointing to them is caching the previous version.

We have a work item in progress to keep the client assets for version n-1.

Hi @patmill thank you for your reply, it's good to know it's gaining some traction.

Regarding your things to try.

1 - If the bundle name doesn't change, does the problem go away?

Our bundle name has never changed, what does change is the generated suffix appended to the javascript filenames, which we have no control over.

2 - If you enable public CDN for the tenant, does the problem go away?

I don't think that's an option for our clients' production environments, though we can certainly try on a test tenant. In saying that, we don't really understand what's happening on the back end or what causes this particular cache issue to manifest, so it may be a while before we know if this has helped or not.

In saying that, @cwdata has observed similar behaviour using their own CDN (assuming configured as public), which pushes me to thinking it won't help.

I can add that this has nothing to do with the storage location of the Javascript files.
We are also sometings seeing old files being loaded but we upload the assets to our own CDN. As we so far never deleted anything from our CDN we don't get 404s, but instead we get the old versions of the files. Sometimes they are several days old. It's not a browser cache issue either because this also happens after clearing the browser cache.

As mentioned though, we can look into doing so.

[HandyAndyOnGitHub]

Hi @patmill , I represent one of the customers that this issue is affecting. We were impacted again by the issue this morning ~09:30am 25 July 2019 New Zealand timezone (UTC+12). Any idea when we're likely to see this resolved?

[patmill]

Hi Andy and Nate. So let's step back a second and see if we can work around your specific problem while waiting for a general solution. I'll be honest, I think there is some specific CI/CD process at work here for your tenants. I'm not aware of anything in the product itself that would change the manifest w/o some external force, in particular on a steady Tuesday cadence. Also, the solution we are planning (n-1 support) won't help if something else is at play, and the solution isn't being updated.

Are you aware of anything that would push changes to your tenant on a regular basis?

Can you try going to the app catalog, finding the solution that contains the component in question that is causing issues, and view the file history of it? Are there changes that correspond to the times you hit issues?

Also, can you write down the js file (full path, including hash) today, and then compare it to when the error occurs?

[sionjlewis]

Hi @patmill,
We have been "around the houses" with this and agree with your previous message:

So I suspect that what is happening here is that the client assets are getting updated, but the manifest pointing to them is caching the previous version.

This is pretty much inline with what I originally suggested when I opened this bug.

Are you aware of anything that would push changes to your tenant on a regular basis?

Answer is No

Can you try going to the app catalog, finding the solution that contains the component in question that is causing issues, and view the file history of it? Are there changes that correspond to the times you hit issues?

The packages are deleted before uploading the new version.

Also, can you write down the js file (full path, including hash) today, and then compare it to when the error occurs?

@nateforsyth has spent a lot of time documenting and sharing what we when the bug occurs (please see his message 21st of May).

[patmill]

What do you hope to see in any logs? Also - are you able to check the version history of the package in the app catalog?

[sionjlewis]

If this was 2013 or earlier, I would start by looking at the ULS Logs to see what process were running in and around the time the bug occurs.

[patmill]

Is there any chance that there is something in common with the solutions? Are they all being implemented by the same ISV or developer? It's really odd that many of the tenants seem to be based in New Zealand.

Can you possibly send me some information, like the tenant domain, and the ID of the site collection and web that the app catalog is in? We don't have any information like URLs, etc. I can see if anything appears. patmill at microsoft.com

[nateforsyth]

Is there any chance that there is something in common with the solutions? Are they all being implemented by the same ISV or developer? It's really odd that many of the tenants seem to be based in New Zealand.

Can you possibly send me some information, like the tenant domain, and the ID of the site collection and web that the app catalog is in? We don't have any information like URLs, etc. I can see if anything appears. patmill at microsoft.com

Hi @patmill

This isn't just occurring in New Zealand, please don't fixate on that. It's occurring to @cwdata in Germany and @migee in Canada too.

cwdata said
I can add that this has nothing to do with the storage location of the Javascript files.
We are also sometings seeing old files being loaded but we upload the assets to our own CDN. As we so far never deleted anything from our CDN we don't get 404s, but instead we get the old versions of the files. Sometimes they are several days old. It's not a browser cache issue either because this also happens after clearing the browser cache.

migee said
Similar issues on our end, had it happen yesterday and today (Mon/Tues) as well. It might have been due to updating the app without incrementing the version as our CI/CD process never appears to have this issue (it auto updates the version), only manual deployments seemed to be affected.

I'll answer your questions and reiterate a few things:

Are you aware of anything that would push changes to your tenant on a regular basis?

This occurs regardless of whether we use our CI processes or not (we refuse to use CD while this issue persists).

For absolute and abundant clarity, this occurs regardless of being manually built or automated.

The issues @HandyAndyOnGitHub alludes to occurred both yesterday and today on different components that were manually built. I caught yesterdays ones by monitoring this URL (which I shouldn't have to do):

https://[tenantName].sharepoint.com/sites/[appCatalogName]/Lists/ComponentManifests/

I'd noticed that the entries were different to the previous days (this is an example, not the actual entries).

We've built this manual checking process into our procedures and do this regularly on every clients' environemtnwe work on.

My reply 21st May was after an Azure DevOps CI build.

Can you try going to the app catalog, finding the solution that contains the component in question that is causing issues, and view the file history of it? Are there changes that correspond to the times you hit issues?

I can do better than that, I have documented where the Client Side Component Manifests List items change inexplicably.

Also, can you write down the js file (full path, including hash) today, and then compare it to when the error occurs?

As above, and please... Refer to my replies 21st and 22nd May above.

In saying all of that, I must caution against blaming a red herring such as build process because these can and do work perfectly for days until SharePoint has been updated, at which point I can see where the failure will occur.

There is absolutely zero doubt that this is server-side.

We just want to know what's going on.

I'll flick you an email with the URLs for the impacted tenants.

[patmill]

So here's the thing. The component manifest list you point to is a list. It's nothing special. It's not a cached view, or anything like that. It gets populated / changed either when someone specifically changes the item, or when the solution that an item is part of is deployed. That data is stored in the database, so updating the code on the server doesn't affect it. If list content randomly changed to older versions on a regular basis across the service, that would be a problem. I'm trying to narrow down what could be causing the change to the component manifest list, what similarities across people experiencing this might be, etc. It's possible that something, somewhere, somehow is deploying an old version, but I've no idea how that would happen.

And again - are you able to see the change history of the solutions in the app catalog (not the component manifest list, but in the app catalog) that contain the components that are causing issues?

[patmill]

One more thing for people that experience this. Can you go to the item in question in the /lists/componentmanifests, select it, hit the 'Items' tab, select View Item, and look at the created/modified data at the very bottom? It should look something like:

Created at 7/25/2019 9:32 AM  by Some User
Last modified at 7/25/2019 9:34 AM  by Some Other User

[patmill]

Whoa, one more thing we just noticed. You have multiple entries in the component manifests list that all share the same component ID. That shouldn't be possible. Does everyone on this thread have that problem? You should only ever have a single entry for a given component ID. For sure this could cause problems, where one entry is updated, but a different entry is returned.

Can you give us any insight into what (if anything) you might have done? This was a bug from a long time ago that (we thought) had been addressed.

Possible ways to get out of this state.
1 - Delete all entries for a given component but 1, and reupload / deploy your package
2 - Delete the package that contains these components, check to make sure that the components are removed, then reupload/redeploy your package.

[sionjlewis]

...You have multiple entries in the component manifests list that all share the same component ID. That shouldn't be possible. ... You should only ever have a single entry for a given component ID. For sure this could cause problems, where one entry is updated, but a different entry is returned.

... This was a bug from a long time ago that (we thought) had been addressed.

@patmill, Please can post a link to or more details about the bug you mentioned above? This info maybe help, particularly if it related to a specific framework or library version.

[patmill]

It's not related to a specific version of the framework - it was a fix the service itself. You shouldn't be able to get into this situation with a new solution, but the fix from last year apparently didn't clean up the situation if you were already in it. The general N-1 work item is up next, and cleaning up the "we have more than 1 entry for a single component" will get cleaned up at the same time.

What happens today is that the first version that we encounter in the list, we update with the changes from the latest package. However the rest of them remain the same. So now we are in a state where there is one client assets folder for the component with code from version X, but there are 8 other entries for the same component in the component manifest list, pointing to various instances from the past. When we load up the component from the list, we may get the latest one that was just updated, or we might get one of the others. That's not to say it's specifically random, just that there are multiple entries in the list, and no guarantee that we will get the one we want.
Uninstalling the package should remove all the entries, reinstalling it will create a single one, and everything should be good (other than the other problem where you upload an update that has different code, and for 5-10 minutes the old manifest can be cached, which points to code that is no longer there. That's the N-1 issue that is getting fixed by keeping the assets from the previous install around.

I'll try and find some closed github bugs that mention this. The actual bug we fixed came in through an escalation though, rather than a github request, so I don't know if there is a straight forward "here's the bug" example I can point to.

[cwdata]

Hi @patmill,
I looked up the componentmanifests list. We have for several packages more than one entry with the same component id. I thought that would be normal, because the version number is different.
One entry is like this:

Solution version: 1.0.0.370, version in the manifest part: 0.0.346
Created: 05/28/2019 10:38am
Modified: 05/28/2019 10:38am

The other one with the same component id:

Solution version: 1.0.0.394, version in the manifest part: 0.0.370
Created: 05/28/2019 10:38am
Modified: 07/02/2019 10:00pm

The user for created/modified is always the same (the one who uploaded the package)
Interesting is: Both entries have the exact same creation date. And as the date shows, we haven't uploaded this last year. So if this behavior is a bug it still exists.

Sometimes the newer one gets loaded on the page, sometimes the older one. Hitting refresh usually loads the newer version of the webpart. We can easily see that here because the design of the webpart changed between the two versions.

[cwdata]

Update:
I also found solutions in the componentmanifests list with the same component id and the same version. The solution package contains 3 features in the package-solution.json:

"features": [
      {
        "title": "Syscovery Business Apps - Extension for calendar management",
        "description": "A list extension to manage the company calendar list",
        "id": "63712fbe-eeaa-4133-a581-182b54e52b4d",
        "version": "1.0.0.11",
        "assets": {
          "elementManifests": [
            "elements.xml"
          ]
        }
      },
      {
        "title": "Syscovery Business Apps - Webpart for open requests",
        "description": "A webpart to show open requests of the current user",
        "id": "c8ba56cf-229b-4041-9ca5-0672a97222cd",
        "version": "1.0.0.1"
      },
      {
        "title": "Syscovery Business Apps - Webpart for open approvals",
        "description": "A webpart to show requests to be approved by the current user",
        "id": "fb8a18cd-3a87-456e-9728-b3bf69435cdc",
        "version": "1.0.0.1"
      }
    ],

Uploading this package to a new site with a new site collection app catalog results in 3 entries for each component in that solution in the componentmanifests list all with the same component id version number and creation date. The feature names seem to be randomly chosen from the 3 features:

After updating the solution only one of those 3 entries gets the newer modified date and the newer version.
Maybe there's something wrong in how we define those features if there are more than one inside the package?

[sionjlewis]

Hi @patmill,
Are you able to give a progress update?

It's not related to a specific version of the framework - it was a fix the service itself. You shouldn't be able to get into this situation with a new solution, but the fix from last year apparently didn't clean up the situation if you were already in it. The general N-1 work item is up next, and cleaning up the "we have more than 1 entry for a single component" will get cleaned up at the same time.

What happens today is that the first version that we encounter in the list, we update with the changes from the latest package. However the rest of them remain the same. So now we are in a state where there is one client assets folder for the component with code from version X, but there are 8 other entries for the same component in the component manifest list, pointing to various instances from the past. When we load up the component from the list, we may get the latest one that was just updated, or we might get one of the others. That's not to say it's specifically random, just that there are multiple entries in the list, and no guarantee that we will get the one we want.
Uninstalling the package should remove all the entries, reinstalling it will create a single one, and everything should be good (other than the other problem where you upload an update that has different code, and for 5-10 minutes the old manifest can be cached, which points to code that is no longer there. That's the N-1 issue that is getting fixed by keeping the assets from the previous install around.

I'll try and find some closed github bugs that mention this. The actual bug we fixed came in through an escalation though, rather than a github request, so I don't know if there is a straight forward "here's the bug" example I can point to.

[cwdata]

I can confirm that this is still not working.
As this issue now persists for more than 6 months and no more progress updates are given it seems that Microsoft is not able to resolve this bug. So I suggest to change the documentation and tell people that a full uninstall/reinstall cycle is necessary when a SPFx solution needs to be upgraded.

[nateforsyth]

I can confirm that this is still not working.
As this issue now persists for more than 6 months and no more progress updates are given it seems that Microsoft is not able to resolve this bug. So I suggest to change the documentation and tell people that a full uninstall/reinstall cycle is necessary when a SPFx solution needs to be upgraded.

Hi @cwdata can you confirm please that you can still fully uninstall and reinstall the SPFx package?

We've had a client environment fall over today, however the normal removal from Client Side Assets and Apps for SharePoint isn't working any longer.

Client Side Component Manifests doesn't update any longer, meaning that there is a new error visible within Apps for SharePoint

Component ID {0} exists more than once in current package

@patmill it appears that the workaround we've been using up until this point may no longer be working as expected - can you please weigh in on this?

Post new work-around update:

I've managed to get around the Component ID {0} exists more than once in current package error on Apps for SharePoint by deleting all entries for the anomalous package from Client Side Component Manifests in addition to the normal work-around.

To be clear, we've never had to do this over the last ~1 year that we've been encountering this issue.

@sionjlewis fyi

[cwdata]

@nateforsyth,
yes we got the same error and did the same, that is removed all entries for that component from the component manifests list. But we only needed to do that one time (long ago) for solutions that were already there before the update of the system. These solutions really had a lot of entries in the manifests list.
Since then we only get a couple of entries when reinstalling. As I`ve said, if your solution consists of for example one Webpart and two Extensions, right after the installation you will end up with a multiple of 3 entries in the component manifest list all sharing the same component id . But so far uninstalling the solution from the catalog removes all of them.
Long story short - we don't run into problems with uninstalling/reinstalling. The old bug that the system tries to load an old version when you simply upgrade your solution still exists though (we try that from time to time...)

[ZabbyCapurin]

Happy to see this issue isn't just me but sad this hasn't been resolved after a year.

For us we've previously had no issues uploading the latest package version. Today we're running into this issue where it's pointing to old versions of our JS files. We're in the US on GCC if that helps.

[nateforsyth]

Happy to see this issue isn't just me but sad this hasn't been resolved after a year.

For us we've previously had no issues uploading the latest package version. Today we're running into this issue where it's pointing to old versions of our JS files. We're in the US on GCC if that helps.

@ZabbyCapurin did you manage to work around the issue? If not, clear out all relevant entries within Client Side Component Manifests and then Client Side Assets, then remove the component package from Client Side Apps. Then clear out the Recycle Bin on the App Catalog site, and then reinstall your component.

As you mention, this issue is still outstanding and quite frankly is a real PITA.

While the end result is the same (SPO attempts to load an old non-existent version of an SPFx Extension), getting there appears to be somewhat different. At the point of uploading a new version, it throws an error that a component already exists with the same ID. There isn't, but it's the Client Side Component Manifests that didn't get updated (hence why I say that it's the same end result). The fix/workaround was to do what I have suggested @ZabbyCapurin to do.

My entire day yesterday was spent working around this problem on multiple clients simply because we had to update a single Extension (again, on a Tuesday).

@patmill how are we doing on this one mate? it's been 14 months, we'd appreciate an update :)

[ZabbyCapurin]

@nateforsyth Yup, pretty much the steps we had to take... which isn't ideal for production...

Had a call with Microsoft support and they just kept saying they couldn't reproduce with the code I gave them (expected this but still). I'm planning to contact them the next time it happens.

[nateforsyth]

@patmill can we get an update on this one please? 17 months now

[nateforsyth]

@patmill can we get an update on this one please? Coming up 19 months now.

@ZabbyCapurin are you still seeing this?

@sionjlewis still open.

[ZabbyCapurin]

@nateforsyth haven't run into this again but I can't say that I think it's fixed and won't happen in the future