Waterfox Current (and Firefox Quantum) cause loss of some types of legacy extension when used to open Waterfox Classic profiles

[kswartz26]

I copied an existing profile and started up the 68a1 release. One of the extensions already installed was Stylus. It's not functional (no surprise), but if I open the addons manager, it does not appear there. However, if I go to AMO and open up the page for the addon, it thinks it is there and only offer me a button to Remove it. So something in the browser thinks it's there, just not the addons page.

[laniakea64]

Works for me, and Stylus is functional.

Did you download the 68.0a1 or build it yourself from latest gecko68 branch? If you downloaded it, how did you use the existing profile?

Before trying v68, what Waterfox version did you last use with this profile?

What version of Stylus do you have?

[kswartz26]

I downloaded the binary, then ran it on a copy of a profile from 56.2.9.1.

But I actually think the bug here is not what I thought. I saw a similar problem when doing an in-place upgrade from 56.2.9.1 to 56.2.10. After that upgrade, one of my addons (Prefbar) just "vanished" from the addons manager. I reinstalled it, and it worked again.

Sure enough, if I re-installed Stylus from a local file, it started working fine.

But there's still a weird bug in that the addon manager did NOT think it was there, but the addon page itself DID.

[grahamperrin]

Stylus

Legacy? Which version?

[kswartz26]

Stylus 1.5.3, which is a webextension. The addon that "disappeared" when going from 56.2.9 to 56.2.10 was PrefBar 7.1.1, which is a legacy addon.

I can nuke the 68a profile, make a new copy, and try to reproduce it.

[grahamperrin]

a webextension.

Yeah, strange that this is in the Classic Add-ons Archive. Ignore that, I had my extension names mixed up in a test profile. Stylus, Stylish. Sorry.

[laniakea64]

The problem is likely "copying" the profile instead of using a profile in-place. This sounds like the same sort of issue I ran into in https://github.com/MrAlex94/Waterfox/issues/947 before the profile location was fixed.

If you want to use a copy of an existing Waterfox 56 profile with Waterfox 68, try this -

1) completely quit all Waterfoxes

2) delete everything in the broken v68 profile, copy in the entire contents of the v56 profile again

3) Before starting Waterfox 68, update all the absolute paths in addonStartup.json.lz4 (use dejsonlz4 and jsonlz4 to edit it) and extensions.json in the copied profile

4) start Waterfox 68 with -purgecaches command-line option the first time you run it with the copied profile.

Does it work?

[kswartz26]

Ah, but the upgrade of 56.2 was in place.

[laniakea64]

Ah, but the upgrade of 56.2 was in place.

The issue you had with the in-place upgrade is likely unrelated to the issue involving Waterfox 68.

[kswartz26]

I agree with that now - rather, I think it's the same issue in both cases, and is not specific to WF 68. (I just forgot to edit the issue title. Which, ironically, I can't do in Waterfox 56. :-) )

[grahamperrin]

edit the issue title. … can't do in Waterfox 56.

Not reproducible here. Would you like to raise a separate issue? Thanks.

[kswartz26]

I am going to pull out a backup and see if I can reproduce it myself (will take me a couple of days). I just looked and found that a second extension disappeared in my 56.2.9 to 56.2.10 update. Would you prefer a separate issue altogether? (I just renamed that one, but can start over if you prefer.)

[grahamperrin]

Thanks,

56.2.10 update. Would you prefer a separate issue altogether?

For Waterfox Classic: yes please.

The separation will help to focus this issue on Waterfox alpha.

[kswartz26]

Ah. Will do - didn't realize separate projects.

• Keith

[grahamperrin]

Meta, tracking: #538

Possible cross reference _part of_ https://redd.it/deuzev for 68.0b2, in particular:

… Most of the extensions had no icons …

I don't plan to reinstall 68.0a1 but I imagine that an icon-free appearance might have been associated with true or false for extensions.htmlaboutaddons.enabled. (true by default in 68.0b1, I can't recall whether things were the same with 68.0a1.)

[grahamperrin]

@kswartz26 at the head of this issue, you might like to edit the subject line. I suggest:

Waterfox Current (and Firefox Quantum) cause loss of some types of legacy extension when used to open Waterfox Classic profiles

I'll add a separate comment to put this in context. Thanks.

[grahamperrin]

Commentary at and under https://www.waterfox.net/blog/waterfox-2019.10-release-download/#commento-comment-card-e4f3a054c5e3445cc1683b02703516712b10c86a7e574fe66f441e8608a38e11 draws attention to losses that occur with this combination of applications:

• Waterfox Classic 2019.10
• Waterfox Current 2019.10

Specifically:

… XUL overlay type extensions, the type that require browser restart after install/update/enabled/disable …

A screen recording of this type of loss, using Waterfox Profile Manager:

• https://put.re/player/cgG7BKuf.mp4
• Waterfox Current 2019.10 versus legacy XUL overlay extensions.zip

The extension in this case:

• Diigo Toolbar 5.1.0.38
• caa:addon/diigo-web-collector
• https://ca-archive.biz.tm/storage/2/2792/diigo_toolbar_annotate_screenshot_bookmark-5.1.0.38-fx.xpi

Losses are silent and likely to cause annoyance so I suggest a bug label for this issue.

Meta, tracking: #582 then #538 – first, document the bug; then aim to fix it.

Related:

• Backport (Firefox 67) downgrade protection to Waterfox Classic · Issue #1246
• Waterfox Profile Manager should gain a distinctive title and content in both Waterfox Classic and Waterfox Current · Issue #1247

If we're to add anything like _upgrade protection_ to Waterfox Current, it need not be as strict as (Firefox 67) downgrade protection.

It should suffice to present a yellow alert ⚠ modal dialogue – with an option to quit – whenever a start of Waterfox Current detects Waterfox Classic profile content. Escape should quit.

Modality in good time – before whatever will trigger the removal of this type of extension.

[grahamperrin]

PrefBar

… The addon that "disappeared" when going from 56.2.9 to 56.2.10 was PrefBar 7.1.1, …

I can nuke the 68a profile, make a new copy, and try to reproduce it.

caa:addon/prefbar

Tested on Manjaro Linux with Waterfox Classic then Waterfox Current (2019.10). Using Waterfox Profile Manager.

Added to Waterfox Classic 56.2.9, restart required. The extension was present and enabled following the restart. Quit.

Started Waterfox Classic 56.2.10. The customary one-off check for compatibility did not result in removal of the extension. Quit.

Started Waterfox Classic 2019.10 (20191023220600). The customary one-off check for compatibility did not result in removal of the extension. Quit.

Started the same profile with Waterfox Current 2019.10. This removes the extension.

[grahamperrin]

A thought-provoking about:support page (not my own) from Firefox 70:

• https://pastebin.com/bUtLzmp9

Three of the four disabled extensions:

Name: Classic Theme Restorer
Version: 1.7.3.1
Enabled: false
ID: ClassicThemeRestorer@ArisT2Noia4dev

Name: Classic Toolbar Buttons
Version: 1.5.9
Enabled: false
ID: CSTBB@NArisT2_Noia4dev

Name: Status-4-Evar
Version: 2017.08.20.15
Enabled: false
ID: [email protected]

Both Classic Theme Restorer caa:addon/classicthemerestorer and Status-4-Evar caa:addon/status-4-evar require a restart when added to Waterfox Classic:

Are those two _XUL overlay_ extensions, or not?

I assume that they're not XUL overlay extensions because neither one is removed by Waterfox Current 2019.10:

[grahamperrin]

There's uncertainty re: _XUL overlay_ as a definition of what's affected.

Unpacked extensions

Arising from discussions in IRC:

… PrefBar uses <em:unpack>true</em:unpack>, so maybe that plays into it. …

A CRX Viewer view of source code for the CAA copy of PrefBar 7.1.1:

https://robwu.nl/crxviewer/?crx=https%3A%2F%2Fca-archive.biz.tm%2Fstorage%2F67%2F67148%2Fprefbar-7.1.1-fx%2Bsm.xpi

Seeking !unpack finds the phrase at line 9 of install.rdf.

From Removing Support for Unpacked Extensions | Mozilla Add-ons Blog (2018-08-22):

With the release of Firefox 62 … Mozilla will discontinue support for unpacked sideloaded extensions. You will no longer be able to load an extension via the Windows registry by creating an entry with an extension’s directory (i.e. unpacked) after Firefox 61. Starting with Firefox 62, extensions sideloaded via the Windows registry must be complete XPI files (i.e. packed).

…

Note: this does not impact the use of unpacked extensions for development. Temporarily loading and debugging an extension either via the about:debugging page or via the web-ext tool will continue to be supported; both methods will still be able to load extensions contained within a filesystem directory. Developers will not be required to sign or pack their extensions into an XPI file for development purposes.

Related:

• Extension Packaging - Archive of obsolete content | MDN

I made removal of PrefBar, from the file system, reproducible with Firefox on FreeBSD. Terminal output below.

[grahamperrin]

⚠ Caution

In this case a symbolic link is used to share a single profile between Waterfox Classic and Firefox Quantum 70.⋯

• ⚠ with or without a symlink, sharing a Waterfox Classic profile with Firefox Quantum (or any Firefox Quantum-like browser, such as Waterfox Current) is NEVER recommended.

grahamperrin@momh167-gjp4-8570p:~ % date ; uname -v
Thu Oct 31 03:02:55 GMT 2019
FreeBSD 13.0-CURRENT #35 r354082: Sat Oct 26 00:25:05 BST 2019     root@momh167-gjp4-8570p:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG 
grahamperrin@momh167-gjp4-8570p:~ % pkg query '%o %v %R' firefox waterfox
www/firefox 70.0_3,1 FreeBSD
www/waterfox 56.2.14 poudriere
grahamperrin@momh167-gjp4-8570p:~ % ls -dhl ~/.mozilla/firefox/d35cb4u3.983
lrwxr-xr-x  1 grahamperrin  grahamperrin    41B Oct 30 20:49 /home/grahamperrin/.mozilla/firefox/d35cb4u3.983 -> /home/grahamperrin/.waterfox/d35cb4u3.983
grahamperrin@momh167-gjp4-8570p:~ % waterfox -p 983
1572491006955   addons.xpi      WARN    Failed to call uninstall for [email protected]: Error: Unknown add-on ID [email protected] (resource://gre/modules/addons/XPIProvider.jsm:6007:11) JS Stack trace: [email protected]:6007:11 < [email protected]:2778:31 < [email protected]:3118:19 < [email protected]:2188:25 < [email protected]:269:12 < [email protected]:739:5 < [email protected]:906:9 < [email protected]:3090:5 < [email protected]:65:9
1572491006957   addons.xpi      WARN    Attempted to remove [email protected] from app-profile but it was already gone
grahamperrin@momh167-gjp4-8570p:~ % waterfox -p 983
grahamperrin@momh167-gjp4-8570p:~ % firefox -p 983
1572491061360   addons.xpi      WARN    Not converting unknown addon type undefined
1572491062654   addons.xpi-utils        WARN    updateMetadata: Add-on {8A6C82A1-F6C9-481a-AAE7-C96444C9A754} is invalid: Error: File /home/grahamperrin/.mozilla/firefox/d35cb4u3.983/extensions/{8A6C82A1-F6C9-481a-AAE7-C96444C9A754} does not contain a valid manifest(resource://gre/modules/addons/XPIInstall.jsm:669:11) JS Stack trace: [email protected]:669:11
[email protected]:226:15
[email protected]:753:22
[email protected]:2817:32
[email protected]:3033:23
[email protected]:3120:31
[email protected]:2924:55
[email protected]:2384:12
[email protected]:213:31
[email protected]:649:17
[email protected]:873:14
[email protected]:3469:26
[email protected]:86:29
1572491064386   [email protected]     WARN    Loading extension '[email protected]': Reading manifest: Invalid extension permission: mozillaAddons
1572491064386   [email protected]     WARN    Loading extension '[email protected]': Reading manifest: Invalid extension permission: telemetry
1572491064386   [email protected]     WARN    Loading extension '[email protected]': Reading manifest: Invalid extension permission: resource://pdf.js/
1572491064386   [email protected]     WARN    Loading extension '[email protected]': Reading manifest: Invalid extension permission: about:reader*
JavaScript error: resource://gre/modules/addons/XPIProvider.jsm, line 1855: Error: Cannot find loader for bootstrap
JavaScript error: resource://gre/modules/addons/XPIProvider.jsm, line 1855: Error: Cannot find loader for bootstrap
…
JavaScript error: resource://gre/modules/addons/XPIProvider.jsm, line 1855: Error: Cannot find loader for bootstrap
JavaScript error: resource://gre/modules/addons/XPIProvider.jsm, line 1855: Error: Cannot find loader for bootstrap

###!!! [Child][MessageChannel] Error: (msgtype=0x37010E,name=PContent::Msg_ReportContentBlockingLog) Closed channel: cannot send/recv


###!!! [Child][MessageChannel] Error: (msgtype=0x370124,name=PContent::Msg_DetachBrowsingContext) Closed channel: cannot send/recv

grahamperrin@momh167-gjp4-8570p:~ % waterfox -p 983
1572491209521   addons.xpi-utils        WARN    Could not find source bundle for add-on [email protected]: [Exception... "Component returned failure code: 0x80520001 (NS_ERROR_FILE_UNRECOGNIZED_PATH) [nsIFile.initWithPath]"  nsresult: "0x80520001 (NS_ERROR_FILE_UNRECOGNIZED_PATH)"  location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm -> resource://gre/modules/addons/XPIProviderUtils.js :: parseDB :: line 491"  data: no] Stack trace: parseDB()@resource://gre/modules/addons/XPIProvider.jsm -> resource://gre/modules/addons/XPIProviderUtils.js:491 < syncLoadDB()@resource://gre/modules/addons/XPIProvider.jsm -> resource://gre/modules/addons/XPIProviderUtils.js:422 < checkForChanges()@resource://gre/modules/addons/XPIProvider.jsm:3168 < startup()@resource://gre/modules/addons/XPIProvider.jsm:2188 < callProvider()@resource://gre/modules/AddonManager.jsm:269 < _startProvider()@resource://gre/modules/AddonManager.jsm:739 < startup()@resource://gre/modules/AddonManager.jsm:906 < startup()@resource://gre/modules/AddonManager.jsm:3090 < observe()@jar:file:///usr/local/lib/waterfox/omni.ja!/components/addonManager.js:65
1572491209526   addons.xpi-utils        ERROR   Failed to load XPI JSON data from profile: Error: Expected passed argument to contain a path (resource://gre/modules/addons/XPIProvider.jsm -> resource://gre/modules/addons/XPIProviderUtils.js:199:11) JS Stack trace: [email protected]:199:11 < [email protected]:498:24 < [email protected]:422:9 < [email protected]:3168:9 < [email protected]:2188:25 < [email protected]:269:12 < [email protected]:739:5 < [email protected]:906:9 < [email protected]:3090:5 < [email protected]:65:9
1572491211981   addons.repository       WARN    Search failed when repopulating cache
ERROR - ContextualIdentityService - Unknown version found in /home/grahamperrin/.waterfox/d35cb4u3.983/containers.json
grahamperrin@momh167-gjp4-8570p:~ % waterfox -p 983
grahamperrin@momh167-gjp4-8570p:~ % 

[grahamperrin]

With apologies for pinging re: obsolete technology …

@mattbasta @muffinresearch https://github.com/MrAlex94/Waterfox/issues/983#issuecomment-548108473 above

Please, can either of you tell/guess what might cause automated removal of this type of extension from the file system?

TIA