User.js: Storage API blah blah & Bug 1418931

[Thorin-Oakenpants]

Edit2: Vote down here: https://github.com/ghacksuserjs/ghacks-user.js/issues/281#issuecomment-347747730

Edit: 1418931 Quota Manager in sanitize.js is not origin-aware

---

Before I left, I reversed a hasty earlier decision to flip 2706 to inactive (FF56->57 diffs, Mozilla flipped the switch on Storage API)

FYI 2706

   // user_pref("dom.storageManager.enabled", false); // (FF51+)
   // user_pref("browser.storageManager.enabled", false); // (FF53+)

Assuming people reset prefs as per changes, this is pretty much USELESS (afaik), because browser.cache.offline.enable (was 1004, now temporarily at 2720) is enforced as false = disabled, and thus the nice pretty UI section addition in "Options>Privacy & Security>Site Data" never gets to finish calculating data used, and the two shiny additional buttons are never used because they too are stuck in disabled-land

So .. WTF?!!?(#&@ ... all THREE prefs should to be on the same page IMO - either we allow them or kill them. Otherwise I see zero benefit in the current mixed setup.

So the question is: Do we allow this, or block it. I'm all for keeping this stuff blocked for now, at least until after 58 lands

[Thorin-Oakenpants]

earthlng said

I suggest we do the following changes for 57:

user_pref("browser.cache.offline.capacity", 1);
user_pref("browser.cache.offline.enable", true);
user_pref("browser.storageManager.enabled", true);
user_pref("dom.storageManager.enabled", true);
user_pref("privacy.firstparty.isolate", false);

Then the new Site Data UI works and offlineApps (Offline Website Data) clearing on shutdown or manually works correctly. A bit sad to disable FPI but IDK what else to do. What do you guys think?

edit: sanitize.js is not origin-aware in the latest nightly, see this vs _removeQuotaUsage.
That's why "Clear All Data" works but clearing it on shutdown and via "Clear Recent History" does not.

[Thorin-Oakenpants]

Relevant ^^ . No way in hell do we disable FPI - are the Tor Uplift guys aware of this (I'm finding it really hard to use the right terms and follow all the tickets for various storage thingies). Someone needs to lodge a ticket, and until it is fixed, we disable Storage API. Said ticket needs to be professionally worded with STR (so that counts me out!)

Class ... discuss!

[earthlng]

https://bugzilla.mozilla.org/show_bug.cgi?id=1418931

until it is fixed, we disable Storage API

what do you mean? dom.storage.enabled ? Because afaik disabling the .storageManager.* stuff doesn't really change anything - you still won't be able to delete FPI-ed IDB etc.

[Thorin-Oakenpants]

No .. this issue is about the two prefs in 2706 which is explicitly referred to as the new Storage API by mozilla

/* 2706: disable Storage API
...
* [2] https://developer.mozilla.org/docs/Web/API/Storage_API

nothing to do with us changing dom.storage.enabled - never even mentioned it

[earthlng]

Site storage - the data stored for a Web site which is managed by the Storage Standard

It manages the stuff but doesn't actually control or disable any of the "real" APIs. I mean, feel free to change it but I don't think it makes any sense.

[Thorin-Oakenpants]

Two issues

• 1) it never finishes trying to calculate data and thus the new UI is about as useful as tits on a bull
• 2) Clearing 1st-party IDB shit

This was about issue 1, I didn't even think of issue 2 because I'm selfish and never have any IDB entries because I only allow about 8 cookies in tots and none of them use IDB

So to fix issue 1 we should set browser.cache.offline.enable->true so the data finishes calculating and the buttons are enabled. Otherwise WTF is the point? Unless I'm missing something here. If the two prefs in 2706 only manage the APIs, thats great .. but WTF does browser.cache.offline.enable open us up to? Do you see what I'm getting at now?

So, make all three active and forget it until FF58, or make all three inactive to provide an extra UI feature with a flaky clear data button?

Issue 2: FPI IDB entries seem to be cleared with recent history as per ticket description. But they fail on that new UI button (so why open users up to that?). Question: is FPI'd IDB cleared on close working?

[Thorin-Oakenpants]

Note: AFAIK, zilla are building on clearing data in a lot of places - on close, clear recent history, in site permissions (I think OWD has been removed as a site pref, but it was there) via addons (localstorage and idb is available in 58), forget site, etc - I bet I missed some. The problem is not FPI, the problem is Storage API - I say we disable it until it is fixed - meanwhile, there are other clearing methods, and let's face it IDB clearing was broken for years and years. What's another 8 weeks

[earthlng]

Issue 2: FPI IDB entries seem to be cleared with recent history as per ticket description. But they fail on that new UI button (so why open users up to that?). Question: is FPI'd IDB cleared on close working?

no it's the opposite. A: no

[Thorin-Oakenpants]

Soz, the opposite - recent history and clear on close are broken but the new UI button is not - so now I see why you want it enabled - so people have a way to clear IDB (which was broken for decades and you can do it manually).

OK - so what is the downside to allowing browser.cache.offline.enable. The FPI issue will get fixed in time.

[Atavic]

Keep'em in false status, Yeah?

[Thorin-Oakenpants]

earthlng said

@Thorin-Oakenpants what's the source for this part in 2706:

and even control whether or not they need to be alerted before the user agent disposes of site data in order to make room for other things.

link [2] in 2706 - https://developer.mozilla.org/en-US/docs/Web/API/Storage_API - 1st paragraph dude

I'm trying to dig into more detail, but I assume it would be something like this: https://w3c.github.io/ServiceWorker/#privacy and resource maps

[Thorin-Oakenpants]

This Storage API is nasty, seriously. I get why its being developed, but it's nasty. From what I can tell, this is for websites - Firefox will work perfectly well without it (for now). Look at the sh*t (mostly understandable given the name of the API) it can access and control (to various degrees?)

• IndexedDB databases
• Cache API data
• Service Worker registrations
• Web Storage API data managed using window.localStorage
• History state information saved using History.pushState()
• Application caches
• Notification data
• Other kinds of site-accessible, site-specific data that may be maintained

I can see this being totally exploited for tracking by created new avenues for regeneration of uniqueness. "Other kinds of data, user-specific" <- fixed that. Yeah, I get it, anything can be user-specific. And that we can shut down some storage methods (eg notifications) and clear persistent data (well, it's all a bit fked at the moment, so humor me), but to me this is just giving sites carte blanche access into your local data, albeit 1st party. 1st party tracking** - even sites you log into, but then subsequently visit incognito.

Tots want to disable this (Storage API) at least until 59. It's only just been flipped by Mozilla - it is not used in ESR yet, or Palemoon, or Waterfox, etc etc. It is *NOT NECESSARY

All in favor of killing the witch, say Aye

[Thorin-Oakenpants]

Vote: (non-binding: just want feedback)

Storage API 2706 vote

:+1: enable Storage API (i.e we make the two prefs inactive)

:-1: disable Storage API (i.e we make the two prefs active as false)

Edit: prefs for reference

   // user_pref("dom.storageManager.enabled", false); // (FF51+)
   // user_pref("browser.storageManager.enabled", false); // (FF53+)

[earthlng]

For the x-ed and last time: https://developer.mozilla.org/en-US/docs/Web/API/StorageManager

The StorageManager interface of the the Storage API provides an interface for managing persistance permissions and estimating available storage.

Methods
StorageManager.estimate() - Returns a StorageEstimate object containing usage and quota numbers for your origin.
StorageManager.persist() - Returns true if the user agent is able to persist your site's storage.
StorageManager.persisted() - Returns true if persistence has already been granted for your site's storage.

FF doesn't even return the exact "usage and quota numbers" but rather an estimate "to obscure the exact size of the data used by a given origin, to prevent these values from being used for fingerprinting purposes."
The StorageManager itself DOES NOT STORE ANYTHING!!

Please explain what's so terribly bad about that?? I'm sorry I just don't see it.
If it makes you feel better to disable them, go ahead and do it.

[Thorin-Oakenpants]

Then what use it is :) Certainly not for the end-user. I see zero benefit (at least for now) to enable this (except maybe the clear IDB w/FPI working in the new UI, but IDB was broken for years and everyone dealt with it, and that will get fixed in time)

For the x-ed time ... in order to make the UI work, these two prefs require browser.cache.offline.enable = true, and we currently enforce that as false. So why bother?

[Atavic]

The Storage API is now available only to secure contexts (FF 55):
https://github.com/Fyrd/caniuse/issues/3122#issuecomment-308404089

[2glops]

The StorageManager itself seems to do nothing, it's just a tool for managing storage.

The Storage_API :
https://developer.mozilla.org/en-US/docs/Web/API/Storage_API

Site storage is a mystery.
Anyway, I'm inclined to deny all kinds of storage unless i have issues.

[Atavic]

As a strawperson, here are the 7 things that we believe matter most in privacy:

• Is you data used for secondary use? And is it shared with 3rd parties?
• Is your data bartered?
• Under what terms is your data shared with the government and with law
  enforcement?
• Does the company take reasonable measures to protect your data in all
  phases of collection and storage?
• Does the service give you control of your data?
• Does the service use your data to build and save a profile for non-primary
  use?
• Are ad networks being used and under what terms?

See: https://www.w3.org/2010/api-privacy-ws/papers/privacy-ws-22.txt

[Thorin-Oakenpants]

^^ not really concerned about that. In this model, all data and websites are treated equally (choosing which bank or email service or social media etc, that's a different story). As for those questions above, they are moot IMO - it's not if but when (company is sold & data sold off, company is hacked, terms are changed etc etc). Point 7: Answer: f**k all ad networks :)

[Thorin-Oakenpants]

• Bug 1418931 Quota Manager in sanitize.js is not origin-aware is being tracked in FPI sticky
• I'll add Storage API to the watching sticky as a reminder to revisit although we still have #235 (persistent data section revamp)
• Am setting Storage API back to disabled.
  
  
  
  • Reasons: 1. It is not necessary eg see ESR/Firefox forks 2. possibly some entropy/data concerns 3. Requires offline cache which we block, so the UI extras are useless anyway
  
  

[Thorin-Oakenpants]

note: bug 1418931 is being tracked in the FPI sticky

[Thorin-Oakenpants]

FYI: 1418931 = closed since it now only pertains to ESR and they'll just not fix it since ESR60 is not far away