User.js: tracking protection (edit: and SB) - why exactly is it disabled?

[RoxKilly]

The implementation guide explains:

Tracking Protection (TP) and Safe Browsing (SB) are turned off (section 0400) -- We think you can do much better than this (wider scope, no over-reach / censorship)

Could someone please elaborate? Is the block because the browser has to connect to a remote server to download blocklists? @Thorin-Oakenpants do you operate under the assumption that this user.js must be used in conjunction with uBlock Origin? If this is meant for both uBO users and non-users, why disable TP?

Setting that aside, let me make the case for TP even for a uBO user like myself: For the vast majority of webpages, TP never plays a role because uBO blocks requests before they get to the TP code (see the last comment from link 3 below). So there is no additional burden on the browser and I don't see an additional privacy exposure (beyond the blocklist downloads).

In some cases, default uBO filter lists and settings let something through the cracks and TP actually catches it (eg: enable Tracking Protection and open this page as of May 4 2017). This is usually a tracking image of some sort. In those cases I'm glad to have TP on.

For Reference

1. How TP works
2. TP wiki
3. TP authors' post -- the comments below the post yield a lot of useful info; for instance although TP uses the SafeBrowsing protocol, there is no interaction with Google..

[earthlng]

In some cases, my uBO filter lists and settings let something through the cracks and TP actually catches it.

afaik TP uses the same lists (Disconnect) that are already available in uBO. Either your uBO lists are not updated or I'd love to see an example of those cases you're mentioning

[RoxKilly]

@earthlng wrote:

afaik TP uses the same lists (Disconnect) that are already available in uBO. Either your uBO lists are not updated or I'd love to see an example of those cases you're taking about.

You probably have a point there. The Disconnect lists were not selected in my uBO filters. Is Tracking Protection (TP) list limited to Disconnect? Doesn't this link suggest otherwise? The next time I come across such an example, I'll try to remember to post it here; it happens rarely though.

If you guys have made the conscious decision to not cater to people who don't also use uBO then I understand the decision better. If this user.js is meant to be privacy-enhancing on its own, then I don't understand why TP would be off by default: its upside for privacy is so huge for those without uBO or something similar.

Besides, if uBO Disconnect list is really a drop-in replacement, I don't see the downside of having TP on because that code path will never be traveled anyway for a user with that uBO list active. As my 3rd link shows, TP is only triggered if extensions have allowed the connection to proceed.

@Thorin-Oakenpants wrote:

As for TP, I have read numerous accounts of this being a cause for breakage - so to me personally, I prefer to have one less item to deal with.

Consider: Firefox gives the option of two lists, a basic list (default) meant to have few breaks, and a strict list meant to provide stronger protection. Because the lists are updated every hour (I think) and are maintained by a company that makes it its business to provide privacy tools (Disconnect), I suspect that breakages in the basic list are few and far between. I have no empirical evidence though. Just personal experience (never seen it myself)

[RoxKilly]

@Thorin-Oakenpants I did read the Implementation wiki. I even quote it in the OP. But the only thing it says about uBO is:

There is nothing wrong with running TP [sic] and SB as well as uBlock Origin

This doesn't get across that people who use this without uBO should not expect privacy. If that is your position, consider making it more obvious because as things stand, such a user would be more exposed using your user.js than using default Firefox in Private Browsing (where TP is ON)

I tried to make the case for Tracking Protection regardless of whether uBO is used; please take another look at my previous post. Thanks for reading.

[earthlng]

Is Tracking Protection (TP) list limited to Disconnect?

it seems so:

The blocklist is created by Disconnect according to their definition of tracking.

source: https://feeding.cloud.geek.nz/posts/how-tracking-protection-works-in-firefox/

---

You can choose between recommended and strict, and that changes urlclassifier.trackingTable

strict:                test-track-simple,base-track-digest256,content-track-digest256
default (recommended): test-track-simple,base-track-digest256

If you look at this both base-track-digest256 + content-track-digest256 are based on this and coming from disconnect.

[earthlng]

Thanks for reading.

Pants can sometimes be a bit salty. @RoxKilly, please don't let this spook you away from here. You contribute quality posts and comments, and I'd hate to see you abandoning us.

[RoxKilly]

@earthlng Thanks for the encouragement. I'm in no danger of leaving; I was being sincere with my thanks. I'm curious: what do you think about following assessment:

• for people without a uBO-like tool, having TP enabled is a major privacy win. Disabling TP exposes them to tracking

• for people with uBO, TP offers little to no added benefit, but no downside either because that code path is never traveled when uBO blocks network requests

• so enabling TP is a win for some, and makes no difference to others.

Gets back to my original question: what is the benefit of disabling it?

[earthlng]

@RoxKilly wrote:

I'm curious: what do you think about following assessment

I see what you mean but I just can't imagine that someone would use something like this user.js but not use uBO. It can be used simply as a better version of ABP. You don't even have to use the more advanced features and IMO it's really easy to setup and use (in easy mode, at least). Not to mention the amazing Element picker.

so enabling TP is a win for some, and makes no difference to others

it makes the slight difference that it downloads the blocklists (every hour?) basically for no reason at all. #NotSoQuietFox

And as for the sentence you two are arguing about ....

There is nothing wrong with running TP and SB as well as uBlock Origin

I'd say the important part in that is: as well as uBlock Origin

IMHO uBO is an absolute must-have addon, and I'd rather we make that abundantly clear instead of enabling TP (and/or SB)

[earthlng]

What is the pref for which TP list to use.

Do you not read, SaltyPants? ;)
https://github.com/ghacksuserjs/ghacks-user.js/issues/102#issuecomment-298346502

When changing it in the options FF needs to restart btw

[RoxKilly]

@Thorin-Oakenpants wrote:

What is the pref for which TP [sic] list to use. What setting do we give it?

I don't yet know which preference determines whether the basic or strict protection list is used. I know how to change it in the UI, if that helps: With privacy.trackingprotection.ui.enabled = true, go to about:preferences#privacy and click the button labeled "Change Block List" to make the change. My advice is to leave it as the default ("basic list", same as that in the uBO Disconnect filter)

@Thorin-Oakenpants wrote:

We'll also have to make sure the prefs used to get the URLs are reset as well

To get tracking protection working in my browser all I had to do was comment out 0410d and set 0420 prefs to true. I did not need to fix the URLs in 0410e and 0410f

[earthlng]

I don't yet know which preference determines whether the basic or strict protection list is used.

seriously? you too? why am I even posting here if nobody reads my shit xD

[RoxKilly]

@earthlng LOL

The download itself doesn't occur every hour. A check is made for whether there is an update every hour:

Every hour, Firefox requests updates from shavar.services.mozilla.com. If new data is available, then the whole list is downloaded again. Otherwise, all it receives in return is an empty 204 response. (source)

[RoxKilly]

@Thorin-Oakenpants I simply commented out those two lines. My copy of user.js has:

   // user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "");
   // user_pref("browser.safebrowsing.provider.mozilla.updateURL", ""); 

In about:config, these are the values I see:

browser.safebrowsing.provider.mozilla.gethashURL = https://shavar.services.mozilla.com/gethash?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2
browser.safebrowsing.provider.mozilla.updateURL = https://shavar.services.mozilla.com/downloads?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2

I've also just confirmed that Tracking Protection is working by disabling uBO, then loading nytimes.com

[earthlng]

Yes earthlng - I read your stuff.

THANK YOU, ffs :)

Like I said, create a PR

Hell no! I like the way it is right now. Beginning to hate @RoxKilly - now you're 2 against 1 - that sucks! xD

/* 0420: disable Tracking Protection (TP)
 * There SHOULD be NO privacy concerns here, but we strongly recommend to use uBlock Origin instead,
 * which offers more comprehensive as well as specialized lists. ... ***/

strongly recommend to use uBlock Origin instead

[RoxKilly]

@Thorin-Oakenpants wrote:

Options>Privacy>Using Tracking Protection: "only in private windows" vs "always" - what pref is this again?

"TP only in private windows" is:

privacy.trackingprotection.enabled = false
privacy.trackingprotection.pbmode.enabled = true

Those are the default settings by the way

"TP always" is:

privacy.trackingprotection.enabled = true
privacy.trackingprotection.pbmode.enabled is irrelevant (I just checked)

Blocklist options - if we use strict/simple - this is across all windows right? normal & PB?

I don't know for a fact but I don't see why it would be any other way.

[earthlng]

Ok, I'll let you two wrap each other in TP while I go watch some funny cat videos.
If you're seriously gonna do this right now, you may want to consider dealing with these too:

pref("privacy.trackingprotection.annotate_channels", false);
pref("privacy.trackingprotection.lower_network_priority", false);

happy TP-ing you two, laterz

[Atavic]

Both set as False here, with the URL removed as Roxkilly.

Potential amazon tracking: https://www.robtex.com/dns-lookup/shavar.services.mozilla.com

[Atavic]

Browser retrieves the hash for updates from:
https://shavar.services.mozilla.com/gethash?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2

EDIT: SAFEBROWSING_ID is not the same as Google API_KEY.

By looking at robtex here, you see the service is hosted at Amazon.

[Atavic]

/* 1200 HSTS

HTTP Strict Transport Security (HSTS) with long duration deployed on this server.

Source

https://github.com/gorhill/uMatrix/issues/389
hsts-tracking

[Atavic]

My problem is AmazonCDN hosting and potential tracking because of it's huge online presence.

[crssi]

Huh... missed the whole party here. :)
Is there any final verdict? Since the English is not my mother language I have a bit trouble to follow up. ;)
What I like, if FF is doing the "protection", is the way FF shows it to the user as seen on http://itisatrap.org pages.... the red page with warning I mean.

To make it work, I have set those:
/* 0410a / user_pref("browser.safebrowsing.malware.enabled", true);
/ 0410a / user_pref("browser.safebrowsing.phishing.enabled", true);
/ 0410d / user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.services.mozilla.com/gethash?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2");
/ 0410d */ user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.services.mozilla.com/downloads?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2");

Does my comment make any sense?

EDIT: never think of what @Atavic said... good point
EDIT2: I am again in "my folks browsing mode"... they understand red warning page as BIG NO NO... but they don't understand uBo warning... it is simply not red color saying GO AWAY FROM HERE. ;)

[crssi]

Thank you pants. What about SB?
I was holding those questions for the lite version, but @RoxKilly opened a pandora box before lite version. ;)

[Atavic]

So Mozilla hosts such services on Amazon servers. Will Amazon use the connections to track the browser? They definitely could. as an API key is a unique value that is assigned to a user of the service

https://github.com/Synzvato/decentraleyes/issues/99

[crssi]

I will shut up now. :)

[fmarier]

pref("privacy.trackingprotection.annotate_channels", false);
pref("privacy.trackingprotection.lower_network_priority", false);

Note that enabling these won't do anything if privacy.trackingprotection.enabled is ON. These prefs exist to de-prioritize known trackers when they're not blocked outright. If you enable tracking protection, then the tracking resources don't need to be de-prioritized because they never load at all :)

Browser retrieves the hash for updates from:
https://shavar.services.mozilla.com/gethash?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2
where SAFEBROWSING_ID is the same as Google API_KEY.

As documented here, SAFEBROWSING_ID is not the Google API Key. Instead, it gets replaced with the contents of browser.safebrowsing.id (which defaults to navclient-auto-ffox).

[RoxKilly]

@Atavic , the use of HSTS by itself does not mean fingerprinting. HSTS is actually a great security feature because it protects against a man-in-the-middle downgrade attacks by forcing the browser to connect only over a secure channel by default.

Using HSTS to fingerprint requires having the browser make many connections to many domains (subdomains usually) and testing whether the browser knows to connect to these domains over HTTPS by default. So unless we have evidence that the Tracking Protection server connection involves many domains with HSTS there can be no fingerprinting of that sort.

In addition, Firefox doesn't save HSTS settings in Private Browsing mode, so in the browser's default configuration (Tracking Protection used during Private Browsing) this fingerprinting technique wouldn't even work. Again, I think we need evidence that there is even fingerprinting going on in the first place for this to be a concern.

@fmarier
I can confirm that my browser.safebrowsing.id is simply set to navclient-auto-ffox, so there is no inherent fingerprinting either in the URL that connects to the blocklist server:

browser.safebrowsing.provider.mozilla.gethashURL 
    => https://shavar.services.mozilla.com/gethash?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2

browser.safebrowsing.provider.mozilla.updateURL 
    => https://shavar.services.mozilla.com/downloads?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2

I'm having a hard time seeing how there is tracking going on. Seems to me that at most, the Amazon hosted server shavar.services.mozilla.com will know that a Firefox browser at my IP is using the Tracking Protection blocklist. This is true in default Firefox installs though so it's useless as a fingerprint vector.

[Gitoffthelawn]

I keep reading that uBo has the same default lists as TP.

Which lists are people talking about? When I compare the Disconnect tracking list in uBo to the TP lists, they appear very different.

[earthlng]

When I compare the Disconnect tracking list in uBo to the TP lists, they appear very different.

Indeed. No idea why that is. But TP thinks allowing google-analytics.com (among others) on google is totally fine. I can't take TP serious when it does things like that.
The mozilla version moved the questionable domains from "properties" to "resources" (compared to the original) but idk how much of a difference that makes.

edit: just tested in a vanilla FF with TP enabled and googlesyndication.com was not blocked, and that's in the same list and category as google-analytics.com. "recommended" or "strict" makes no difference for this.
I'm sorry but TP is a joke. I guess we could disable the whitelist part but why bother - use uBO and be done with it

[Atavic]

@Roxkilly Thanks for reminding me that Private Browsing mode allows to surpass the potential privacy problem.

Regarding AmazonCDN servers, I keep an eye on my browser's connections with TCPView by Sysinternals and these servers are almost always present on top sites.

Is there a potential tracking issue here? Yes! (that's enough to me)

It's also worth noting that if a browser got a single HTTP -> HTTPS redirect and then picked up HSTS on one site, it'd never see it again on any other site using the CDN.

Source: https://github.com/MaxCDN/bootstrap-cdn/issues/750#issuecomment-240039399

Also, an interesting read as _Example of HSTS cross-origin history sniffing_ (Page 28):
http://web.mit.edu/zyan/Public/appseccali.pdf

[earthlng]

Is browser.safebrowsing.provider.mozilla.gethashURL even used for TP? I don't think so

[fmarier]

Indeed. No idea why that is. But TP thinks allowing google-analytics.com (among others) on google is totally fine. I can't take TP serious when it does things like that.

TP is designed to block _third-party_ tracking. If you're visiting GMail for example, they are clearly able to track you regardless of whether or not you block google-analytics.com. They have you in their server logs and they can run all the JavaScript they want in your browser.

On the other hand, Google shouldn't have to know that you're visiting CNN so google-analytics.com will be blocked by TP there.

Is browser.safebrowsing.provider.mozilla.gethashURL even used for TP? I don't think so.

Not currently. The TP list doesn't need it since it contains the full hashes, not the partial hashes like the other Safe Browsing lists.

[RoxKilly]

@earthlng wrote:

But TP thinks allowing google-analytics.com (among others) on google is totally fine. I can't take TP serious when it does things like that...and googlesyndication.com was not blocked

Technically, google-analytics on a google site is not 3rd party tracking. Also consider: The objective of TP is to reduce tracking without breaking sites. Where sites break, functionality may supersede privacy. Firefox (or Disconnect) may have discovered that in the wild, a lot of sites were breaking. Mozilla even wrote a special article about the breakage here.

What uBO does -- and one reason it's such an amazing tool -- is use its redirect syntax to replace the tracking functions with a neutered one, so that sites won't break. For people without uBO & Co., Tracking Protection is a big deal and it significantly reduces 3rd party exposure; I wouldn't be so dismissive.

[earthlng]

@fmarier, thanks. But the google-analytics.com scripts are doing all (or most) of the tracking, don't they?

So how does that work exactly - ELI5 please - all the domains in "properties" can load whatever they want from the domains listed in "resources" ? (list here)
I'm confused because you mentioned gmail but that is listed under "resources". Or can all domains in "properties" AND "resources" access all domains in both lists? Just out of curiosity.

Is browser.safebrowsing.provider.mozilla.gethashURL even used for TP? I don't think so.

Not currently.

Why would it change and suddenly use it too, if it works perfectly fine without it?

@RoxKilly so you're trying to tell me that google.com would break if google-analytics.com would be blocked? Come on - even if that was the case it was deliberately created that way, exactly because, as I suspect, google-analytics.com scripts are google's main tracking feature.
I can use google.com with scripts blocked and uBO doing its thing in the background - no problems at all.

I wouldn't be so dismissive

TP is definitely better than no protection but I'm arguing in the context of this user.js here

[fmarier]

But the google-analytics.com scripts are doing all (or most) of the tracking, don't they?

That's a question for Google :) They certainly have the technical capability to track their users in lots of other ways when they connect to their own servers.

I would assume that they have the ability to track me when I use their services regardless of the add-ons / privacy settings I have. The possible exception would be if I'm using the Tor Browser Bundle.

all the domains in "properties" can load whatever they want from the domains listed in "resources" ?

Yes, that's essentially how it works.

I'm confused because you mentioned gmail but that is listed under "resources". Or can all domains in "properties" AND "resources" access all domains in both lists?

GMail is confusing because the app is on google.com (which is in properties) and not gmail.com.

so you're trying to tell me that google.com would break if google-analytics.com would be blocked?

For an example where things are quite broken, try blocking twimg.com all the time. Twitter is unusable without it. With the entity list, we can group twitter.com and twimg.com together (since they are the same company) and eliminate that breakage while blocking twimg.com as a third-party tracker on other sites.

[earthlng]

That's a question for Google :)

I doubt that they would be all too willing to let us know :(
"Don't be evil" my ass

Yes, that's essentially how it works.

Essentially? with exceptions? if ELI5 is too much to ask, ELI12 or so is fine too :) nvm, open-source

twimg.com

allowing domains that host primarily css/images vs allowing domains who's sole purpose is to serve "analytics" scripts is quite different.

edit:
In retrospect my statement about TP being a joke was admittedly a bit harsh.
Maybe you were hinting at this (without feeling at liberty to say so directly maybe), but I see now that if you blocked GA & co on google etc, they could just adapt to it and work around it - not to mention maybe being pissed off a bit and potentially throwing you a smaller bone next time ;)

It's great that TP is included in FF, even enabled by default in PB, and I'm sorry if I offended someone. Keep up the great work

(and NO - Putin did not make me edit that [#CNN] - hindsight is a bitch)

[Gitoffthelawn]

@earthlng What Disconnect list are you using in uBo to cover a superset of everything in TP?

[earthlng]

in uBO? none. I block domain lists in uMatrix. Apart from the default ones I use these (with limited usefulness)

https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt‎ : 0 used out of 2'703
https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt‎ : 568 used out of 5'337
https://s3.amazonaws.com/lists.disconnect.me/simple_malware.txt‎ : 0 used out of 2'601
https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt‎ : 0 used out of 34

Disconnect doesn't exactly make it very easy to find these, so if you know of better ones that aren't included by default in uBO or uMatrix, please share

[RoxKilly]

@earthlng wrote:

so you're trying to tell me that google.com would break if google-analytics.com would be blocked?

No I was referring to other sites that rely on google-analytics in a way that breaks the site if the analytics aren't loaded properly. Anyway I feel like we're going in circles. We're in general agreement (uBO > TP > nothing). I only jumped in because you initially hinted that TP was a joke. but I took exception to that.

[Gitoffthelawn]

In #103, the great @Thorin-Oakenpants wrote:

TP uses the same list as uBo (note: there are two lists, simple+strict, default is simple and we would leave it at that but include the pref for info/enforcing strict)

What are you talking about? What list? Is uBo configured by default to use this list? Are these lists always identical?

[Gitoffthelawn]

@Thorin-Oakenpants No problem. Although it's certainly likely, I haven't seen anything conclusive that uBo (especially out of the box) actually covers everything that TP covers. Of course, uBo (or any content blocker) can block whatever TP blocks, but that requires a list in a compatible format.

[RoxKilly]

uBO doesn't use the Disconnect list by default. Here is its default setup. The 3rd party lists loaded are:

• EasyList
• Peter Lowe’s Ad server list
• EasyPrivacy
• Malware Domain List‎
• Malware domains

[RoxKilly]

Yes. I know this because as I pointed out in the OP:

In some cases, my uBO filter lists and settings let something through the cracks and TP actually catches it. This is usually a tracking image of some sort

I don't use the Disconnect list in uBO, though I have EasyList, Peter Lowe's, and EasyPrivacy ON. These occurrences are rare though; the next time I run across one, if I remember, I'll post it here

[crssi]

Those I have additionaly to uBo standard for TP:
https://raw.githubusercontent.com/quidsup/notrack/master/trackers.txt
https://raw.githubusercontent.com/piperun/iploggerfilter/master/filterlist
https://raw.githubusercontent.com/metaphoricgiraffe/tracking-filters/master/trackingfilters.txt
https://filters.adtidy.org/extension/chromium/filters/3.txt

I don't go FB, but to whome who does:
https://easylist-downloads.adblockplus.org/message_seen_remover_for_facebook.txt

I have also other collections for SB and others, if anyone interested.
Maybe this (uBo usage sharing information) should go into separate topic?

EDIT: If anyone sees some in this list that are total nonsense, please let me know ;)

[Gitoffthelawn]

@Thorin-Oakenpants wrote

thanks roxkilly : check the pretty picture above. My question now would be, is there ANYTHING covered in TP that is not covered by uBo's default?

I just checked these and updated:
Basic tracking list by Disconnect​ - 29 used out of 34
Malvertising filter list by Disconnect - ​​​​​639 used out of 5,334

Thanks. On the surface, those results indicate that much is covered by Disconnect (and hence possibly TP) that is not covered by the default uBo setup.

I'm not sure if the analysis is so simple, however. Wouldn't it depend on which order uBo processes lists? IOW, if 2 lists have foobar.com, then I think uBo would count the item as "used" in one list and not the other.

[crssi]

What I really miss in uBo is statistics... but I understand why those can't really be implemented. If those would be implemented the uBo would take a lot of CPU resources to do so.
What I mean by stats is that uBo would have a counter per list, how many hits overall and how many are unique to the specific list.
With that info we could easily decide, which lists are really needed and which are just sitting there doing nothing.

Crap, this comment might be just a spam here and should go to uBo "issue" instead.

[crssi]

Under Malware filter list by Disconnect I have 2 used out of 2598, checked now.
Two days ago I had 0 used out of XXX.
But I don't use this lists for obvious reason as they are covered by some other lists I am using.

[earthlng]

I think what @crssi means is a counter about which filters (and from which list) you actually encounter in your browsing.

[crssi]

Doh... my English again. :)

Yes, @earthlng, you are correct.

[Gitoffthelawn]

I was thinking about this issue a bit more, and a strong argument to keep TP enabled is that its list can be automatically updated multiple times per day (I believe).

To the contrary, most uBo lists are only automatically updated every 3 days, some only get automatically updated every 7 days, and I believe some may be scheduled to take even longer.

Yes, you can manually force uBo updates more frequently, but it's not really reasonable to expect users to all do that.

[earthlng]

the blacklist rarely changes though

edit: the whitelist neither

looking for updates every hour is completely unnecessary

[Gitoffthelawn]

@earthlng I tend to agree. And at the same time, I tend to disagree. :)

I agree because when the frequency of updates is infrequent, then looking for updates frequently isn't productive.

I disagree because it doesn't take any effort from the user, and only the most severely bandwidth-restricted users will notice. Also, sometimes a really invasive tracker can come along and needs to be blocked ASAP. That, as you pointed out, is relatively rare.

Now, when it comes to SB (vs TP), I am starting to think it's really important to have blocklists updated more frequently than every 3 days. And SB provides that. From what I can tell, SB would have blocked the phishing scam described in https://www.ghacks.net/2017/05/04/fell-prey-to-google-docs-phishing-scam-do-this/ but uBo would not have blocked it for a majority of its users.

[RoxKilly]

I wrote earlier:

In some cases, my uBO filter lists and settings let something through the cracks and TP actually catches it. This is usually a tracking image of some sort...I don't use the Disconnect list in uBO, though I have EasyList, Peter Lowe's, and EasyPrivacy ON. These occurrences are rare though; the next time I run across one, if I remember, I'll post it here

I got a hit. Turn on Tracking Protection then open this page with default uBO settings. uBO would not protect the user, but the built-in TP does.

[earthlng]

@Gitoffthelawn wrote

From what I can tell, SB would have blocked the phishing scam

What makes you think that?

Google has blocked the account in the meantime, removed the fake pages, and pushed updates to Safe Browsing on top of all that.

I'm reading that as SB did NOT protect users. It would be interesting to know what updates they pushed, because I don't really understand atm what exactly the problem was.

@RoxKilly wrote

I got a hit.

Can't replicate. Fresh profile with default uBO and this user.js but with TP enabled: "No tracking elements detected on this page". What isn't blocked that should be blocked?
And who said to use uBO with its default settings btw?

[RoxKilly]

@earthlng wrote:

Can't replicate. Fresh profile with default uBO and this user.js but with TP enabled: "No tracking elements detected on this page". What isn't blocked that should be blocked?

To replicate:

1. start with fresh Firefox 53 profile
2. Install uBlock Origin Add-On version 1.12.1 (current as of May 5 2017)
3. In uBO settings >> 3rd-party filters, click "Update now" to downlod latest default lists
4. Download the latest ghacks-user.js live master and place it in Firefox profile folder
5. Change pref 0420 to user_pref("privacy.trackingprotection.enabled", true);
6. Comment out the two prefs under 0410d because tracking protection relies on the safebrowsing API (maybe you forgot this step?)

//user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "");
//user_pref("browser.safebrowsing.provider.mozilla.updateURL", ""); 

1. Save the file and restart the browser.
2. Visit this page and you should get a Tracking Protection hit. The specific resource blocked is logged to the console.

Without TP, I would not have avoided this particular tracking mechanism. I get such a hit once or twice a week I think. If you don't get the same result, I suspect it's either because you skipped step 6 above or the Amazon server doesn't send the tracking bug to your PC, or your uBO settings aren't the default.

@earthlng wrote:

"Google has blocked the account in the meantime, removed the fake pages, and pushed updates to Safe Browsing on top of all that."
I'm reading that as SB did NOT protect users. It would be interesting to know what updates they pushed, because I don't really understand atm what exactly the problem was.

You guys were discussing the benefits of the frequent update check by the SafeBrowsing engine as an advantage over uBO malware lists. I think the point @Gitoffthelawn made is that because Google pushed a fix to SB infrastructure within an hour of the exploit being public, and since SB checks for updates every hour, people who use SB for protection would have had a much shorter vulnerable window (the quick update would protect most people who actually receive and eventually open the phishing email) than people who use just uBO with this user.js (they would have been exposed until their next manual update I think).

[Gitoffthelawn]

@RoxKilly wrote

You guys were discussing the benefits of the frequent update check by the SafeBrowsing engine as an advantage over uBO malware lists. I think the point @Gitoffthelawn made is that because Google pushed a fix to SB infrastructure within an hour of the exploit being public, and since SB checks for updates every hour, people who use SB for protection would have had a much shorter vulnerable window (the quick update would protect most people who actually receive and eventually open the phishing email) than people who use just uBO with this user.js (they would have been exposed until their next manual update I think).

Exactly. Your paraphrase of what I wrote made it much more clear though. :)

I think uBo would benefit from the ability to customize the list check interval. Actually, do you know if it checks lists for updates or always just downloads the most recent versions?

The downside, of course, is the additional traffic if too many people start asking for lists to be updated too frequently.. it would amount to a DDOS. But the number of people that would actually adjust such a setting would likely be minimal compared to the installed base.

Are the SB and TP lists available anywhere in a format compatible with uBo?

[earthlng]

@RoxKilly wrote

Comment out the two prefs under 0410d because tracking protection relies on the safebrowsing API (maybe you forgot this step?)

I _did_ forget that, sorry. Ok so amazoncustomerservice.d2.sc.omtrdc.net is the tracker in this case.
omtrdc.net is listed in the Adobe section of the TP list, and also in Dan Pollock’s hosts file in uBo/uM.
So if you use more than only the default settings in uBO, OR also use uMatrix with its default settings, nothing slips through the cracks (for that site at least).

@Gitoffthelawn wrote

I think uBo would benefit from the ability to customize the list check interval.

https://github.com/gorhill/uBlock/wiki/Advanced-settings

Actually, do you know if it checks lists for updates or always just downloads the most recent versions?

afaik it checks a checksum file from gorhill's github repo before updating lists.

Are the SB and TP lists available anywhere in a format compatible with uBo?

Not that I know. The TP list could however easily be parsed and converted into a compatible format. idk if the same is possible for SB. Would this be the format we want ... ?

## AddThis
||addthis.com^$third-party
||addthiscdn.com^$third-party
||addthisedge.com^$third-party

[Atavic]

FYI These lists have been implemented into Ubo Lists: https://github.com/chrisaljoudi/uBlock/issues/1406#issuecomment-105517545

[RoxKilly]

@Gitoffthelawn wrote:

Actually, do you know if it checks lists for updates or always just downloads the most recent versions?

@earthlng wrote:

afaik it checks a checksum file from gorhill's github repo before updating lists.

That may no longer be the case. I just came across this statement from gorhill from Jan 2017

uBO no longer uses checksums.txt resource hosted on GitHub to find out whether some specific assets have changed: the update logic is now completely time-based -- checksums.txt will be deprecated and will no longer be updated. Eventually in some future it will be removed from the repository

[crssi]

Interestingly this very page/tracker is explicitly allowed on the following lists:
EasyPrivacy (under uBo privacy)
English filter
Spyware filter

[Gitoffthelawn]

Hmmm... Popular lists such as EasyList have a string at the top to indicate how often to perform updates. For example: ! Expires: 4 days (update frequency).

IIRC, popular blocking programs like AdBlock Plus and uBlock Origin (uBo) will not automatically update the list from its source until that time has expired.

IIRC, when a user of AdBlock Plus manually updates a list, it will force an update, overriding the value specified in the header. IIRC, uBlock Origin will not update the list in this situation, even if the user performs a manual update.

IIRC, in uBo, even if the user specifies a lower value by using the autoUpdatePeriod advanced setting, it will still not override the expiration interval. Thus, it will not perform a manual update.

In uBo the user can purge all uBo caches and then perform a manual update of every list. AFAIK, this will manually update the lists. AFAIK, in uBo, there is no obvious way to manually purge and update a single list before it has expired, which can be days.

That's a lot of IIRC and AFAIK, so you may want to confirm.

[RoxKilly]

@Thorin-Oakenpants

Consider writing out SB >> SafeBrowsing and TP >> Tracking Protection in your last post, because it's one of reference (almost like documentation). Also you've mistakenly used TB where you meant TP. You've done it a couple of times in this thread or in the VOTE thread, but on this last post I think it's important to get it right. It might be linked to a few times in the future and people who come directly to it may wonder what SB, TP and TB are until they scroll up etc...