User.js: referrer add-ons discussion

[Thorin-Oakenpants]

moved from the sticky add-ons topic which is for recommending add-ons for adding/removal.

---

publicarray wrote:

regarding Smart Referer [github link]: my option is to to set the Mode to "Send nothing as referer, looking like a direct hit" and set a whitelist manually. The syntax can be inferred from the default whitelist

earthlng wrote:

Pants: I have used Smart Referer before and am not convinced of the "smarts"

IMO the smarts look pretty smart. If you remove the default whitelist, there's no more questionable smarts left that I can see and it's fully user-customizable.
It even allows for from>to. Without having ever used it before, I may give it a try, because I don't see how an addon could get it done any better than what this already does.
Hey mozilla, look there, a great add-on - go kill it :(

earthlng wrote:

but we do recommend to use an extension as our first and "most recommended" option. But neither me nor Pants are using one. I feel like a preacher who preaches all day long not to drink alcohol, only to go back home in the evening and get completely wasted.

publicarray wrote:

I don't need the extension but I do like to have more control and that is exactly what the extension provides. It also leaks a lot less but with more initial breakage. I'm currently experimenting anyway I might uninstall it if I find it too cumbersome.

publicarray wrote:

Actually It looks like someone has already worked on making it a WebExtention: https://github.com/meh/smart-referer/pull/48

---

@Gitoffthelawn wrote:
Can anyone recommend a flexible referer (aka referrer) extension that is e10s-compatible?

The most flexible referer extension is Referrer Control available at: https://github.com/muzuiget/referrer_control

• Not updated since early 2015
• Unresolved bugs
• Not e10s-compatible
• Not WebExtension

Next in line is RefControl, available at https://addons.mozilla.org/firefox/addon/refcontrol/

• Very popular, but...
• Not updated since 2014
• Not e10s-compatible
• Not WebExtension

Then there is Smart Referer, available at https://github.com/meh/smart-referer

• Not updated since 2013
• Not currently marked as e10s-compatible
• Not WebExtension
• Requires a separate extension (that is currently broken and is not e10s-compatible) to add a toggle button

Has anyone found any e10s-compatible extensions that let you control whether or not the referer is sent on a per-domain basis? IOW, allow referers to be sent from sitefoo.com, but not from sitebar.com? As a bonus, also allow referers to be sent to siteabc.com, but not to sitedef.com? (Referrer Control can do all this, but due to the issues mentioned above, it's not currently very usable.)

---

@Gitoffthelawn wrote
> "jeriko5" I can be wrong but Umatrix has http referrer and many other useful functions.

Thanks for pointing that out. uMatrix does not allow referer control directly within it's matrix (which is really too bad), so I didn't think it handled referers at all.

But you're right, it does have some handling for referers. I need to dive into its source code to see what it is actually doing, but it might be a possibility.

uMatrix offers a rule to spoof referers, but the UI does not currently have a way to control the data used for the spoof, nor the ability to set it to null. Once I read the source code, I'll have a better understanding of what it can really do.

Unfortunately, compared to tools like RefControl and Referrer Control, using uMatrix to quickly turn off referer spoofing involves more steps, and is much less obvious. With those other tools, the user can just click a toolbar button, and instantly all referers are unmodified from the defaults specified by Firefox prefs. This is real important when a site is not working properly, and the user needs it to work.

That said, I really appreciate your suggestion. uMatrix is e10s-compatible. The Firefox version doesn't currently use the WebExtension API, but I believe that is planned.

TL;DR uMatrix is a possibility, but its usability and flexibility for controlling referers is not up to par with the tools mentioned above (Referrer Control, RefControl, Smart Referer). Bottom line: I really appreciate the suggestion from @jeriko5 and definitely welcome additional suggestions for an e10s-compatible extension to control referers.

Update on uMatrix and referers: I performed some testing, and from what I could initially tell (and I prefer to be mistaken), uMatrix does referer blocking only in one direction. It seems to only block referers to sites, but not from sites. Please correct me if I am mistaken

[Thorin-Oakenpants]

@publicarray wrote:

Gitoffthelawn what is the problem with Smart Referer? There is code for the web-extension (not published on AMO I think) and even the one that is published on AMO functions fine. BTW the Last commit on github was on Jan 10, 2017.

---

@Gitoffthelawn wrote:

publicarray Thank you for posting https://github.com/meh/smart-referer/pull/52

Of all the referer extensions, I'm the least familiar with Smart Referer. But you may be able to help.

From what I can tell, outside of 3 small commits, it has not been updated since 2013. But since 2013, Firefox, and how it handles referers (and "secure" referers) has had significant changes. Is Smart Referer fully functional and compatible when used with the latest Firefox stable release? Is it compatible with all the new referer prefs that Mozilla has added to Firefox, or does it require them to be set a certain way?

Do you know where the e10s-compatible version of Smart Referer is located? Has anyone tested it to be e10s-compatible? https://www.arewee10syet.com/ lists Smart Referer as "unknown".

Where is the source code for the WebExtensions version?

Is there an e10s-compatible way to add a button to toggle Smart Referer on and off? BTW, if you do fork it, it would be wonderful to have that functionality included.

---

@publicharray wrote:

@Gitoffthelawn Thanks, I'll try my best to answer all of your questions :)

From what I can tell, outside of 3 small commits

Yes, I agree the extension could use some love.

Firefox, and how it handles referers (and "secure" referers) has had significant changes.

Yes, but the extension does not use the FF prefs (white-listing would not be possible otherwise) instead it modifies the header directly (the web extension uses browser.webRequest.onBeforeSendHeaders I would assume the legacy extension does it in a similar fashion)

Is Smart Referer fully functional and compatible when used with the latest Firefox stable release?

I use FF nightly and yes no problems so far so I assume it works in stable.

Is it compatible with all the new referer prefs that Mozilla has added to Firefox?

Not sure how this works. I assume the extension has higher priority so your prefs don't matter (I will investigate tomorrow if this is the case)

Do you know where the e10s-compatible version of Smart Referer is located? Has anyone tested it to be e10s-compatible? https://www.arewee10syet.com/ lists Smart Referer as "unknown".

As you can see in the pull request all it takes is to flip the switch so yes the code seems compatible but at the moment it is still using shims: https://developer.mozilla.org/en-US/Add-ons/Install_Manifests#multiprocessCompatible

It looks like arewee10syet.com just lists the value of multiprocessCompatible.

Where is the source code for the WebExtensions version?

https://github.com/meh/smart-referer/tree/wext/webextension

Is there an e10s-compatible way to add a button to toggle Smart Referer on and off?

The web extension has the button by default. I hope it will land on AMO soon :tada:

Sorry for the brief answers but here in [mod edit] it is close to midnight and I need some rest. I'll look into it further tomorrow.

[ghost]

Pardon me for being the joker who must have missed something but I don't understand the pertinence of referrer dedicated tools for Firefox, be they add-ons or scripts.

Where am I mistaking when I think and write that network.http.referer.XOriginPolicy when set to _1_ allows sites to send their referer if base domains match (and even to _2_ if hosts match is required, though more problematic often than _1_), and as such is all a user needs? Unless to consider that even between 2 sites of a same domain the user could be trapped? But trapped in what way? Different domains need not to know that they have been accessed by a given user, but within the same domain, where is the problem?

Obviously there are several points I miss in the argumentation which credits the very add-ons/script mentioned above, the associated work still active. My point includes the idea that granularity control is maybe excessive given the per-site problems it creates compared to the actual benefits.

[Atavic]

@zymase See: http://www.ghacks.net/2015/01/22/improve-online-privacy-by-controlling-referrer-information/

@Thorin-Oakenpants https://github.com/ghacksuserjs/ghacks-user.js/issues/12#issuecomment-291003070

[ghost]

@Atavic I had read the article you mention but understanding the different built-in Firefox settings to manage http headers is not my problem; what is my concern is to wonder if these settings, considering they seem to handle all possible schemes, are not sufficient by themselves to avoid brainstorming over what would appear as excessive per-site granular control regarding these http headers.

Not that I dislike brainstorming, on the contrary! But is it really necessary when it comes to http headers' management in the Firefox browser?

[Atavic]

Yes. You'll encounter some sites giving you breakages due to the fact that they pretend more referral info for them to work, as Soundcloud or Twitter.
I think uMatrix is a good option: https://github.com/gorhill/uMatrix/wiki/Per-scope-switches#referrer-spoofing

the referrer information will now be foiled using the root URL derived from the URL of the request.

https://github.com/ghacksuserjs/ghacks-user.js/issues/5#issuecomment-280870963

[ghost]

So. That's all I was wondering of, @Atavic . Thanks for the info.

I could re-install another FF add-on I've used in the past, not mentioned here above, which handles only the network.http.sendRefererHeader setting with values 0-1-2 and has the advantage of allowing a 10seconds permission for sites requiring value=1or2 before resetting to 0
_Change Referer Button_ at https://addons.mozilla.org/en-US/firefox/addon/change-referer-button/

Or, mentioned above, with a per-site handling, the _RefControl_ FF add-on
Or use _uMatrix_ when up to now _uBlockO_ makes it fine for me.

What triggers my lack of enthusiasm is the pertinence, but reading what you above link to might make me change my mind.

[Atavic]

That one and https://addons.mozilla.org/en-US/firefox/addon/no-referer/ worked fine.

[ghost]

I've never tried this _No-Referer_ FF add-on for the simple reason I hadn't heard about it.
It is mentioned that it is not available for Firefox 52 : if the add-ons mentioned above are seniors, _No-Referer_ is a grand-pa : last updated in 2008!

[Atavic]

Right click and open with no referer... T'was great.

[Gitoffthelawn]

@publicarray Thank you for all your answers. Their brevity was not a problem at all! I look forward to reading your follow-up whenever you have the time.

[Thorin-Oakenpants]

alexander255 wrote:

@Gitofthelwan:

From what I can tell, outside of 3 small commits, it has not been updated since 2013.

You were looking in the wrong place :wink:. A rewrite is happening here: https://github.com/meh/smart-referer/tree/wext

But since 2013, Firefox, and how it handles referers (and "secure" referers) has had significant changes.

And none of these changes were actually relevant for Smart Referer…

Please remember that Smart Referer is a very small and intentionally simple add-on. It works, it uses stable APIs there simply wasn't any reason for any huge changes, like… ever.

By far the majority of the maintenance time was also actually spent on the whitelist. Regular commits coming up…

Is Smart Referer fully functional and compatible when used with the latest Firefox stable release?

Yes.

Is it compatible with all the new referer prefs that Mozilla has added to Firefox, or does it require them to be set a certain way?

No idea. I suppose those prefs are applied before the data is passed to the internal network layer? In that case the new Referer-values they set will what Smart Referer considers to be The Truth.

Do you know where the e10s-compatible version of Smart Referer is located? Has anyone tested it to be e10s-compatible? https://www.arewee10syet.com/ lists Smart Referer as "unknown".

On AMO, it NEVER WAS NOT e10s compatible. Just nobody bothered to push a new version that set the flag accordingly, yet.

Where is the source code for the WebExtensions version?

Here: https://github.com/meh/smart-referer/tree/wext
Discussion thread: https://github.com/meh/smart-referer/pull/48

Is there an e10s-compatible way to add a button to toggle Smart Referer on and off? BTW, if you do fork it, it would be wonderful to have that functionality included.

The WExt version includes a button…

Full disclosure: I'm Smart Referer's main contributor and effectively it's only developer since 2015.

I hope that clears some things up for you!

[Gitoffthelawn]

@alexander255 Thank you! We look forward to the WebExtensions version too!

[publicarray]

Q:

Is it compatible with all the new referer prefs that Mozilla has added to Firefox, or does it require them to be set a certain way?

A:
I just investigated the behaviour of FF prefs by adding console.logs() to the extension. In short to the FF prefs take precedence.

Default prefs:

referer as received by request.requestHeaders: "http://www.onlinehtmleditor.net/?d=asd"
Rejecting referer "http://www.onlinehtmleditor.net/?d=asd" for "https://www.whatismyreferer.com/"  
new referer:

set network.http.referer.trimmingPolicy to 2:

referer as received by request.requestHeaders: "http://www.onlinehtmleditor.net/"
Rejecting referer "http://www.onlinehtmleditor.net/" for "https://www.whatismyreferer.com/"
new referer:

set network.http.sendRefererHeader to 0:

<no output as expected>

This applies all extensions that read the referer header from the API. Note: extensions still have the capability to do whatever they want with this information though.

So if you want the extension to allow referer headers to certain sites than the FF preferences will have to allow this.

I suppose those prefs are applied before the data is passed to the internal network layer?

yes

---

Also note that the Smart Referer does not modify same origin requests (similar to network.http.referer.XOriginPolicy 1 or 2). In the future adding an option to block same origin referers would be a nice addition for advanced users.

[Thorin-Oakenpants]

look at this

• Smart Referer :pushpin:
  
  
  
  • Will be disabled by FF53 (multiprocessCompatible flag is missing)
  
  
  • WebExtension under development
  
  

I guess someone better get a move on and fix that missing flag

[publicarray]

@Thorin-Oakenpants Thanks. Yes I hope @meh finds the time :)

[earthlng]

FYI: addons already on AMO can still be updated after the deadline with FF53. They will be only temporarily disabled until a new version with the flag is released by the addon devs.

[Thorin-Oakenpants]

It's 6 months since the last comment :) Smart Referer on AMO still shows a legacy 0.2.2 version. When and if it turns up, feel free to start a new topic or just let us know in the extensions sticky

[publicarray]

@Thorin-Oakenpants the developers pushed it to AMO about the same time you commented here. See: https://github.com/meh/smart-referer/pull/61 With the automated reviews I think it will only take a few days to land on AMO.

[meh]

Yeah, that wasn't a coincidence, I saw the notification at a time where I had nothing to do, and remembered that pull request was waiting for early October, and early October it was 🐼

[publicarray]

thanks @meh 😍

@Thorin-Oakenpants The (web) extension is now live on AMO.

[Thorin-Oakenpants]

Thanks @meh ... Sweet .. will move it up from Legacy in the wiki

Sidenote: CanvasBlocker became WE on AMO about 12 hrs ago and uMatrix a couple of days ago .... we're getting there (not sure where there is but too late to get off the train)

[Thorin-Oakenpants]

@meh .. still no privacy policy :-( just a simple one on AMO like this goes a long long way with users :)

[vertigo220]

I'm trying to make sense of these referer add-ons and hope someone can provide some insight. As has been mentioned, uMatrix eliminates tracking by spoofing 3rd-party referers, and I haven't had any issues due to this (that I know of) in the ~2-3 years I've been using it. And there's the fact, as was also mentioned, that referers can be handled through about:config. So I'm struggling to understand the point of separate add-ons like RefControl, Smart Referer, etc. What additional benefit, if any, do these provide? It seems there's no need for them and continuing to just use uMatrix for this should be sufficient, but I want to be sure I'm not missing something.

from what I could initially tell...uMatrix does referer blocking only in one direction. It seems to only block referers to sites, but not from sites.

First, what's meant by "from sites?" Are you saying that clicking a link on site A to go to site B will send info from site B back to site A? If so, it doesn't make sense how that would be an issue, since A already knows you're going to B. And did you ever confirm whether or not it is the case?

[crssi]

uM has limitations since you can control referer only by source and not by destination or preferably both.
See https://www.reddit.com/r/uMatrix/comments/86samo/referal_improvement_suggestion/
while with SmartReferer you can control both, see example here https://github.com/meh/smart-referer/issues/77#issuecomment-382174730.

I never used/tryed RefControl, so I can't comment on this one.

Cheers

[Thorin-Oakenpants]

and I haven't had any issues due to this (that I know of) in the ~2-3 years I've been using it

That's because it's a rather basic one (as crssi says). If you were blocking ALL tracking referrers (non 1st party) then you would be seeing lots of website breakage.

[vertigo220]

Thanks for the replies, though I'm still not understanding it. Why would you want to control it by source/destination? It seems the key is preventing site B from knowing you came from site A, and AFAICT uM does this. And you say if it were blocking all non-1st party referers it would break stuff, but isn't that what it's doing? My understanding is there's 1st-party (same domain) and 3rd-party (cross-domain), and uM blocks 3rd-party, which not only are the important ones, but is exactly what you say it would break stuff if it did. Clearly I'm missing something with all of this, and I'd love to know what, because I want to be sure I'm not less protected than I think.

[crssi]

@vertigo220 with all the respect, have you even look at the second link in my post?

[vertigo220]

Thanks. I thought I did, but I guess I didn't drill down deep enough. Looking at the ghacks-user.js issue #373 link, if I'm understanding it and the overall subject correctly, blocking 3rd-party referers, which uM does, does in fact provide full tracking protection, but it can break things that require what are essentially reverse referers, such as embedded Vimeo videos (apparently Vimeo won't allow videos to play if the player that's embedded on another site doesn't receive the referer indicating it's coming from Vimeo). I can't recall ever seeing an embedded Vimeo video, and I don't think I've used the main site in probably at least 2-3 years, so it explains why I haven't noticed it, but I can see that such cases do necessitate the ability to whitelist certain referers, and therefore why something more than uM is needed for those fringe cases. Am I understanding that all correctly? So it seems I should install Smart Referer, add a whitelist for referers _from_ Vimeo _to_ other sites, and disable uM's referer protection, correct?

[crssi]

If you don't have a breakages then you should stick to uM, there is no need to overbloat your profile with extensions that you do not need.
If you do, then go for SR. You do not need to white-list vimeo in SR since its already white-listed... the second link, the one you also failed to read the second time, its exactly about that... but in the same time its also a nice example of difference between uM and SR.
There is no reverse referers... it is saying any source site going to player.vimeo.com as destination should preserve referals.

[Thorin-Oakenpants]

First you need a definition of exactly what uMatrix does

... the referer information will now be foiled using the root URL derived from the URL of the request. For example, if the URL of a request is http://www.example.com/blahblahblah/boring.html and the referer is http://google.com, the referer will be spoofed using the http://www.example.com/ string.

[vertigo220]

I'm sorry that my lack of understanding what was happening in that thread due to the fact I've just started learning about all of this gives the impression that I failed to read it. Now that I have a better understanding of it, rereading that link I see that it was added to the built-in whitelist, though I'm not sure why that was necessary if you can Anyways, despite the fact I haven't (to my knowledge) experienced any breakage from using uM for this, I'd rather ensure there's as little chance of problems as possible while still maintaining adequate protection, both for my own sake and because I want to be able to set others up and their browsing habits and ability to solve said issues are different from my own. So based on what I've learned here, it seems SR is a better option. And hopefully others with the same question will be find the answer here.

One other question: I'm wondering if there would be any benefit to spoofing the referer to always look like I'm coming from duckduckgo (or another, similar site). Their privacy policy states they make revenue when people link from them to sites like Amazon and eBay and make a purchase, so I thought maybe I, and others that are interested, could make it always look like I'm coming from DDG. I'm just not sure if it would work, or how to do it. Looking at their links, I don't see any obvious referer info.

[vertigo220]

@Thorin-Oakenpants Yeah, I'm aware that's what it does (one of the many things I've learned in researching this over the past couple days), but I don't know enough about how referers work to get much more from that info than to know that linking to example.com from a google search will make example.com think I linked from them, not google. But I didn't realize that could/would break anything. It doesn't seem like it would, but apparently there are rare (or not so rare?) cases that it does. My end goal is just to keep site A and site B from sharing info and building a bigger, more accurate picture of me. I don't necessarily mind them knowing how I got there, since it can help them know how to better reach their audience/customers, but unfortunately tracking and profiling has gotten so out of hand that I'd rather put myself first and do what I need to in order to protect myself and others, which is why I'm trying to learn about this stuff and a lot more.

[Thorin-Oakenpants]

As I type this up and am almost finished, you posted again (edit: and another post!! slow down :smile: ). I too am learning - I'm not very knowledgeable on referers under the hood, always relied on just what I could do via extensions such as RefControl (legacy) etc and how that affected site breakage... but I see as such

1. You click a link from source.com to open a page on destination.com and the referer tells destination where you came from
2. You load source.com which has 3rd party content from destination.com (such as those embedded vimeo videos) and each image/video/js/whatever request could have a referer telling those 3rd parties which website you are on.
   
   
   
   • some web servers block parts of their website to web browsers that do not send the right referrer information, in an attempt to prevent deep linking or unauthorised use of images (bandwidth theft). ^wiki
   
   
   • this is your breakage AFAIK
   
   
3. Not sure about this reverse referer you refer to: If I understand it:
   
   
   
   • source.com requests vimeo.com and says Hi I'm SOURCE
   
   
   • vimeo says sure, here's the video for SOURCE
   
   
   • source loads the vimeo video only because it was told it was for SOURCE
   
   
   • ^^ I guess this is controlled by Vimeo themselves with their free widget or whatever, and now I'm confused, but I totally get that this can be done, right?
   
   

The other thing is what you can do with a referer

• strip it out completely
• fake it
• use the destination
  
  
  
  • eg if you request a video from vimeo, the referer sent to vimeo is vimeo
  
  
  • if you click on a link to goats.com from llama.com
  
  
• use the source

^^ I know that I used to just block all referers by default. That sucked a bit (was a couple of years ago now), so I switched to faking. Can't remember now. But if you truly tried to hide the source (i.e the page you came from or were trying to load 3rd party content into)

PS: I probably have source and destination back to front
PPS: google tracking in search results is not a good example - they use utms

I'm wondering if there would be any benefit to spoofing the referer to always look like I'm coming from duckduckgo (or another, similar site)

That's what I did. I NEVER ever wanted to give away the site I was on (or the site I came from). All I know is with uM I never get breakage, but with RefControl (back in the day), I got breakage because I blocked or faked, and in order for some sites to work I had to give the correct info - cogito ergo sum - uM is maybe not as robust as eg SR (which is the same as RefControl - fine grained, source/destination rules etc). IDK. :confused:

[Thorin-Oakenpants]

If you are setting up others - use uM (you can still toggle the referer option per scope, its the menu dropdown in the pic below). Its more of a set and forget. Although I am confused now as well - if uM always says the request (eg vimeo), then it's never leaking the page asking for it (I really need to get source and destination terms correct). So I'm confused. Maybe I never see breakage because I have so many things screwed down, I do not notice it

edit: forgot the pic

If you use SR I guess it offers more options. My only concern is the built-in whitelist, the one that contains vimeo. Can this be turned off, otherwise I am at the whims of someone else's list. i,e when a 3rd party request is made, I WANT it to FAIL if it gives me away because 1. it protects me and 2 it alerts me to a problem that I could maybe work around (eg fake the referer)

[crssi]

@Thorin-Oakenpants: the SR white-list is not definite, you can simply opt-out.
Even better, you can add your own white-list source which you can share with others. ;)
See https://github.com/meh/smart-referer/issues/78

[Thorin-Oakenpants]

cheers. I know you can whitelist etc your own sh^t - otherwise what is the point of granular control. My only question was the built-in whitelist - so good to know.

I'm in the same boar as @vertigo220 and seriously wondering if I need SR. uM works for me.

@vertigo220 : SR is more granular because it has source-destination level rules:
In uM it is only the scope. Imagine Site A pulling in content from Sites B, C, and D (and sending referers to those three sites). If you block the referers in UM for scope Site A and you have breakage on the Site C content, you would have to allow referers for ALL of them where in SR you could only have to allow the Site C ones

[vertigo220]

Glad I could confuse you. At least now I'm not the only one 😉

I don't even know anymore which way I was thinking of it with the Vimeo video. But I _think_ (though clearly I'm no expert on this) that the site the video is embedded on is the source and it's requesting the video from vimeo.com, which is the destination, and I'm guessing that if vimeo.com gets a referral from itself to play an embedded video, it cries foul play. Perhaps @crssi can inform if I've got that right or if I'm totally backwards. And I think the issue with uM per crssi is that it can whitelist one way but not the other, so maybe it can whitelist vimeo.com so it can send its referer to other sites, but not so other sites can send their referer to it, which is (I think) what's necessary (and I see you just posted something along these lines--slow down yourself!! 😛 )

As for SR's whitelist, I figured it's scrutinized by users and hopefully at least mostly good. I guess it's a matter of using uM and getting rare breakage or using SR and possibly having leakage every now and then. Maybe uM for me and SR for my parents would be a reasonable strategy. The biggest problem with uM here is not the breakage, but that it's not even readily apparent. If I had gone to that site without knowing about the video, I never would have realized it was there.

[crssi]

The uM biggest problem is that you can "white-list" only source.
So when you do that on particular site, you will white-list referers to any destinations from this site.

[vertigo220]

On a side-note, but I think still related, I've also been looking at add-ons such as clearurls, neat url, and dont track me google, which all appear to do the same thing of cleaning links of their garbage. Interestingly, I'm not even sure if I need that, since apparently NoScript does it as well. In fact, I had to disable all my privacy addons and settings before finally getting to NS (I really didn't think it was responsible, plus I've stopped keeping it on my add-on bar, so I didn't do it until last) and realizing it was the one cleaning them up. But I can't for the life of me figure out what setting in it is responsible, and I'm in the process of searching to see if I can find anything about whether the others are needed if using NS or not.

[Thorin-Oakenpants]

Someone put me out of my misery

I am on SITE A which loads content from SITE B (and sends site b a referer to say "Sup dude, I'm currently at site a)

Which site is SOURCE and which site is DESTINATION

[Thorin-Oakenpants]

As for SR's whitelist, I figured it's scrutinized by users and hopefully at least mostly good

It's not about the list existing (and I trust the maintainers of it). The list will reduce breakage. FOR ME its that I want the request to fail so I KNOW about it and can make my own decisions. As crssi said, the built-in whitelist it can be disabled (this is separate from your own whitelist etc).

[vertigo220]

Haha! I think I've got it. As crssi said, you can whitelist vimeo.com, so I went there and told uM not to spoof it. But the video still doesn't work, unsurprisingly, because (again, I'm not 100% sure about this) I've told uM to let vimeo.com _send_ a referer, but I can't whitelist it to allow any site to send one _to_ it. So in this case, vimeo.com appears to be the destination, and the host site (where the video is embedded), is the source, which sends a referer, but that referer is spoofed, causing vimeo to break. Interestingly, allowing the player in uBo but not in uM gives me the ability to click a link to play the video on vimeo, but even then it doesn't work. So vimeo won't even play a video on _its own site_ if the referer isn't right. Crazy.

[vertigo220]

So it seems like the requests to be made for each add-on would be:

SR: Option to notify user when whitelisted item is activated
uM: Option to whitelist domain for referers being sent TO the domain in addition to the current ability to whitelist referers coming FROM it / Edit: AND option to notify when activated

Does that sound right?

[crssi]

@Thorin-Oakenpants SITE A is a source domain.
@vertigo220: going to vimeo and white-list it in uM wont work, since you are telling the uM to preserve referals on from when navigating (or getting any other info) from vimeo to any other site.
But in this case I would need from any site as a source to vimeo as destination (this cannot be accomplish in uM).

[vertigo220]

Right. So my understanding was correct. Sounds like if uM could be updated with the features I stated, it could replace SR and similar add-ons, making one less needed. Otherwise, if SR is updated to provide notifications, it would meet Pants' requirements, and would work in place of uM's spoofing without concern about possible unknown privacy leaks.

[Thorin-Oakenpants]

AFAIK, SR already meets all my requirements. The question is do I need it - my uM is set to referrer-spoof: * true and I have no per scope overrides, and I am yet to see breakage (this is due to my setup and the fact I don't piss around with things like vimeo or video or twitter sh^t etc - most 3rd party is blocked anyway).

As I already said (see my example about sites a, b,c,d), and then crssi about 16 mins after me ["The uM biggest problem is that you can "white-list" only source."], is that SR allows source, destination, and source-destination control. uM only offers a per domain (scope) setting.

Without scrolling up, someone (crssi?) already suggested a uM type interface for referers which I thought would be cool (i.e three columns, red/green/grey = source/something/destination). Maybe someone can link to that, I can't be arsed looking for it.

[vertigo220]

Well, SR meets your requirements if you disable the built-in whitelist. I was thinking more along the lines of you being able to use it to reduce breakage if they implemented a notification feature to warn you when something in the whitelist was making a request.

As for uM, that's why I said it would be nice if gorhill could add the functionality to whitelist either way. I agree that adding a column would be a good way to do it, since it would make it readily available, could work like the columns in uBo (pretty much as you described), and make the feature more visible (I didn't even realize you could whitelist that until crssi pointed it out, though, again, I only just started looking into referer stuff). I'll play around with SR and probably submit those as feature requests for both add-ons later. Right now I'm trying to unbreak Reddit, which seems to require allowing some trackers in order to view some threads...

[Thorin-Oakenpants]

re: reddit

in uBO i'm only allowing reddit domains thru
in uM it is like this

note: I am using old reddit because until the new design is fully rolled out, it's rather random and often pages won't load until I refresh at least one - edit: I use this https://addons.mozilla.org/firefox/addon/old-reddit-redirect/

I also have a bunch of uBO filters

www.reddit.com##.commentsignupbar.infobar
www.reddit.com##.listingsignupbar.infobar
www.reddit.com##.newsletter-container.newsletterbar.infobar
www.reddit.com###block-homepage
old.reddit.com##.commentsignupbar.infobar
old.reddit.com##.listingsignupbar.infobar
old.reddit.com##.newsletter-container.newsletterbar.infobar
old.reddit.com###block-homepage
reddit.com##.arrow

d.reddit.com
reddit.com/api/register
reddit.com/api/comment
reddit.com/api/friend
reddit.com/api/submit
reddit.com/api/
reddit.com/errors
oops.redditmedia.com
reddit.com/api/$xmlhttprequest,domain=reddit.com
reddit.com/web/$xmlhttprequest,domain=reddit.com
reddit.com$xmlhttprequest,domain=reddit.com
https://www.redditstatic.com/reddit.en.Y64Sg2dUcbw.js

That last lot from here -> https://old.reddit.com/r/privacy/comments/8fjpse/redditcom_posts_obfuscated_data_to_its_root_domain/

[Thorin-Oakenpants]

^^ Note: I do not log in, I have never had an account, and neither do I allow any reddit cookies (and thus no local storage or IDB and no service workers etc)

[vertigo220]

Still working on the Reddit issue, but it doesn't seem related to trackers. Even though it randomly worked then didn't work then worked again and so on when I enabled or disabled uBo or certain domains within it, I've now narrowed down the problem to a specific setting in Reddit Enhancement Suite (which took me the past couple hours, ugh). Unfortunately, this is in Waterfox, and in Firefox with a mostly fresh profile the issue doesn't exist, so now I have to figure out what's conflicting to cause it. So much fun...

[Thorin-Oakenpants]

A domain can track an individual user across itself if it wanted to (assuming some JS, maybe same IP for a session, or maybe all they need is the SSL session ticket id which I think lasts for 3 days), and it can def do it if you're logged in on an account. You account can of course be not YOU - eg throw away email, one off unique name/avatar (OpSec, don't reuse stuff). If these metrics help them design the site better or make better in-site suggestions, that's cool.

But what we can't control is data sharing behind the scenes. There are a number of massive data brokers in the world, who buy and sell this stuff. With the FPjs2, used across enough sites and shared, then it's feasible to connect the dots and build a history - even if that user is a "shadow profile". At some point, and it will happen, there will be ONE single failure (where your real ID is connected to the profile) and that's it, game over.

Anyway, by blocking that stupid FP js at reddit, I'm giving them the finger that I disapprove. But ultimately, blocking 1st party FP'ing is a game of whack-a-mole (i.e on sites you need to allow JS) - they'll just obfuscate the file names or make the site totally break unless you allow it.

Anyway, as soon as I get time, I will write up the referer bit in OP on #350