Https-everywhere: Cultivate a list of HTTPS Everywhere downstream ruleset dependencies

[Hainish]

Today, Cloudflare announced that it will be using HTTPS Everywhere rulesets in its new Automatic HTTPS Rewrites, and Brave uses it in the various versions of its browser. We should cultivate a list of our downstream dependencies so that we're clear on who needs to be notified if we change ruleset formatting, or change the location of rulesets, or when we make a new release.

We may also want to create some kind of a ruleset API for these providers as a longer-term goal. The implementation is up to these projects themselves, of course, but standardizing the way the dependencies fetch rules is advisable, as it help reduce overhead later and minimize the possibility of fragmentation.

cc @diracdeltas @grittygrease

[J0WI]

Nice! I hope CloudFlare will also share their "internal list of HTTPS-enabled domains" with us. 👍

[Hainish]

I did see that and made a note to ping them about that.

[Hainish]

https://github.com/grindhold/libhttpseverywhere provides a C-bindable API for HTTPS Everywhere. Apparently Epiphany browser (Gnome) will source this. (cc @grindhold)

[jonathanKingston]

Splitting the rulesets away into it's own repo from the extension itself might help with this perhaps? It also would make finding issues and PR's easier of features vs ruleset changes.

[Hainish]

@jonathanKingston this is something we've considered before. It would involve creating a subrepo which HTTPS Everywhere sources, just as we do with translations. I think it would be good to move in this direction. No timeline is set for this, though.

[Hainish]

The Tenta browser will be another downstream dependency. @jwriteclub is the relevant contact there.

[Bisaloo]

@Hainish, following this comment, do you know if duckduckgo is still using HTTPS Everywhere ruleset to rewrite URLs, as reported in this EFF blog post?

[Hainish]

Good find @Bisaloo. I'm not sure, I'll reach out to them and see.

[Hainish]

I've sent an email and will update this when I get the word back.

[Hainish]

DDG says: "Thanks for reaching out, Bill. We should be considered a downstream dependency, yes"

[jonathanKingston]

@Hainish looking quickly at the repo I can't see anything preventing a separate repo as you mentioned back in October.

Is it worth creating a separate repo of just the rules or was the plan bigger than that?

I'm happy to help review the existing bugs for rules and move them over to a new repo.

Do we know if these downstreams are parsing and lifting the format in anyway?

[Hainish]

Currently I'm in the process of decoupling the rulesets download and the extension download. This will land in the next month or two. I'm wary of moving rulesets into a a separate repo until this is completed.

[Bisaloo]

Not sure if you count that as a dependency but GNU IceCat includes (legacy) HTTPS-Everywhere by default.

I don't know what their plan is regarding the deprecation of XUL/XPCOM.

[Hainish]

Thanks, yes this is using an old version of HTTPS Everywhere. I'll reach out to them to get them to upgrade.

[diracdeltas]

FWIW Brave is using 'chrome/content/rulesets.json' from the XPI releases on eff.org. The ruleset format in the webextension releases is different, so we are stuck on the last pre-webextension release for now (5.21).

[Bisaloo]

https://github.com/EFForg/https-everywhere/issues/13606

PureBrowser, a fork from Firefox, ships HTTPS-Everywhere as well but apparently still uses a one year old version.

[Bisaloo]

@Hainish, what about creating a wiki on Github tracking all these dependencies as well as the current version of HTTPS Everywhere?

[Bisaloo]

Tenta browser now includes a modified version of HTTPS Everywhere:

https://www.eff.org/deeplinks/2017/11/tenta-browser-now-includes-https-everywhere

https://github.com/tenta-browser/go-https-everywhere

[Hainish]

@Hainish, what about creating a wiki on Github tracking all these dependencies as well as the current version of HTTPS Everywhere?

@Bisaloo Sounds like a good idea! Would you like to create one, or should I?

[Hainish]

@Bisaloo https://github.com/EFForg/https-everywhere/wiki is now enabled.

[Bisaloo]

https://github.com/EFForg/https-everywhere/wiki/List-of-downstream-dependencies

Great! Thank you @Hainish!

I started a page but it's still missing some info and I am not sure about the format. Feel free to update it.

[Hainish]

@Bisaloo great, thanks. I just updated it to change the "Last check for versions" date format to YYYY.[M]M.[D]D and add a link back to this issue. I think this wiki page should be the canonical list of dependencies, though, so I'm going to close this issue.