Https-everywhere: Release 2019.6.27 forthcoming

[Hainish]

Type: other

@Bisaloo @cschanaj @J0WI @pipboy96

[J0WI]

18112

[Hainish]

@J0WI I already had the files signed by the time I saw this. Let's run this before the next ruleset release.

[pipboy96]

@Hainish Are we close to the next full-fetch run?

[Hainish]

@pipboy96 I agree we should do this at some point soon.

[Hainish]

@Bisaloo @cschanaj @J0WI @pipboy96 I'll be off on vacation and largely unavailable except for emergencies the entirety of next week. If something urgent comes up, please contact [email protected] (@jgillula) and he'll get in touch with me. When I get back I can review any PR backlog.

[Hainish]

Release made!

[fmarier]

Looks like the ruleset at https://www.https-rulesets.org/v1/ was not updated for the 2019-06-27 release (see #17958.

[pipboy96]

@fmarier Thanks for pointing out!

[Hainish]

@fmarier this is expected. We want users to download and install the full extension and use the rulesets bundled with the extension when a new extension version is released, rather than downloading a new version of the rulesets separately. Thus ruleset releases are no longer made in conjunction with full extension releases.

[fmarier]

Thus ruleset releases are no longer made in conjunction with full extension releases.

@Hainish We're using the ruleset, but not the extension, in the Brave Browser. Does this mean we should point our update script to the .crx / .xpi (and unpack/extract the ruleset from it) instead of using https://www.https-rulesets.org/v1/?

[Hainish]

@fmarier Ah, interesting edge case (and good to know third parties are using https://www.https-rulesets.org/!)

I'm curious, in Brave's implementation are the rulesets verified with our signing key once downloaded?

I think we can resolve this by adding an additional piece of metadata in the ruleset releases to indicate that this corresponds with an extension release, which Brave can safely ignore.

[andresbase]

Hi @fmarier ! We're curious about the uses of rulesets for others. Would you mind chatting next week? You can email me on andres [at] eff dot org.
Thanks!

[fmarier]

I'm curious, in Brave's implementation are the rulesets verified with our signing key once downloaded?

Not currently. We rely on the HTTPS cert.

The whole process (not 100% happy with it, so suggestions welcome) looks like this:

1. I have a cronjob that pulls https://www.eff.org/files/Changelog.txt once a day looking for changes.
2. I run our update script locally, which pulls down https://www.https-rulesets.org/v1/latest-rulesets-timestamp.
3. From the latest timestamp, the script goes and fetches https://www.https-rulesets.org/v1/default.rulesets.${Number(timestamp)}.gz.
4. We parse the JSON and turn it into a leveldb database.
5. I test the end result manually.
6. If tests look good, I kick off a Jenkins job to go through steps 2-4 and update the file on our endpoint.

We are hoping to automate this at some point, so having dev signatures to check would be great.

[Hainish]

@fmarier ah, interesting. Keep in mind for each new ruleset release a new signature file is also generated, which can be verified: rulesets-signature.${Number(timestamp)}.sha256.

The key is embedded in the extension, but for convenience sake:

JWK:

{"kty":"RSA","e":"AQAB","n":"1cwvFQu3Kw-Pz8bcEFuV5zx0ZheDsc4Tva7Qv6BL90_sDLqCW79Y543nDkPtNVfFH_89pt2kSPp_IcS5XnYiw6zBQeFuILFw5JpvZt14K0s4e025Q9CXfhYKIBKT9PnqihwAacjMa6rQb7RTu7XxVvqxRb3b0vx2CR40LSlYZ8H_KpeaUwq2oz-fyrI6LFTeYvbO3ZuLKeK5xV1a32xeTVMFkIj3LxnQalxq-DRHfj7LRRoTnbRDW4uoDc8aVpLFliuO79jUKbobz4slpiWJ4wjKR_O6OK13HbZUiOSxi8Bms-UqBPOyzbMVpmA7lv_zWdaLu1IVlVXQyLVbbrqI6llRqfHdcJoEl-eC48AofuB-relQtjTEK_hyBf7sPwrbqAarjRjlyEx6Qy5gTXyxM9attfNAeupYR6jm8LKm6TFpfWkyDxUmj_f5pJMBWNTomV74f8iQ2M18_KWMUDCOf80tR0t21Q1iCWdvA3K_KJn05tTLyumlwwlQijMqRkYuao-CX9L3DJIaB3VPYPTSIPUr7oi16agsuamOyiOtlZiRpEvoNg2ksJMZtwnj5xhBQydkdhMW2ZpHDzcLuZlhJYZL_l3_7wuzRM7vpyA9obP92CpZRFJErGZmFxJC93I4U9-0B0wg-sbyMKGJ5j1BWTnibCklDXtWzXtuiz18EgE"}

PEM:

-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA1cwvFQu3Kw+Pz8bcEFuV
5zx0ZheDsc4Tva7Qv6BL90/sDLqCW79Y543nDkPtNVfFH/89pt2kSPp/IcS5XnYi
w6zBQeFuILFw5JpvZt14K0s4e025Q9CXfhYKIBKT9PnqihwAacjMa6rQb7RTu7Xx
VvqxRb3b0vx2CR40LSlYZ8H/KpeaUwq2oz+fyrI6LFTeYvbO3ZuLKeK5xV1a32xe
TVMFkIj3LxnQalxq+DRHfj7LRRoTnbRDW4uoDc8aVpLFliuO79jUKbobz4slpiWJ
4wjKR/O6OK13HbZUiOSxi8Bms+UqBPOyzbMVpmA7lv/zWdaLu1IVlVXQyLVbbrqI
6llRqfHdcJoEl+eC48AofuB+relQtjTEK/hyBf7sPwrbqAarjRjlyEx6Qy5gTXyx
M9attfNAeupYR6jm8LKm6TFpfWkyDxUmj/f5pJMBWNTomV74f8iQ2M18/KWMUDCO
f80tR0t21Q1iCWdvA3K/KJn05tTLyumlwwlQijMqRkYuao+CX9L3DJIaB3VPYPTS
IPUr7oi16agsuamOyiOtlZiRpEvoNg2ksJMZtwnj5xhBQydkdhMW2ZpHDzcLuZlh
JYZL/l3/7wuzRM7vpyA9obP92CpZRFJErGZmFxJC93I4U9+0B0wg+sbyMKGJ5j1B
WTnibCklDXtWzXtuiz18EgECAwEAAQ==
-----END PUBLIC KEY-----