Brave-browser: Change Linux signing key via keyring package

[diracdeltas]

details in https://github.com/brave/devops/issues/361

Note that this involves:

1. Rotating the key (details in issue above)
2. Updating the linux install documentation to show the new key

QA steps (only needs to be done on Linux):

1. For each linux platform, download the key that is linked in https://brave-browser.readthedocs.io/en/latest/installing-brave.html#linux in the Release Channel section (for instance https://brave-browser-apt-release.s3.brave.com/brave-core.asc)
2. Import the downloaded key into gpg: gpg --import /path/to/downloaded/file
3. Run the command gpg --list-keys $KEY_ID where KEY_ID is the ID of the key that you just imported. This should be displayed in the terminal output from step 2.
4. There should be no keys that show rsa2048, only rsa4096.

[kjozwiak]

@diracdeltas @tomlowenthal labelling this as QA/No and release-notes/exclude. Please let me know if you disagree. If there's something that QA can do here, please remove QA/No, add QA/Yes and include some STR/Test Cases.

[diracdeltas]

@kjozwiak i have added QA steps for linux. for release notes, can just say Updated Release Channel signing key for Linux

[kjozwiak]

@diracdeltas thanks for the STR 👍 Does this issue require any more work or can we close this off?

For each linux platform, download the key that is linked in https://brave-browser.readthedocs.io/en/latest/installing-brave.html#linux in the Release Channel section (for instance https://brave-browser-apt-release.s3.brave.com/brave-core.asc)

By platforms, do you mean checking it against several of the popular distro's? Ubuntu, Mint, Debian, Fedora? Do you have any other recommended distro's that we should check this against?

[diracdeltas]

@kjozwiak i think we should close this when @bkero deploys packages using the new keys and the docs have been updated (unless there is a separate issue open for the doc update)

[tomlowenthal]

Marking this as no longer blocking release. As discussed in the big chat we just had, the Friday release will use the current signing key, but the instructions will tell people to install the Brave keyring package. Initially, the keyring package will be empty. In a future revision, the keyring package will add the new signing key to the repo, after more testing to make sure everything goes right.

[diracdeltas]

@tomlowenthal in light of that, could you make a new issue for empty linux keyring + add it to the docs?

[diracdeltas]

^ Referring to https://bravesoftware.slack.com/archives/C8MP8ME4C/p1541617859061500?thread_ts=1541617757.061100&cid=C8MP8ME4C, let me know if you disagree that this is a next-release issue.

[diracdeltas]

@tomlowenthal nvm i just did it. https://github.com/brave/brave-browser/issues/2037

[Mikaela]

I just started getting apt errors about the key changing. Was this announced anywhere or how were users supposed to know that the key change has been intented? I didn't see anything in this repository with search NO_PUBKEY nor at Brave community and thus think that this wasn't communicated to users as well as it could have.

W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://brave-browser-apt-dev.s3.brave.com buster InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 4FE13824E3FFC656     
W: Failed to fetch https://brave-browser-apt-dev.s3.brave.com/dists/buster/InRelease  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 4FE13824E3FFC656

I fixed this by following the readthedocs again, but would probably simplify the instruction into sudo apt-key adv --recv-keys D8BAD4DE7EE17AF52A834B2D0BB75829C2D4E821, but maybe it's better for the public key to be on your server. After apt update there was an update to brave-browser-dev, but I was unable to find brave-keyring package.

[diracdeltas]

@Mikaela we haven't changed the key yet; the expected one for dev releases is still https://brave-browser-apt-dev.s3.brave.com/brave-core-nightly.asc. actually we are not planning on changing the key at all for dev/beta, only for stable.

@bkero does that error look familiar to you?

[bkero]

@Mikaela Hi there. Thank you for reporting this. Yesterday when the 0.58.9 dev build was released it was signed with the wrong key, as you've discovered. I just resigned the packages and put them up for you to use. Please report back if you experience any issues.

[bsclifton]

Moving to 0.58.x after discussion with @bkero and @diracdeltas 😄

[Mikaela]

Hi, I have started seeing this on my family PC (which they probably haven't booted to Debian during the two weeks I was at my home):

Err:10 https://brave-browser-apt-release.s3.brave.com buster InRelease         
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0B31DBA06A8A26F9
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://brave-browser-apt-release.s3.brave.com buster InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0B31DBA06A8A26F9
W: Failed to fetch https://brave-browser-apt-release.s3.brave.com/dists/buster/InRelease  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0B31DBA06A8A26F9
W: Some index files failed to download. They have been ignored, or old ones used instead.

Searching for the short KEYID, I found out that someone on Linuxquestions.org has also been surprised about this, but importing the key with the instructions from Readthedocs.io worked as linked there.

Surprisingly when I ran apt update and apt install brave-browser brave-keyring, I received brave-keyring is already the newest version (1.0-1)., so I think somtehing is wrong with it as it didn't install this new key.

[tomlowenthal]

@Mikaela That wasn't part of the key transition. We accidentally signed a general release with our nightly key. If you update from the repo again, you should get a valid signature. Sorry for the confusion!

[mossy-nw]

@tomlowenthal this sort of thing sketches me out esp. since your signing key and repo are hosted on the same (amazon s3) site. Would have appreciated some advance notice (on your twitter) of the key change, since a sudden unexplained signing key change would be consistent with a security compromise, correct?

Could you please post the correct signing key fingerprint to your twitter? And/or please just direct people to download your public key from your git repo instead?

[tomlowenthal]

@mossy-nw Again: this wasn't a deliberate key change. This was a human error. We signed a release package with the wrong Yubikey. It has been reverted. There was no compromise, and no risk of compromise at any time. The only problem was the apt warning folks saw — and rightly freaked out about.

You're right that the fingerprint should be posted on the install page. When make the switchover, we're going to use a brave-keyring package to manage the keys going forward, so it should be a pretty much silent update.

[mossy-nw]

@tomlowenthal okay thank you! Any chance you can post the fingerprint here in the meantime?

[tomlowenthal]

I've opened a PR to add them to the install instructions (#2462), and you can check out the fingerprints in my commit.

[mossy-nw]

@tomlowenthal hey, thank you very much!

[tomlowenthal]

The only code change in brave-browser will be updating the Linux install docs. The bulk of this work is devops-only.

[Mikaela]

Hi again, does this say anything to you?

└┌(#:~)┌- curl -s https://brave-browser-apt-release.s3.brave.com/brave-core.asc | sudo apt-key --keyring /etc/apt/trusted.gpg.d/brave-browser-release.gpg add -
OK

Err:15 tor+https://brave-browser-apt-release.s3.brave.com trusty InRelease
  The following signatures were invalid: EXPKEYSIG 4FE13824E3FFC656 Brave Software <[email protected]>

I managed to fix it by apt-key adv --keyserver keys.openpgp.org --refresh-keys, but I wouldn't recommend it and I think your instructions link to outdated public key. After the next apt update, there was an update for brave-keyring.

• https://brave-browser.readthedocs.io/en/latest/installing-brave.html#linux

Edit: to clarify this system has had Brave installed previously, but didn't have it for weeks/months and today I decided to install it again.

[fmarier]

Edit: to clarify this system has had Brave installed previously, but didn't have it for weeks/months and today I decided to install it again.

My guess as to what's happening is that you installed Brave prior to our changing the installation instructions and fixing the key update mechanism. If that's the key, there would have been an old key in /etc/apt/trusted.gpg on your machine (instead of the new location of /etc/apt/trusted.gpd.d/brave-browser-release.gpg). This will have been cleaned up automatically in the upgrade of brave-keyring and as long as you keep that package up-to-date, you shouldn't run into the same problems again.

[mbacchi]

Closing as we've released the brave-keyring package and rotated multiple signing keys/expirations since this was opened. If I missed something please let me know.