This is a tracking issue for the upcoming release of SecureDrop 0.9.0 - tasks may get added or modified.
Feature freeze: August 22, 2018
String freeze: August 29, 2018
Pre-release announcement: August 29, 2018
Release date: September 5, 2018
_SecureDrop maintainers and testers:_ As you QA 0.9.0, please report back your testing results as comments on this ticket. File GitHub issues for any problems found, tag them "QA: Release", and associate them with the 0.9 milestone for tracking.
Test debian packages will be posted on https://apt-test.freedom.press signed with the test key. An Ansible playbook testing the upgrade path is here.
securedrop-keyring package (#3723)0.9.0~rc2 on test apt server - @emkll 0.3.3.9 is currently on apt-test)release/0.9 off develop 0.9.0~rc1 on test apt server - @emkll After each test, please update the QA matrix and post details for Basic Server Testing, Application Acceptance Testing and 0.9.0-specific testing below.
securedrop-keyring, linux-image, linux-firmware, and tor packages)develop develop in prep for 0.10.0 releaseFor both upgrades and fresh installs, here is a list of functionality that requires testing. You can use this for copy/pasting into your QA report. Feel free to edit this message to update the plan as appropriate.
./securedrop-admin setup successful./securedrop-admin sdconfig successful./securedrop-admin install successful./securedrop-admin tailsconfig successfulInstall on 0.8.0.
./qa-loader.py -m 25 in /var/www/securedrop (this is a script that loads a large quantity of random, fake data):sudo su
cd /var/www/securedrop
./qa_loader.py
This might take some time. Note that it is the first time we are using this script in QA, so get loud if you notice any errors or anything confusing.
Now, you should verify that the UUID columns added in the 0.9.0 release are present in each table, and that the new password hash columns exist:
Open db.sqlite with sqlite3 and validate:
sqlite3 /var/lib/securedrop/db.sqlite
select * from <table>
sudo apt list --installed | grep linux-image does not include linux-image-4.4.115-grsec (3.14.79, 4.4.135 and 4.4.144)tor --version returns 0.3.3.9./securedrop-admin setup works without error./securedrop-admin config works without error./securedrop-admin install works without error./securedrop-admin tailsconfig works without error./securedrop-admin backup works without errorapt-key list on app and mon server return an expiry date of 2019-10-03 for the SecureDrop release Key (00FAD77) (#3723)/var/securedrop/wheelhouse contains Flask 1.0.2 and werkzeug 0.14.1 (#3740)[ ] Submitting a blank message shows a relevant error message
[ ] Script in test plan of https://github.com/freedomofpress/securedrop/pull/3774 completes without error
NOTE: I did not test the journalist API nor did I test the database migrations. I only installed cleanly, details are below.
[x] On submit page, file less than 500 MB submitted successfully
[x] Nonexistent codename cannot log in
[x] 2fa immediate reuse cannot log in
[x] Filter by codename works
[] Selecting all and clicking "Download all" works
[x] You can submit a reply and a flashed message and new row appears
[x] Update the kernel testing matrix if you are using dedicated hardware
[x] tor bootstaps successfully and source/journalist interfaces are reachable
[x] tor --version returns 0.3.3.9
[x] Reply to a source, and the reply is visible in the journalist interface
_Note: API db migration tests TK_
Install on 0.8.0.
[x] Upgrade to 0.9.0. The database migration occurs smoothly.
:exclamation: #3733 was opened to track this - db migration v.v.slow
[x] You can log in as your test user, the data should still be present in the journalist interface.
Now, you should verify that the UUID columns added in the 0.9.0 release are present in each table, and that the new password hash columns exist:
Open db.sqlite with sqlite3 and validate:
sqlite3 /var/lib/securedrop/db.sqlite
select * from <table>
To use the tbb-0.9.0 branch against an external rc server, one has to fill up an instance_information.json file inside of the tests/functinoal directory as given in https://github.com/freedomofpress/securedrop/tree/tbb-0.9.0/securedrop/tests/functional#to-test-in-prod-vms , make sure to have a "sleep_time": 30 or more.
Next, you can run all the functional tests by
cd securedrop
./bin/dev-shell ./bin/run-test --capture=no -v tests/functional/
It is better to run each of the test files inside of the functional tests directory one by one. Remember to reset the terminal if you get a lot of debugging error log. The errors (Python tracebacks) will be mentioned in the top section of the output.
What should the expected behaviour be when restoring a backup from a 0.8.0 instance onto a 0.9.0 one? Is that simply unsupported? If not, is it expected to work? If so, it might be good to have a test case like
securedrop-admin backup:exclamation: https://github.com/freedomofpress/securedrop/issues/3732 was opened to track the previous version backup applying database migrations.
Install on 0.8.0.
qa-loader.py (this is a script that loads a large quantity of random, fake data):sudo su
cd /var/www/securedrop
./qa_loader.py
This might take some time. Note that it is the first time we are using this script in QA, so get loud if you notice any errors or anything confusing.
Now, you should verify that the UUID columns added in the 0.9.0 release are present in each table, and that the new password hash columns exist:
Open db.sqlite with sqlite3 and validate:
sqlite3 /var/lib/securedrop/db.sqlite
select * from <table>
sudo apt list --installed | grep linux-image does not include linux-image-4.4.115-grsec (3.14.79, 4.4.135 and 4.4.144)tor --version returns 0.3.3.9./securedrop-admin setup works without error./securedrop-admin config works without error./securedrop-admin install works without error./securedrop-admin tailsconfig works without error./securedrop-admin backup works without errorThat's a great point @zenmonkeykstop, thanks! Based on my testing for 0.8.0 and 0.9.0, the application will break, I have opened a ticket to track this https://github.com/freedomofpress/securedrop/issues/3732 and updated the administration section of the test plan. Feel free to edit the test plan further, as you see fit.
sudo dpkg-reconfigure securedrop-app-code on the app server fixes the install. (did not test)../securedrop-admin setup successful./securedrop-admin sdconfig successful./securedrop-admin install successful./securedrop-admin tailsconfig successfulInstall on 0.8.0.
SSH into the application, and run the qa-loader.py (this is a script that loads a large quantity of random, fake data):
sudo su
cd /var/www/securedrop
./qa_loader.py
This might take some time. Note that it is the first time we are using this script in QA, so get loud if you notice any errors or anything confusing.
Now, you should verify that the UUID columns added in the 0.9.0 release are present in each table, and that the new password hash columns exist:
Open db.sqlite with sqlite3 and validate:
sqlite3 /var/lib/securedrop/db.sqlite
select * from <table>
sudo apt list --installed | grep linux-image does not include linux-image-4.4.115-grsec (3.14.79, 4.4.135 and 4.4.144)tor --version returns 0.3.3.9Edit: opened as #3758
I believe I found a bug in upgrading from 0.8 to 0.9rc2 in that sources can only submit documents or documents and messages. If I try to send _only_ messages or a blank form field I get a "Bad Request, the browser or proxy sent a request that this server could not understand" error.
First I installed 0.8.0 on hardware and create a journalist. I then ran ./qa-loader.py -m 25. I logged in and and see submissions. I then added the apt-test key and updated sources.list to apt.freedom.press to apt-test.freedom.press. Finally I ran sudo cron-apt -i -s to update to 0.9rc2
I also enabled apache debug logging and attempted to patch the source_app/ code to log anything related to CSRF violations and I was was not able to trigger a debug log.
Notes: I know we are cutting rc4 debs soon, this list is for posterity.
/etc/apache2/sites-available/journalist.confInstall on 0.8.0.
[x] Upgrade to 0.9.0. The database migration occurs smoothly.
[x] You can log in as your test user, the data should still be present in the journalist interface.
Now, you should verify that the UUID columns added in the 0.9.0 release are present in each table, and that the new password hash columns exist:
Open db.sqlite with sqlite3 and validate:
sqlite3 /var/lib/securedrop/db.sqlite
select * from <table>
./securedrop-admin setup successful./securedrop-admin sdconfig successful./securedrop-admin install successful./securedrop-admin tailsconfig successfulInstall on 0.8.0.
./qa-loader.py -m 25 in /var/www/securedrop (this is a script that loads a large quantity of random, fake data):sudo su
cd /var/www/securedrop
./qa_loader.py
This might take some time. Note that it is the first time we are using this script in QA, so get loud if you notice any errors or anything confusing.
Now, you should verify that the UUID columns added in the 0.9.0 release are present in each table, and that the new password hash columns exist:
Open db.sqlite with sqlite3 and validate:
sqlite3 /var/lib/securedrop/db.sqlite
select * from <table>
sudo apt list --installed | grep linux-image does not include linux-image-4.4.115-grsec (3.14.79, 4.4.135 and 4.4.144)tor --version returns 0.3.3.9./securedrop-admin setup works without error./securedrop-admin config works without error./securedrop-admin install works without error./securedrop-admin tailsconfig works without error./securedrop-admin backup works without errorapt-key list on app and mon server return an expiry date of 2018-10-03 for the SecureDrop release Key (00FAD77) (#3723)/var/securedrop/wheelhouse contains Flask 1.0.2 and werkzeug 0.14.1 (#3740)[ ] Submitting a blank message shows a relevant error message
[ ] Script in test plan of https://github.com/freedomofpress/securedrop/pull/3774 completes without error
NOTE: Tested on both rolled back 3.14 kernels and on the new kernels. After upgrading to 0.9rc5, the server worked as expected. Ran through all tests and then rolled the server forward to the new kernel and it works as intended.
!! Did not test #3732
./securedrop-admin setup successful./securedrop-admin sdconfig successful./securedrop-admin install successful./securedrop-admin tailsconfig successfulInstall on 0.8.0.
./qa-loader.py -m 25 in /var/www/securedrop (this is a script that loads a large quantity of random, fake data):sudo su
cd /var/www/securedrop
./qa_loader.py
This might take some time. Note that it is the first time we are using this script in QA, so get loud if you notice any errors or anything confusing.
Now, you should verify that the UUID columns added in the 0.9.0 release are present in each table, and that the new password hash columns exist:
Open db.sqlite with sqlite3 and validate:
sqlite3 /var/lib/securedrop/db.sqlite
select * from <table>
sudo apt list --installed | grep linux-image does not include linux-image-4.4.115-grsec (3.14.79, 4.4.135 and 4.4.144)tor --version returns 0.3.3.9./securedrop-admin setup works without error./securedrop-admin config works without error./securedrop-admin install works without error./securedrop-admin tailsconfig works without error./securedrop-admin backup works without errorapt-key list on app and mon server return an expiry date of 2018-10-03 for the SecureDrop release Key (00FAD77) (#3723) (is 2019-10-03)/var/securedrop/wheelhouse contains Flask 1.0.2 and werkzeug 0.14.1 (#3740)[x] Submitting a blank message shows a relevant error message
[x] Script in test plan of https://github.com/freedomofpress/securedrop/pull/3774 completes without error
Most helpful comment
SecureDrop 0.9.0 pre-release QA Checklist
For both upgrades and fresh installs, here is a list of functionality that requires testing. You can use this for copy/pasting into your QA report. Feel free to edit this message to update the plan as appropriate.
Basic Server Testing
Command Line User Generation
Administration
Application Acceptance Testing
Source Interface
Landing page base cases
First submission base cases
Returning source base cases
Journalist Interface
Login base cases
Index base cases
Individual source page
0.9.0-specific testing
Tails 3.9~rc1 specific testing
Upgrade path
./securedrop-admin setupsuccessful./securedrop-admin sdconfigsuccessful./securedrop-admin installsuccessful./securedrop-admin tailsconfigsuccessfulJournalist API:
Database Migrations (upgrade testing from 0.8.0)
Install on 0.8.0.
SSH into the application, and run
./qa-loader.py -m 25in/var/www/securedrop(this is a script that loads a large quantity of random, fake data):This might take some time. Note that it is the first time we are using this script in QA, so get loud if you notice any errors or anything confusing.
Now, you should verify that the UUID columns added in the 0.9.0 release are present in each table, and that the new password hash columns exist:
Open db.sqlite with sqlite3 and validate:
Linux kernel 4.4.144
sudo apt list --installed | grep linux-imagedoes not include linux-image-4.4.115-grsec (3.14.79, 4.4.135 and 4.4.144)Tor 0.3.3.9
tor --versionreturns 0.3.3.9Journalist Reply refactor
Tails 3.9~rc1 testing
./securedrop-admin setupworks without error./securedrop-admin configworks without error./securedrop-admin installworks without error./securedrop-admin tailsconfigworks without error./securedrop-admin backupworks without error0.9.0-rc2 specific testing
apt-key liston app and mon server return an expiry date of 2019-10-03 for the SecureDrop release Key (00FAD77) (#3723)/var/securedrop/wheelhousecontains Flask 1.0.2 and werkzeug 0.14.1 (#3740)0.9.0-rc4 specific testing
[ ] Submitting a blank message shows a relevant error message
Journalist API can be accessed through Apache #3772
[ ] Script in test plan of https://github.com/freedomofpress/securedrop/pull/3774 completes without error
0.9.0-rc5 specific testing