Securedrop: Ansible fails to "Add firewall exemption for OSSEC agent registration."

[jvoisin]

Bug

Description

When running the install part, I've got a stacktrace from Ansible, in the "Add firewall exemption for OSSEC agent registration." step.

Steps to Reproduce

I'm simply running ./securedrop-admin install. I __carefully_ followed the documentation to setup everything, even bought the recommended hardware.

Expected Behavior

Ansible should finish successfully.

Actual Behavior

PLAY [Configure SecureDrop Monitor Server.] ************************************

PLAY [Configure SecureDrop Application Server.] ********************************

TASK [setup] *******************************************************************
ok: [app]

TASK [ossec-agent : Install securedrop-ossec-agent package.] *******************
ok: [app]

TASK [ossec-agent : Check whether iptables rules exist.] ***********************
ok: [app]

TASK [ossec-agent : Add firewall exemption for OSSEC agent registration.] ******
fatal: [app]: FAILED! => {"failed": true, "msg": "The conditional check 'hostvars[groups.securedrop_monitor_server.0].ossec_agent_already_registered == false and iptables_rules_check_result.stat.exists == true' failed. The error was: error while evaluating conditional (hostvars[groups.securedrop_monitor_server.0].ossec_agent_already_registered == false and iptables_rules_check_result.stat.exists == true): 'dict object' has no attribute 'ossec_agent_already_registered'\n\nThe error appears to have been in '/live/persistence/TailsData_unlocked/Persistent/securedrop/install_files/ansible-base/roles/ossec-agent/tasks/agent_config.yml': line 17, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Add firewall exemption for OSSEC agent registration.\n  ^ here\n"}
    to retry, use: --limit @/live/persistence/TailsData_unlocked/Persistent/securedrop/install_files/ansible-base/securedrop-prod.retry

PLAY RECAP *********************************************************************
app                        : ok=55   changed=1    unreachable=0    failed=1   
localhost                  : ok=23   changed=0    unreachable=0    failed=0   
mon                        : ok=10   changed=0    unreachable=0    failed=1   

TASK: common : Install tmux. ------------------------------------------- 80.36s
TASK: common : Install ntp for ntpd. ----------------------------------- 80.28s
TASK: grsecurity : Install the grsecurity-patched kernel from the FPF repo. -- 11.40s
TASK: common : Perform safe upgrade to ensure all the packages are updated. --- 3.99s
TASK: common : Remove hold on tor package, to permit automatic upgrades. --- 3.76s
TASK: common : Check whether tor will be upgraded. ---------------------- 2.56s
TASK: common : Set sysctl flags for net.ipv4 config. -------------------- 1.89s
TASK: grsecurity : Remove generic kernel packages. ---------------------- 1.21s
TASK: validate : Confirm public key file and fingerprint match. --------- 1.08s
TASK: validate : Confirm public key file and fingerprint match. --------- 1.08s

Playbook finished: Fri Sep 29 13:59:04 2017, 81 total tasks.  0:03:24 elapsed. 

Traceback (most recent call last):
  File "./securedrop-admin", line 312, in <module>
    args.func(args)
  File "./securedrop-admin", line 215, in install_securedrop
    '--ask-become-pass'], cwd=ANSIBLE_PATH)
  File "/usr/lib/python2.7/subprocess.py", line 186, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/live/persistence/TailsData_unlocked/Persistent/securedrop/./install_files/ansible-base/securedrop-prod.yml', '--ask-become-pass']' returned non-zero exit status 2
amnesia@amnesia:/live/persistence/TailsData_unlocked/Persistent/securedrop$ 

[b-meson]

Thanks for the report @jvoisin, I'm really sorry to see this report particularly because I did a new install of 0.4.3 a few weeks ago and everything worked as expected on a new clean, hardware install. Can you verify the git tags you are using are the exact latest and the proper version?

1. Let's check we are on the proper branch and remote

amnesia@amnesia:~/Persistent/securedrop$ git remote -v 
origin  https://github.com/freedomofpress/securedrop.git (fetch)
origin  https://github.com/freedomofpress/securedrop.git (push)

1. Let's verify we are on the correct branch (this will change in a bit)

amnesia@amnesia:~/Persistent/securedrop$ git branch
* develop

1. Make sure you have the latest code checked out (i.e. git pull)

2. Checkout the latest git tag we have released

amnesia@amnesia:~/Persistent/securedrop$ git checkout 0.4.3
Note: checking out '0.4.3'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 57ba5a5a... SecureDrop 0.4.3

1. Verify the hash you are on is the latest (HEAD is now at 57ba5a5a)

amnesia@amnesia:~/Persistent/securedrop$ git log
commit 57ba5a5ab2c69c35736edace7e2348bb09ff7b57

    SecureDrop 0.4.3

1. Check out the latest git tag.

amnesia@amnesia:~/Persistent/securedrop$ git tag -v 0.4.3
object 57ba5a5ab2c69c35736edace7e2348bb09ff7b57
[snip]
gpg: Signature made Tue 12 Sep 2017 05:45:06 PM UTC
gpg:                using RSA key 22245C81E3BAEB4138B36061310F561200F4AD77
gpg: Good signature from "SecureDrop Release Signing Key" [unknown]

Once we have confirmed 1-6 are all correct, lets continue to the some further debugging. In the _latest_ documentation (not stable) for running the Ansible playbooks we have the following:

The install process will take some time, and it will return the terminal to you when it is complete. If an error occurs while running the install, please check all of the details of the error output and if needed, make edits to the file located at install_files/ansible-base/group_vars/all/site-specific as described above. If you continue to have issues please submit a detailed GitHub issue or send an email to [email protected].

One thing we don't have documented right now (there are open issues around this, see for example #2098) is that you should be able to re-run Ansible playbooks for as much as needed. In your case, did you attempt to rerun the playbooks? If so, did it fail at the same place? If not, can you attempt to rerun the Ansible playbooks and let us know if it continues to fail at the same place?

We are investigating this error as well and attempting to reproduce.

[conorsch]

Great write-up, @freddymartinez9. I was _not_ able to reproduce this issue following the workflow you outlined. In particular, I note that the working directory for the commands as posted as non-standard:

amnesia@amnesia:/live/persistence/TailsData_unlocked/Persistent/securedrop$

As @freddymartinez9 mentioned, commands should be run from ~/Persistent/securedrop, which maps to the same location. Ran through the ./securedrop-admin install flow from the non-standard directory, as well, in case there was a discrepancy in permissions—but encountered no errors.

Let us know how you fare with the detailed instructions, @jvoisin, I'm sure we'll be able to help you resolve.

[jvoisin]

I attempted to re-run the Ansible playbooks (hoping that there were idempotent), but I got the same results. I did it again from the right™ location.

I failed to mention that there is also a warning during the installation of the ntp package on mon, stating that "No package matching 'ntp' is available", but the ntp package is successfully installed on the host :/

The installation step about tmux for mon is taking _way longer_ than the one for app, but is still successful.

Thank you very much for taking the time to help ♥

[jvoisin]

Apparently, the issue that I'm having is exactly the same that you would get if you forgot to hit the "apply" button on the firewall in the OPT2 section.

It's now solved, thank you, and sorry for the noise :<

[conorsch]

Thanks for the quick response, @jvoisin! The firewall config is a painful process, but we're working on automating it. Glad you're forging ahead with the install, keep us posted on any problems.