Securedrop: Re-run of playbook against existing server fails on OSSEC registration

[msheiny]

Bug

Description

Upon re-run of the install playbook against an existing production instance, I found a really weird condition where a few things line up to cause the ossec agent registration tasks to break. ossec fails to detect a registered client AND then bombs out on attempt to register. This is against an existing install that's already been rebooted (and iptables clamped down).

Steps to Reproduce

Attempt to run ./securedrop-admin install against an existing installation where the IP is listed twice under the ossec-agent list tool. I'm not certain of when this scenario would have occurred but I've found it on mine so I'm guessing it may be present somewhere else.

Expected Behavior

Playbook runs successfully.

Actual Behavior

Playbook bombs out at task Register OSSEC agent. under install_files/ansible-base/roles/ossec-agent/tasks/agent_config.yml . I confirmed on the app server that there is no iptables hole punched for port 1515 outbound.

Comments

• https://github.com/freedomofpress/securedrop/blob/develop/install_files/ansible-base/roles/ossec-server/tasks/authd.yml#L23 -- detection of existing agent registered can be incorrect. For instance, in my prod, my agent is listed registered as - {{ app_ip }}-{{ app_ip }} is available. (vs the expected {{ app_hostname }}-{{ app_ip }} is available.)
• okay, that scenario should be fine and ansible attempts to re-register BUT the firewall rule needed (1515) will not open right away b/c the logic in https://github.com/freedomofpress/securedrop/blob/develop/install_files/ansible-base/roles/ossec-agent/tasks/agent_config.yml#L17 is flawed. The opening up of the port will not take affect until the handlers are flushed. Which is not done until the role runs ... but it'll fail before that. The entire iptables logic is problematic here and I'm assuming it works on first install because there is no iptables restrictions on in a default ubuntu installation.

[emkll]

Seems like i'm getting the same behaviour as well. The operation times out (~60 seconds) when attempting to register to the ossec server. It did not do that at install time with 0.4.3.

[b-meson]

To get past this we had add the INPUT and OUTPUT iptables rules for port 1515 on the monitor server. I suspect after the mon playbook finishes it doesn't have port 1515 open. Since the app server can't connect to mon for ossec agent registry, it fails the playbook here.

[b-meson]

Took another look at this and I think the behavior is somewhere in these files in ansible-base.

/roles/ossec-agent/tasks/agent_config.yml
roles/ossec-server/tasks/authd.yml:
group_vars/securedrop_application_server.yml
group_vars/securedrop_monitor_server.yml

[b-meson]

We had to manually add the following IP tables rules to the mon server to allow the app server to register the OSSEC agent.

-A INPUT -s 10.20.3.10/32 -p udp -m udp --dport 1515 -m state --state NEW,RELATED,ESTABLISHED -m comment --comment "Allow OSSEC agent to register" -j ACCEPT
-A INPUT -s 10.20.3.10/32 -p tcp --dport 1515 -m state --state NEW,RELATED,ESTABLISHED -m comment --comment "Allow OSSEC agent to register" -j ACCEPT
-A OUTPUT -d 10.20.3.10/32 -p tcp --sport 1515 -m state --state RELATED,ESTABLISHED -m comment --comment "Allow OSSEC agent to register" -j ACCEPT

[emkll]

For anyone having issues reproducing, de-register the agent on the mon server and re-run the playbook:

• ssh into mon
• /var/ossec/bin/manage_agents
• Remove the app agent
• Quit and restart ossec: /var/ossec/bin/ossec_control restart
• Re-run ansible playbooks and observe failure

[redshiftzero]

Moved to 0.6 milestone as #2748 is not quite ready yet.