Rkt: Docker interferes with rkt's IP masquerading

[blalor]

I'm running rkt and Docker on my hosts, currently. Docker 1.13.0 (changelog) introduced a change that sets the default FORWARD policy to DROP. After upgrading Docker from 1.12.5 to 1.13.1 I found that my rkt containers could no longer reach the network. I think rkt should be more diligent in setting up the iptables rules to work under these conditions.

Environment

rkt Version: 1.25.0
appc Version: 0.8.10
Go Version: go1.7.4
Go OS/Arch: linux/amd64
Features: -TPM +SDJOURNAL
--
Linux 4.11.0-1.el7.elrepo.x86_64 x86_64
--
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
--
systemd 219
+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN

What did you do?

rkt run --interactive --insecure-options=image,ondisk docker://alpine --exec=/bin/ping -- -c 1 8.8.8.8

What did you expect to see?

PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=37 time=8.576 ms

--- 8.8.8.8 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 8.576/8.576/8.576 ms

What did you see instead?

PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.

--- 8.8.8.8 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

[squeed]

This is a good catch, and it's unfortunate that Docker is such a bad citizen.

I think the right solution is a chained CNI plugin that automatically fixes the firewall rules. We'll add that to the default rkt networking configurations.

As a quick fix, rkt always allocates out of 172.16.28.0/24 - so you can add that statically to your iptables rules, if you like.

[blalor]

Indeed it is; I spent two days troubleshooting this. It's obvious in retrospect, but my iptables-fu is seriously lacking. My workaround right now is adding a systemd drop-in to Docker's system unit with ExecStartPost=iptables -P FORWARD ALLOW. What would that look like to target just rkt's address space?

[squeed]

I believe CentOS7 now uses firewalld. That means you should just be able to do

sudo firewall-cmd --add-source=172.16.28.0/24 --zone=trusted

once and it should work, including across reboots. Let me know if it works?

[blalor]

I'm not using firewalld, depending on AWS security groups instead. I was looking for a raw iptables command, but I'll figure it out. :-)

[squeed]

Ah, Then just do

iptables -A FORWARD -s 172.16.28.0/24 -j ACCEPT
iptables -A FORWARD -d 172.16.28.0/24 -j ACCEPT

[blalor]

w00t. thanks!