Docker-mailserver: Configuring `PERMIT_DOCKER=network` allows use as open relay with IPv6 enabled on host but disabled in Docker

[Rillke]

Configuring PERMIT_DOCKER=network allows use as open relay

What was the behaviour observed?

Configuring PERMIT_DOCKER=network allowed the an external IP to send E-Mail to an external address.

What was the expected behaviour?

Only E-Mail from valid (SPF checks passing) external hosts are accepted.

Steps to reproduce:

1. Configure the mail container:

• DKIM not configured
• ENABLE_SRS=0
• PERMIT_DOCKER=network
• HOSTNAME=mail
• DOMAINNAME=DOMAINNAME.com
• CONTAINER_NAME=mail

1. Get that image:

tvial/docker-mailserver                          latest              4b4724934af6        2 weeks ago         544MB

1. Start the container based on it with docker-compose up.

2. Run telnet on an arbitrary host (not even in your host's network) that allows outgoing connections on port 25:

$ telnet DOMAINNAME.com 25
Trying 2c23:xxx:xx:1021:xxx:873c:0:1...
Connected to DOMAINNAME.com.
Escape character is '^]'.
220 mail.DOMAINNAME.com ESMTP Postfix (Debian)
EHLO mail.DOMAINNAME.com
250-mail.DOMAINNAME.com
250-PIPELINING
250-SIZE 10240000
250-ETRN
250-STARTTLS
250-AUTH PLAIN LOGIN
250-AUTH=PLAIN LOGIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
MAIL FROM: <[email protected]>
250 2.1.0 Ok
RCPT TO: <[email protected]>
250 2.1.5 Ok
DATA
354 End data with <CR><LF>.<CR><LF>
Subject: test

Test message.
.
250 2.0.0 Ok: queued as ECAB5243619
QUIT
221 2.0.0 Bye
Connection closed by foreign host.

1. Pling qulong. [email protected] just got email.

Container logs:

mail    | Feb 21 01:05:34 mail postfix/master[1007]: daemon started -- version 3.1.12, configuration /etc/postfix
mail    | Feb 21 01:07:12 mail postfix/postscreen[1361]: cache btree:/var/lib/postfix/postscreen_cache full cleanup: retained=0 dropped=0 entries
mail    | Feb 21 01:07:12 mail postfix/postscreen[1361]: CONNECT from [172.25.0.1]:45636 to [172.25.0.2]:25
mail    | Feb 21 01:07:12 mail postfix/postscreen[1361]: WHITELISTED [172.25.0.1]:45636
mail    | Feb 21 01:07:12 mail postfix/smtpd[1362]: connect from unknown[172.25.0.1]
mail    | Feb 21 01:07:12 mail opendmarc[213]: ignoring connection from [172.25.0.1]
mail    | Feb 21 01:09:06 mail postfix/smtpd[1362]: ECAB5243619: client=unknown[172.25.0.1]
mail    | Feb 21 01:09:09 mail postfix/submission/smtpd[1811]: warning: hostname zgXXXXX-XXX.XXXXXX.com does not resolve to address 19X.XXX.XX5.29: Name or service not known
mail    | Feb 21 01:09:09 mail postfix/submission/smtpd[1811]: connect from unknown[19X.XXX.XX5.29]
mail    | Feb 21 01:09:09 mail postfix/submission/smtpd[1811]: disconnect from unknown[19X.XXX.XX5.29] ehlo=1 quit=1 commands=2
mail    | Feb 21 01:09:31 mail postfix/cleanup[1802]: ECAB5243619: message-id=<>
mail    | Feb 21 01:09:31 mail opendkim[206]: ECAB5243619: can't determine message sender; accepting
mail    | Feb 21 01:09:31 mail postfix/qmgr[1010]: ECAB5243619: from=<[email protected]>, size=244, nrcpt=1 (queue active)
mail    | Feb 21 01:09:32 mail postfix/smtpd[1904]: connect from localhost[127.0.0.1]
mail    | Feb 21 01:09:32 mail postfix/smtpd[1904]: B2161243662: client=localhost[127.0.0.1]
mail    | Feb 21 01:09:32 mail postfix/cleanup[1802]: B2161243662: message-id=<[email protected]>
mail    | Feb 21 01:09:32 mail postfix/qmgr[1010]: B2161243662: from=<[email protected]>, size=723, nrcpt=1 (queue active)
mail    | Feb 21 01:09:32 mail amavis[1012]: (01012-01) Passed BAD-HEADER-7 {RelayedOutbound,Quarantined}, LOCAL [172.25.0.1]:45636 <[email protected]> -> <[email protected]>, quarantine: j/badh-juwrtxl8uFt9, Queue-ID: ECAB5243619, mail_id: juwrtxl8uFt9, Hits: 2.743, size: 294, queued_as: B2161243662, 1101 ms
mail    | Feb 21 01:09:32 mail postfix/smtp[1897]: ECAB5243619: to=<[email protected]>, relay=127.0.0.1[127.0.0.1]:10024, delay=46, delays=44/0.01/0.01/1.1, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B2161243662)
mail    | Feb 21 01:09:32 mail postfix/qmgr[1010]: ECAB5243619: removed
mail    | Feb 21 01:09:38 mail postfix/smtp[1905]: Anonymous TLS connection established to redirect.ovh.net[213.186.33.5]:25: TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)
mail    | Feb 21 01:09:38 mail postfix/smtp[1905]: B2161243662: to=<[email protected]>, relay=redirect.ovh.net[213.186.33.5]:25, delay=5.5, delays=0.01/0.01/5.4/0.09, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 2FF13AD)
mail    | Feb 21 01:09:38 mail postfix/qmgr[1010]: B2161243662: removed
mail    | Feb 21 01:09:39 mail postfix/smtpd[1362]: disconnect from unknown[172.25.0.1] ehlo=1 mail=1 rcpt=1 data=1 quit=1 commands=5

Compose configuration:

version: '2'
services:
  mail:
    image: tvial/docker-mailserver:latest
    hostname: ${HOSTNAME}
    domainname: ${DOMAINNAME}
    container_name: ${CONTAINER_NAME}
    ports:
      - "25:25"
      - "143:143"
      - "587:587"
      - "993:993"
    volumes:
      - maildata:/var/mail
      - mailstate:/var/mail-state
      - maillogs:/var/log/mail
      - ./config/:/tmp/docker-mailserver/
      - certs:/etc/letsencrypt/live
    env_file:
      - .env
      - env-mailserver
    cap_add:
      - NET_ADMIN
      - SYS_PTRACE
    restart: always
    networks:
      mail:
        aliases:
          - mailgate
          - mail.DOMAINNAME.com

networks:
  mail:

volumes:
  maildata:
  mailstate:
  maillogs:
  certs:
    external:
      name: some_name_where_certs_are_stored

# vim: tabstop=2 softtabstop=0 expandtab shiftwidth=2 smarttab

[erik-wramner]

Your problem is probably that you are losing the real external IP somewhere. From your logs:

CONNECT from [172.25.0.1]:45636

The connection is from 172.25.0.1. According to the docs PERMIT_DOCKER=network will trust everything on 172.16.0.0/12 so this is the expected behavior.

In other words you need to check your networking setup to ensure that the real external IP addresses for clients are preserved.

[Rillke]

@erik-wramner Thanks. That's strange. I tried to verify what you say

On the host that should run the mail server:

$ docker-compose run --service-ports --rm mail bash -c "nc -vvl 25"
Listening on [0.0.0.0] (family 0, port 25)

Then, I connect from ip-reverse-name.com to the mail host using

$ nc DOMAINNAME.com 25

And this is the result from inside the Container:

Connection from ip-reverse-name.com 42154 received!

Where ip-reverse-name.com is the reverse DNS entry I supplied for the connecting host's public IP.

So I assume the IP address is properly submitted into the Container. Is there another factor to consider? EDIT: Netcat (nc) only supports IPv4. If I would have used ncat or socat in IPv6 mode on the remote host (ip-reverse-name.com), like I did with telnet, while still listening on IPv4 only in the Container, I would have noticed the Gateway IP.

~Can it be somehow caused because I also specified mail.DOMAINNAME.com as an alias for the Container in its local network, perhaps? If so, why?~

[erik-wramner]

There is no need to assume, the logs clearly show that you are connecting from 172.25.0.1. You need to fix the configuration so that the real remote addresses are present in the logs, otherwise things won't work very well. Try to find out where the remote addresses are lost. What is your network topology? Is the host running the mail server (in docker) connected to the Internet with a public IP, or is there something else in front? Do you use a proxy (HAProxy, Nginx) or do you connect directly?

[Rillke]

Is the host running the mail server (in docker) connected to the Internet with a public IP?

Yes.

$ ifconfig
ens3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 5.35.xxx.xxx  netmask 255.255.255.255  broadcast 5.35.xxx.xxx
        inet6 ...  prefixlen 128  scopeid 0x0<global>
        inet6 ...  prefixlen 64  scopeid 0x20<link>
        ether 00:1c:42:xx:xx:xx  txqueuelen 1000  (Ethernet)
        RX packets 2235361  bytes 8548913105 (8.5 GB)
        RX errors 0  dropped 7316  overruns 0  frame 0
        TX packets 1906310  bytes 272608960 (272.6 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

or is there something else in front?

There shouldn't be, except the Host Europe's Router; the contract says, 1 public IPv4 and ifconfig listed that IP.

Do you use a proxy (HAProxy, Nginx)

We use those on Port 80 and 443 but not on any other port, as you can see from the docker-compose.yaml in the initial post.

What is your network topology?

Since I noticed that, I had multiple docker-compose downs and ups so the docker network changed. However I will reproduce the issue on another server and report back. Thanks for your help.

[Rillke]

Now I see something like:

Feb 23 18:00:23 mail postfix/postscreen[3632]: CONNECT from [192.168.48.1]:41296 to [192.168.48.2]:25

in the Logs when Google's Mail Exchanger connects but also

Feb 23 17:54:18 mail dovecot: imap-login: Login: user=<[email protected]>, method=PLAIN, rip=141.xx.xxx.xxx, lip=192.168.48.2, mpid=2243, TLS, session=<qPyXXXXXXXXXXXX>

So Postfix gets the Docker Proxy's Gateway IP on Port 25 and Dovecot the correct remote IP on port 143.

root@mail:/# netstat -tupan
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:993             0.0.0.0:*               LISTEN      189/dovecot         
tcp        0      0 127.0.0.1:10023         0.0.0.0:*               LISTEN      212/postgrey --inet 
tcp        0      0 127.0.0.1:10024         0.0.0.0:*               LISTEN      311/amavisd-new (ma 
tcp        0      0 127.0.0.1:10025         0.0.0.0:*               LISTEN      995/master          
tcp        0      0 0.0.0.0:587             0.0.0.0:*               LISTEN      995/master          
tcp        0      0 0.0.0.0:143             0.0.0.0:*               LISTEN      189/dovecot         
tcp        0      0 0.0.0.0:465             0.0.0.0:*               LISTEN      995/master          
tcp        0      0 0.0.0.0:25              0.0.0.0:*               LISTEN      995/master          
tcp        0      0 127.0.0.1:8891          0.0.0.0:*               LISTEN      197/opendkim        
tcp        0      0 127.0.0.1:8893          0.0.0.0:*               LISTEN      204/opendmarc       
tcp        0      0 127.0.0.11:41021        0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.1:36124         127.0.0.1:10025         ESTABLISHED 1000/amavisd-new (c      
tcp        0      0 192.168.48.2:25         5.xxx.xxx.xxx:58848     ESTABLISHED 8409/smtpd
tcp        0      0 127.0.0.1:10025         127.0.0.1:36124         ESTABLISHED 3661/smtpd                        
tcp        0      0 192.168.48.2:143        141.xx.xxx.xxx:56232    ESTABLISHED 2240/dovecot/imap-l                
tcp6       0      0 :::993                  :::*                    LISTEN      189/dovecot         
tcp6       0      0 :::587                  :::*                    LISTEN      995/master          
tcp6       0      0 :::143                  :::*                    LISTEN      189/dovecot         
tcp6       0      0 :::465                  :::*                    LISTEN      995/master          
tcp6       0      0 :::25                   :::*                    LISTEN      995/master          
udp     4608      0 127.0.0.11:51099        0.0.0.0:*                           -           

[Rillke]

Aha. Compose created an IPv4 network, since IPv6 is not enabled for Docker on that host. When another host connects to the host running the container using IPv6, the Docker Proxy translates IPv6 to IPv4 and inserts itself/its Gateway as Source IP Address, essentially doing NAT.

I think that scenario isn't uncommon @erik-wramner - since by default many servers you can rent have IPv6 enabled initially but Docker disabled that by default.

Additionally, If IPv6 addressing is desired, the enable_ipv6 option must be set, and you must use a version 2.x Compose file. IPv6 options do not currently work in swarm mode. (compose reference).

I am in favor of a big warning on that. Easy to miss (while setting up and testing) and ugly impact. (And surely not the container's author's fault.)

[Rillke]

Sadly, I can't find any documentation about the Docker IPv6 to IPv4 NAT behavior.

Thinking about solutions, we might block the gateways using iptables when PERMIT_DOCKER is set to anything which is not empty or host and emit a warning on startup? Should we additionally document this case? @erik-wramner

[Rillke]

As an interim solution I bound port 25 to the public IPv4 interface instead of any interface. If my host wants to send email, it has to authenticate now, but that's okay.

    ports:
-      - "25:25"
+      - "the_IP_v4:25:25"

[erik-wramner]

Sorry for not answering, I've been busy. I don't want to firewall anything in the container, not least because I don't want the container to have that access right. I know it is needed for fail2ban, but people not using fail2ban can skip granting it and be safer for it.

I'm all for a loud and clear warning in the README and Wiki and by all means if you want to log it during startup. Not sure how you would detect it without false alarms though?

[polarathene]

Should this warning also be made clear in the wiki example? It demonstrates enabling the PERMIT_DOCKER=network for allowing integration with other docker containers which is great, but if the user has not seen the warning elsewhere, are they now providing an open relay?

The main README links to wikipedia about open relays, which does have a section on closing them:

In particular, a properly secured SMTP mail relay should not accept and forward arbitrary e-mails from non-local IP addresses to non-local mailboxes by an unauthenticated or unauthorized user.

How that should be done has not been communicated though?

Aha. Compose created an IPv4 network, since IPv6 is not enabled for Docker on that host. When another host connects to the host running the container using IPv6, the Docker Proxy translates IPv6 to IPv4 and inserts itself/its Gateway as Source IP Address, essentially doing NAT.

Is this also going to be an issue then with reverse proxies? I don't think nginx-proxy supports these ports, but I think others like Traefik do?(which can be containerized or not)

As an interim solution I bound port 25 to the public IPv4 interface instead of any interface. If my host wants to send email, it has to authenticate now, but that's okay.

So that I'm understanding this correctly... that's the IPv4 IP for the host server that's exposed to the internet? Is it only important for port 25? The IP needs to be a specific one(public IP to server that the domain name points to?), 0.0.0.0 (all ipv4 interfaces) would suffer the same problem right? (::0 is equivalent for ipv6 interfaces afaik, may also include ipv4, not sure)

I guess that there is rarely any reason for having a proxy layer via a container like Traefik for these ports(not done any multi-server distributed container deployments with things like kubernetes, so not sure if that could be a case).

Is there a benefit to this vs the default localhost?(containers 127.0.0.1 from my understanding?) Instead of being able to direct other containers to the mail service container IP within the internal docker network, requests need to be sent out back to the host server IP on port 25 to reach the mail container?

I may be misunderstanding something as I'm new to setting up a mail server. I imagine a common use-case is to setup one for another container service to be able to send out e-mails, the docs aren't entirely clear on this, PERMIT_DOCKER=network is implied as necessary though?

I am in favor of a big warning on that. Easy to miss (while setting up and testing) and ugly impact. (And surely not the container's author's fault.)

Big warning would be useful, but more clarity on what the user should do to avoid the vulnerability would be good too. Besides locking the port to the host external IP, does enabling IPv6 in docker also prevent open relay issue?

[polarathene]

Another mention for PERMIT_DOCKER=network in the FAQ regarding use with Rancher.

[Rillke]

Besides locking the port to the host external IP, does enabling IPv6 in docker also prevent open relay issue?

Most likely. You can verify that by connecting via IPv6 at the same time watching the logs.

[aendeavor]

This should be taken seriously. Has anything happened in the meantime here? I will make sure the README explains this, and we'll need to log this.

[aendeavor]

I patched the mentioned FAQ / WIKI entries with the warning message @Rillke provided in #1415. The README provides this information already due to #1415. I would consider this done here. But please re-open if you think there should be done more.