Docker-mailserver: How to audit all traffic flowing through mail server?

[saigelosli]

My mail server is only about 10 days old. Up til today, my server was not on any blacklist, but today I am listed on 3 of them. I have only exchanged mails between my domain and my other accounts on yahoo, hotmail and gmail for testing purposes. I would like to know how to audit all successful logins and mails sent from my server so that I can identify why I show up on a blacklist.

Context

Even as good as this software is, setting it up to actually exchange mails is a delicate task. So I test with mails sent between my new domain and my established accounts on yahoo, hotmail and gmail. As far as I know, my test emails are the ONLY emails sent through my server, yet it is now showing up on blacklists. I would like to know how to determine all successful logins and email sent so that I can find out why I'm blacklisted.

Expected Behavior

Actual Behavior

Possible Fix

Steps to Reproduce

1.
2.
3.
4.

Your Environment

• Amount of RAM available: 24GB
• Mailserver version used: latest
• Docker version used: 3
• Environment settings relevant to the config:
• Any relevant stack traces ("Full trace" preferred):

[erik-wramner]

The logs are saved to /var/log/mail. You will want to create a volume to keep them when the container is restarted, see templates in the readme. For a quick look you can use docker logs, but for more detailed analysis you will want the real files.

You can also enable the reports (also covered in readme) to get a more readable summary of your traffic. Again that works best if the logs are on a volume. Good luck!

[rackaid]

Blacklisting for new IPs is not uncommon. Please make sure you have your DNS records (PTR, DKIM, SPF) in order. I also now highly recommend DMARC for identifying invalid email sources.

I've found Yahoo and Hotmail/Outlook to be very sensitive to new IP addresses sending email. Here's a quick how to for removing your IP from the hotmail blacklist.

You can use https://mail-tester.com to verify that your email headers are working properly. The most important part of the test is the Email Authentication setting. If there are any failures here, address those before contacting the email service providers for help.

I use Dmarcian to process my DMARC reports.

[saigelosli]

Thank you to the previous commenters who have given me very valuable input!

Further diagnosing my blacklist problem, I discovered that in fact, my mail server is an open relay, which of course explains why I wound up on blacklists. So the next part of my journey is to discover how to close the relay so that only authorized parties can send email.

[erik-wramner]

You didn't post your configuration. The default configuration should be secure, so when you find out what went wrong please let us know.

[saigelosli]

Again, the issue is that with my current configuration, my mail server is an open relay. Here is my section of the docker-compose.yaml file (with domain name changed to shield the guilty):

  marketing:
    container_name: marketing
#    image: tomav/docker-mailserver:latest
#    image: tvial/docker-mailserver:latest
    image: saigel/docker-mailserver:latest
    restart: always
    domainname: domain.com
    networks:
      default:
        ipv4_address: '172.16.0.112'
    expose:
      - '25'
      - '143'
      - '465'
      - '587'
      - '993'
    volumes:
      - /var/docker/marketing/maildata:/var/mail
      - /var/docker/marketing/mail-state:/var/mail-state
      - ./config/marketing/:/tmp/docker-mailserver/
      - ./letsencrypt/live/mail.domain.com/:/etc/letsencrypt/
      - /root/.ssh/:/root/.ssh/
    environment:
      - DMS_DEBUG=0
      - ENABLE_CLAMAV=1
      - ONE_DIR=0
      - ENABLE_POP3=0
      - ENABLE_FAIL2BAN=1
      - ENABLE_MANAGESIEVE=0
      - OVERRIDE_HOSTNAME=mail.domain.com
      - [email protected]
      - POSTSCREEN_ACTION=enforce
      - REPORT_RECIPIENT=0
      - REPORT_SENDER=
      - REPORT_INTERVAL=daily
      - SMTP_ONLY=
      - SSL_TYPE=manual
      - SSL_CERT_PATH=/etc/letsencrypt/public.crt
      - SSL_KEY_PATH=/etc/letsencrypt/private.key
      - TLS_LEVEL=modern
      - SPOOF_PROTECTION=1
      - ENABLE_SRS=0
      - PERMIT_DOCKER=
      - VIRUSMAILS_DELETE_DELAY=
      - ENABLE_POSTFIX_VIRTUAL_TRANSPORT=
      - POSTFIX_DAGENT=
      - POSTFIX_MAILBOX_SIZE_LIMIT=
      - POSTFIX_MESSAGE_SIZE_LIMIT=
      - ENABLE_SPAMASSASSIN=1
      - SA_TAG=2.0
      - SA_TAG2=6.31
      - SA_KILL=6.31
      - SA_SPAM_SUBJECT=***SPAM*****
      - ENABLE_FETCHMAIL=0
      - FETCHMAIL_POLL=300
      - ENABLE_LDAP=0
      - LDAP_START_TLS=
      - LDAP_SERVER_HOST=
      - LDAP_SEARCH_BASE=
      - LDAP_BIND_DN=
      - LDAP_BIND_PW=
      - LDAP_QUERY_FILTER_USER=
      - LDAP_QUERY_FILTER_GROUP=
      - LDAP_QUERY_FILTER_ALIAS=
      - LDAP_QUERY_FILTER_DOMAIN=
      - DOVECOT_TLS=
      - DOVECOT_USER_FILTER=
      - DOVECOT_PASS_FILTER=
      - ENABLE_POSTGREY=1
      - POSTGREY_DELAY=300
      - POSTGREY_MAX_AGE=35
      - POSTGREY_AUTO_WHITELIST_CLIENTS=5
      - POSTGREY_TEXT=Delayed by postgrey
      - ENABLE_SASLAUTHD=0
      - SASLAUTHD_MECHANISMS=
      - SASLAUTHD_MECH_OPTIONS=
      - SASLAUTHD_LDAP_SERVER=
      - SASLAUTHD_LDAP_SSL=
      - SASLAUTHD_LDAP_BIND_DN=
      - SASLAUTHD_LDAP_PASSWORD=
      - SASLAUTHD_LDAP_SEARCH_BASE=
      - SASLAUTHD_LDAP_FILTER=
      - SASLAUTHD_LDAP_START_TLS=
      - SASLAUTHD_LDAP_TLS_CHECK_PEER=
      - SASL_PASSWD=
      - SRS_EXCLUDE_DOMAINS=
      - SRS_SECRET=
      - DEFAULT_RELAY_HOST=
      - RELAY_HOST=
      - RELAY_PORT=25
      - RELAY_USER=
      - RELAY_PASSWORD=
    privileged: true
    cap_add:
      - NET_ADMIN
      - SYS_PTRACE
    labels:
      - traefik.enable=true
      - traefik.frontend.rule=Host:mail.domain.com
    hostname: mail.domain.com

By the way, my custom build is because I needed to add vim, ssh and a couple other packages so that I could use this to fit my needs. To wit: My mail server actually resides at home, and port 25 is blocked by my ISP. So I use ssh to tunnel past it to reach a droplet on DigitalOcean, which becomes my public IP address.

[erik-wramner]

My guess is that the SSH tunnel is the problem here. You have left PERMIT_DOCKER blank which should mean that only connections from localhost are trusted (mynetworks in postfix-main.cf). But if you tunnel the connections they may appear to be from localhost. What remote addresses do you have in your logs?

[rackaid]

An SSH tunnel would require a valid user account on the local network. I have an article about using ssh tunnels to send spam but it is more geared to web hosts rather than Docker.
How did you verify you have an open relay? I recommend you use any of the online tools or telnet directly from a non-trusted network and run the test.
This tool will run a test.
https://mxtoolbox.com/diagnostic.aspx

[saigelosli]

My guess is that the SSH tunnel is the problem here. You have left PERMIT_DOCKER blank which should mean that only connections from localhost are trusted (mynetworks in postfix-main.cf). But if you tunnel the connections they may appear to be from localhost. What remote addresses do you have in your logs?

You must be right. The address for everything in the mail log is 127.0.0.1, like the following excerpt from the log:

Sep 23 01:17:30 mail opendmarc[220]: ignoring connection from localhost
Sep 23 01:17:34 mail postfix/postscreen[1144]: CONNECT from [127.0.0.1]:35168 to [127.0.0.1]:25
Sep 23 01:17:34 mail postfix/postscreen[1144]: WHITELISTED [127.0.0.1]:35168
Sep 23 01:17:34 mail postfix/smtpd[1148]: connect from localhost[127.0.0.1]

Recap and further info: my home network is on 192.168.201.x/24, my Docker containers are on 172.16.0.x/24, my mail server tunnels out to a droplet on DigitalOcean.

Is it possible to configure Docker to close the open relay, but still allow me to send/receive emails through this server properly?

[saigelosli]

An SSH tunnel would require a valid user account on the local network. I have an article about using ssh tunnels to send spam but it is more geared to web hosts rather than Docker.
How did you verify you have an open relay? I recommend you use any of the online tools or telnet directly from a non-trusted network and run the test.
This tool will run a test.
https://mxtoolbox.com/diagnostic.aspx

mxtoolbox.com is exactly the tool I used to confirm it.

[saigelosli]

I should have also mentioned that my mail server co-exists with web server, database server, et al, behind a Traefik proxy. Maybe my problem is that Traefik needs to supply more complete information to the mail server so that it can distinguish local from remote traffic?

I'm learning my way through this, so I probably made several blunders when setting this up...

[erik-wramner]

I think that is the case. Somehow you need to preserve the remote addresses. I don't know Traefik well enough to point to the solution. I think there are solutions for similar problems with Kubernetes in the wiki or in other issues though, perhaps you can find inspiration there?

[erik-wramner]

Can we close this now (long silence)? If not, is there something you need help with?

[saigelosli]

I believe that it would be fair to close the ticket.
I have scripted an edit to the /etc/postfix/main.cf file on my server to remove entries from the 172.16.0.0/16 network from "mynetworks". This effectively closes the "open relay" issue that I was experiencing, since my mail server resides in a Docker container in the "default" network behind an SSH tunnel to the "real world".
However, I am still searching for a way to use FAIL2BAN to block repeated attempts to connect to my server from bad actors. Although I have FAIL2BAN enabled, it is essentially useless because all connections to my server are coming through 127.0.0.1 (again, because I am tunneling from my mail server out to my droplet on DigitalOcean and my static IP). I have discovered that the "real" IP address of the parties connecting to me are available if I ask for verbose messages from SSH (using the "-v" option), but I haven't figured out how to pass that information to FAIL2BAN to block the repeated attempts login to my server.
I'm sure this seems very convoluted, and I applaud you if you can understand my ramblings above! :-)

[erik-wramner]

Right, I'll close it then. Good luck!

[oblitum]

Hey folks, can anyone help me with a similar setup?

I'm also trying to use ssh port forwarding for my case, the VPS is being used solely for that, and it almost works. I can receive emails, but for sending I'm having issues, because my current VPS provider sadly does not allow sending to port 25. I can receive on it but I can't send. To send I need to use a proxy/relay that they provide.

I've logged in my VPS instance and can in fact telnet to this proxy on its port 25, so it all seems that access VPS->RELAY:25 is fine.

I've then tried setup DEFAULT_RELAY_HOST= to this relay, but when I run the server on my Pi, sending always timeout when trying to send to the relay. If I'm understanding it correctly, relaying won't happen through the forwarded ports and my server is trying to connect to the relay through my network instead of the VPS?

If so, how to fix this?

It's my first time trying to setup my own mail server, looks like I'm almost there, but not yet. Not sure whether there's a fix for my case..

[oblitum]

Ah... reading this thread, I see many other issues regarding this setup, around docker networks addresses, FAIL2BAN, etc...

[oblitum]

If so, how to fix this?

Ah OK... thinking about it, I think I should try local forwarding the relay itself too, besides the remote ports.

[saigelosli]

Here is what I wound up doing:

I have two mail servers (one for "transactional" mail, one for "promotional" mail--which seems to be the best practice these days). They are both running as separate docker instances on my home server. I have written some custom scripts to make them connect out to separate VPS hosts, which I am essentially using only for their non-firewalled access to port 25 and their static IP addresses.

I am using GitHub - halhen/pvpn: Poor mans VPN over SSH on each mail server to "forward" all outgoing traffic from my Docker mail servers _to_ my VPS host, and I'm using ssh port forwarding to forward all incoming traffic to the Docker mail servers _from_ my VPS host, so I'm _not_ using my ISP's mail server relay at all.

I am also using a custom script to monitor the IP addresses of attackers and pushing those addresses back to the VPS host in real time to block them with FAIL2BAN.

By modifying the main.cf configuration file for each mail server vis-à-vis my reply of October 7 2019, I have also closed the open relay problem that I was having.

One more thing, I have written custom scripts which coordinate the Lets Encrypt certificates between Traefik and the two mail servers.

Hope these suggestions help you.

[oblitum]

@saigelosli many thanks for your input, I wished there was a more detailed configuration write up, specially regarding the scripts to make FAIL2BAN work etc. Regarding my limitations, sadly, with this VPS, I have to use a relay they provide me for sending, telneting on port 25 of any mail server there just proves it's blocked, and they don't allow it even on request. They're essentially forcing me to use their relay servers. I didn't yet finish configuring this part, but I think initially, as I said, I just need to remote forward not only the ports that docker-mailserver needs, but also have a local forward to access this relay through the VPS.

I'm just barely making a damn email message to reach its destination, I didn't yet setup letsencrypt renewing, which I'll do later after verifying all this will work. I never touched Traefik, and will study it later. So, just saying, a blog post of yours on your setup would be so much valuable :) But anyways, many thanks for the pointers already.

[oblitum]

For what is worth, I thought it was worth the effort going forward for a less hacky approach than relying on ssh tunnels and I managed to get a so much cleaner approach with wireguard and linux network namespaces. I'll probably write a post documenting my process, I'm not an expert on the subject but I've reduced the variables involved to the minimum and got a completely network-isolated container with a single interface (wireguard) that I've created on the host and send to the docker-mailserver container namespace for its exclusive use. And as wireguard has recently been merged to the kernel, things for this method can only improve with time.