Mailu: Traefik guide needs more love

Created on 30 May 2019  ·  29Comments  ·  Source: Mailu/Mailu

Hi,

first of all - thank you for this amazing project. I'm seeing lots of room for improvement, but at the same time I'm :clap: all of what you've accomplished so far.

I'm trying to use mailu as a mail-sender/receiver for a dockerized GitLab server setup so I've started following mailus guide on how to use traefik.

https://mailu.io/master/reverse.html#traefik-as-reverse-proxy

I found it quite hard to follow. I've just found the example for traefik.toml and the docker-compose.yml.
So the actual suggested way is to launch the dumper with the same docker-compose file. I don't feel good about storing the certificates for other traefik services in the same mount as the mail-server lives.

  1. starting traefik (with the certificate dumper service as well)
  2. mailu_frontend bind mounts the DOMAIN.{key,crt} from the traefik dump
  3. mailu starts all services up => creates a directory for the mount point since traefik didn't request a certifciate yet and the dumper not having dumped the not yet generated certificate
  4. mailu stops frontend => traefik doesn't see frontend healthy => doesn't request a certificate

I'm trying to find a solution for this problem.

Also I'm trying to figure out how much everything is going to change with traefik 2.0 (and tcp support).

I think with traefik 2.0 it would be nice to use traefik as TLS termination and to expose TLS for imap/pop/smtp through traefik instead of doing it within mailu.

backlog typenhancement
Source
picture norpol
👍3

Most helpful comment

Hi There,

The Mailu-Project is currently in a bit of a bind! We are short on man-power, and we need to judge if it is possible for us to put in some work on this issue.

To help with that, we are currently trying to find out which issues are actively keeping users from using Mailu, which issues have someone who want to work on them — and which issues may be less important. These a less important ones could be discarded for the time being, until the project is in a more stable and regular state once again.

In order for us to better assess this, it would be helpful if you could put a reaction on this post (use the :smiley: icon to the top-right).

  • 👍️ if you need this to be able to use Mailu. Ideally, you’d also be able to test this on your installation, and provide feedback …
  • 🎉 if you find it a nice bonus, but no deal-breaker
  • 🚀 if you want to work on it yourself!
    We want to keep this voting open for 2 weeks from now, so please help out!
picture Nebukadneza on 18 Aug 2020
👍12 🎉5

All 29 comments

To add on this. With the new TCP support (https://github.com/containous/traefik/pull/4587) we should be able to replace nginx completely, if I’m not mistaken

Asherslab picture Asherslab on 9 Jun 2019
👍4

I'm currently testing mailu behind traefik 2. Traefik routes all http/tcp traffic to mailu frontend container. My mailu setup doesn't expose ports any longer. Since traefik #4875 is fixed, all LGTM.

I use following traefik labels on my mailu frontend container:
``

  labels:
    - "traefik.enable=true"

    - "traefik.http.routers.mailu_http.rule=Host(\`mail.domain.tld\`)"
    - "traefik.http.routers.mailu_http.entrypoints=http"
    - "traefik.http.routers.mailu_http.middlewares=http-chain@file"
    - "traefik.http.routers.mailu_http.service=mailu_https"
    - "traefik.http.routers.mailu_https.rule=Host(\`mail.domain.tld\`)"
    - "traefik.http.routers.mailu_https.entrypoints=https"
    - "traefik.http.routers.mailu_https.tls=true"
    - "traefik.http.routers.mailu_https.middlewares=https-chain@file"
    - "traefik.http.routers.mailu_https.service=mailu_https"
    - "traefik.http.services.mailu_https.loadbalancer.server.port=80"
    - "traefik.http.services.mailu_https.loadbalancer.server.scheme=http"

    - "traefik.tcp.routers.mailu_smtp_relay.rule=HostSNI(\`*\`)"
    - "traefik.tcp.routers.mailu_smtp_relay.entrypoints=smtp-relay"
    - "traefik.tcp.routers.mailu_smtp_relay.service=mailu_smtp_relay"
    - "traefik.tcp.services.mailu_smtp_relay.loadbalancer.server.port=25"

    - "traefik.tcp.routers.mailu_smtp_ssl.rule=HostSNI(\`*\`)"
    - "traefik.tcp.routers.mailu_smtp_ssl.entrypoints=smtp-ssl"
    - "traefik.tcp.routers.mailu_smtp_ssl.tls.passthrough=true"
    - "traefik.tcp.routers.mailu_smtp_ssl.service=mailu_smtp_ssl"
    - "traefik.tcp.services.mailu_smtp_ssl.loadbalancer.server.port=465"

    - "traefik.tcp.routers.mailu_smtp.rule=HostSNI(\`*\`)"
    - "traefik.tcp.routers.mailu_smtp.entrypoints=smtp"
    - "traefik.tcp.routers.mailu_smtp.service=mailu_smtp"
    - "traefik.tcp.services.mailu_smtp.loadbalancer.server.port=587"

    - "traefik.tcp.routers.mailu_pop3.rule=HostSNI(\`*\`)"
    - "traefik.tcp.routers.mailu_pop3.entrypoints=pop3"
    - "traefik.tcp.routers.mailu_pop3.service=mailu_pop3"
    - "traefik.tcp.services.mailu_pop3.loadbalancer.server.port=110"

    - "traefik.tcp.routers.mailu_pop3_ssl.rule=HostSNI(\`*\`)"
    - "traefik.tcp.routers.mailu_pop3_ssl.entrypoints=pop3-ssl"
    - "traefik.tcp.routers.mailu_pop3_ssl.tls.passthrough=true"
    - "traefik.tcp.routers.mailu_pop3_ssl.service=mailu_pop3_ssl"
    - "traefik.tcp.services.mailu_pop3_ssl.loadbalancer.server.port=995"

    - "traefik.tcp.routers.mailu_imap.rule=HostSNI(\`*\`)"
    - "traefik.tcp.routers.mailu_imap.entrypoints=imap"
    - "traefik.tcp.routers.mailu_imap.service=mailu_imap"
    - "traefik.tcp.services.mailu_imap.loadbalancer.server.port=110"

    - "traefik.tcp.routers.mailu_imap_ssl.rule=HostSNI(\`*\`)"
    - "traefik.tcp.routers.mailu_imap_ssl.entrypoints=imap-ssl"
    - "traefik.tcp.routers.mailu_imap_ssl.tls.passthrough=true"
    - "traefik.tcp.routers.mailu_imap_ssl.service=mailu_imap_ssl"
    - "traefik.tcp.services.mailu_imap_ssl.loadbalancer.server.port=993"

``

Additionally I use ldez/traefik-certs-dumper docker image to extract my letsencrypt certificate from traefik's file storage (acme.json) to file system so mailu is able to reuse it for TLS.

Hope this is useful for anyone ...

qm78 picture qm78 on 30 Jun 2019
👍7 👀3 👎2 ❤1

Thanks, but how will Traefik handles non-TLS and TLS connections on the same TCP ports, eg. 25? SMTP on TCP 25 may have STARTTLS or not. 465 is never used for SMTPS. Traefik configuration for port 25 looks like unsecure.

Historically, port 465 was briefly registered as the "smtps" port.
This registration made no sense, as the SMTP transport MX
infrastructure has no way to specify a port, so port 25 is always
used.

helmut72 picture helmut72 on 29 Aug 2019

@helmut72 sadly traefik cannot be used as a reverse proxy for none secure connections, i've looked into that. however it does allow you to simply route any connection on that port, to your machine. (i.e. no reverse proxy feature on none-tls)

Asherslab picture Asherslab on 29 Aug 2019

Thanks, I guessed that. Then, Traefik is useless for my (or any?) mail server setup.

helmut72 picture helmut72 on 29 Aug 2019

@helmut72 after rereading what you've said, i'm not entirely sure. i believe you most likely will be able to do it fine, just no reverse proxy features. however if i'm not mistaken most services won't be able to do that either

Asherslab picture Asherslab on 29 Aug 2019

What we like about nginx is its ability to run some logic, for instance authentication logic. We cannot fully replace it with traefik for that reason.

That said your configuration uses the frontend container only and I like that. I would maybe prefer that it let the front container handle statttls for ports 25 and 143, so we still would need a way to provide nginx with the cert.

kaiyou picture kaiyou on 29 Aug 2019

@helmut72 not sure if I got you:

telnet mail.domain.tld 25
ehlo xxx

openssl s_client -starttls smtp -crlf -connect mail.domain.tld:25
ehlo xxx

Both works for me. Traefik forwards mail traffic to mailu. There had been this issue https://github.com/containous/traefik/issues/4875 regarding MTA, I reported to traefik, which has been fixed.

@kaiyou: I use traefik together with letsencrypt to terminate https connections. To provide certs for mailu's nginx frontend I use https://github.com/ldez/traefik-certs-dumper. Unfortunately traefik's file cert store (acme.json) format has changed in traefik 2.x and there's open issue https://github.com/ldez/traefik-certs-dumper/issues/34 waiting to be fixed.

qm78 picture qm78 on 1 Sep 2019

@qm78
Maybe I misunderstood the configuration:

openssl s_client -starttls smtp -crlf -connect mail.domain.tld:25
ehlo xxx

Which daemon handles the certificate, Traefik or Postfix?

From the feature lists, Traefik 2.0 looks like it can avoid this type of configurations:

To provide certs for mailu's nginx frontend I use https://github.com/ldez/traefik-certs-dumper.

... because Traefik 2.0 is the one and only place for certificates.

helmut72 picture helmut72 on 1 Sep 2019

@helmut72:

  • traefik terminates encryption only for http/https protocol. Plain http is redirected to https within traefik.

  • Traefik forwards / passes through all mail related traffic (smtp, pop3, imap - ssl/tls) to mailu -> so mailu's nginx frontend container needs to know the certificate as well not just traefik alone. The little hacky part is to extract (in my case letsencrypt) certificate from traefiks custom file cert store (acme.json) to file system to make it available to mailu's nginx frontend container. For this I use ldez's traefik-certs-dumper. I wished there would be an additional file store option in traefik to store certs in an exploded file system hierarchy for exactly that use case ...

qm78 picture qm78 on 1 Sep 2019

Thanks for clarify. Then I need to distribute the certificates from the reverse proxy if need them somewhere else, regardless if I use Traefik 2.0 or my current solution.

helmut72 picture helmut72 on 1 Sep 2019

fyi: ldez/traefik-certs-dumper docker image has been updated to version 2.5.4 and now supports traefik 2.0 acme.json format - LGTM.

There's a new argument to support traefik 2.0 acme.json format: '--version v2'

  certdumper:
    image: ldez/traefik-certs-dumper:v2.5.4
    command: "file --watch --domain-subdir=true --version v2"
    volumes:
      - /opt/docker.swarm/traefik/acme.json:/acme.json:ro
      - certs:/dump
    deploy:
      replicas: 1
qm78 picture qm78 on 7 Sep 2019
👎1

@Pewsplosions probably related to https://github.com/ldez/traefik-certs-dumper/issues/46

Edit: Fixed in 2.5.5.

Edit 2: Not quite fixed yet.

timoschwarzer picture timoschwarzer on 30 Sep 2019
👍1

@Pewsplosions now it is fixed ;)

timoschwarzer picture timoschwarzer on 30 Sep 2019
🎉1

@qm78 why do you pass through all the traffic encrypted? Couldn't you decrypt the TLS traffic and just forward the unencrypted traffic?

jcgruenhage picture jcgruenhage on 6 Dec 2019

@jcgruenhage Traefik terminating TLS connections and forwarding unencrypted traffic to MTA (postfix) didn't work. I thinks its because of smtp protocol handchake expecting "STARTTLS tag" to start an TLS encrypted connection to MTA.
Conceptional wise also my first idea was to let Traefik handle all SSL/TLS encrypted traefik, which works fine for HTTP traefik, but doesn't work for MTA TLS case.
I've tested both scenarios with mxtoolbox mail server test and only above described solution worked.

I'm open for alternative solutions as I'd appreciate to handle all SSL/TLS traffic from Traefik only!

qm78 picture qm78 on 7 Dec 2019

I thinks its because of smtp protocol handchake expecting "STARTTLS tag" to start an TLS encrypted connection to MTA.

But does the MTA even need to know that the transmission was TLS encrypted? Traefik could (to the best of my knowledge) just hand over the unencrypted bits and postfix wouldn't need to care.

jcgruenhage picture jcgruenhage on 7 Dec 2019

@qm78

Traefik terminating TLS connections and forwarding unencrypted traffic to MTA (postfix) didn't work. I thinks its because of smtp protocol handchake expecting "STARTTLS tag" to start an TLS encrypted connection to MTA.

As far as I remember postfix is configured to only accept connections through the nginx container

norpol picture norpol on 7 Dec 2019

@jcgruenhage, @norpol

I remember now, norpol you're right all traffic is routed through mailu's frontend container and according to it's nginx configuration terminating SSL/TLS mail traffic I had to forward mail traffic encrypted through traefik. It should be possible to overwrite nginx config to remove SSL/TLS traffic termination and to experiment with traefik to handle SSL/TLS. I'd be curious if this will work...

qm78 picture qm78 on 10 Dec 2019

WARNING: Examples in this thread have typos and will not work!
Besides, these examples are incomplete.
Far more work is needed in order to have this working.

frgomes picture frgomes on 4 Feb 2020

@frgomes could you suggest some changes and maybe PR against the documentation in the repo?
Otherwise I suggest we close this issue and open some more specific ones about specific problems of the traefik documentation.

kaiyou picture kaiyou on 7 Feb 2020

@qm78 Would you mind also sharing your traefik.yoml or yaml file? I would ove to see how you fully configured this. I am looking at doing the same thing with my Zimbra server. Thanks.

rpelletierrefocus picture rpelletierrefocus on 5 Mar 2020

@qm78 Could you elaborate on how you configured this completely?

krjw picture krjw on 14 Apr 2020

How do you secure your connections with eg. fail2ban? In the mail.log I only see Traefiks IP as source IP. It's useless to block Traefik if there are to many failed logins.

helmut72 picture helmut72 on 14 Jul 2020

Hi There,

The Mailu-Project is currently in a bit of a bind! We are short on man-power, and we need to judge if it is possible for us to put in some work on this issue.

To help with that, we are currently trying to find out which issues are actively keeping users from using Mailu, which issues have someone who want to work on them — and which issues may be less important. These a less important ones could be discarded for the time being, until the project is in a more stable and regular state once again.

In order for us to better assess this, it would be helpful if you could put a reaction on this post (use the :smiley: icon to the top-right).

  • 👍️ if you need this to be able to use Mailu. Ideally, you’d also be able to test this on your installation, and provide feedback …
  • 🎉 if you find it a nice bonus, but no deal-breaker
  • 🚀 if you want to work on it yourself!
    We want to keep this voting open for 2 weeks from now, so please help out!
Nebukadneza picture Nebukadneza on 18 Aug 2020
👍12 🎉5

Just Setting up a new Swarm cluster. I could test it with Traefik 2.

romanstark picture romanstark on 1 Sep 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] picture stale[bot] on 23 Sep 2020

Bumping this as I'm currently blocked with cert renewal failures and have decided that putting Mailu behind my current Traefik is the only path forward...

fracture-point picture fracture-point on 28 Oct 2020

Alright I strong-armed my way through it. I successfully have mailu running behind Traefik. Traefik handles all the inbound connections (mailu has zero world-accessible ports) and forwards all traffic to the NGINX proxy that mailu uses. Traefik generates the certs and they get dumped to NGINX by the cert-dumper. Traefik only terminates SSL/TLS for the https connections.

I'm going to core dump while it's fresh, and hopefully folks find this helpful. NOTE - I have this running on a pi, so you'll notice the images I'm using are a fork for ARMv7

Lessons learned:

  1. As far as I can tell, you MUST enable tls.passthrough on 465, and NOT enable it on 25 or 587, otherwise that breaks STARTTLS.
  2. The HostSNI(*) references for the TCP routers do not need a domain name, since it's TCP (different level of the stack)
  3. Adding multiple domains separated by a comma (as is typical for Traefik) to the http routers Host() rule creates a mismatch when cert-dumper tries to find the right file. Maybe you can get it to work, but I couldn't.
  4. NGINX needs to be in the traefik.docker.network (web, in my case).

Here's the modified front container in the mailu docker-compose.yml:

        front:
            image: leolivier/mailu_nginx:rpi-armv7
            restart: always
            env_file: mailu.env
            logging:
            driver: json-file
            # ports:
            # - "${ENTRYPOINT_IP}:80:80"
            # - "${ENTRYPOINT_IP}:443:443"
            # - "${ENTRYPOINT_IP}:25:25"
            # - "${ENTRYPOINT_IP}:465:465"
            # - "${ENTRYPOINT_IP}:587:587"
            # - "${ENTRYPOINT_IP}:110:110"
            # - "${ENTRYPOINT_IP}:995:995"
            # - "${ENTRYPOINT_IP}:143:143"
            # - "${ENTRYPOINT_IP}:993:993"
            labels:
                - "traefik.enable=true"
                - "traefik.docker.network=web"

                - "traefik.http.routers.mailu_http.rule=Host(`${WEBMAIL_DOMAIN}`)"
                - "traefik.http.routers.mailu_http.entrypoints=web"
                - "traefik.http.routers.mailu_http.service=mailu_http_and_https"

                - "traefik.http.routers.mailu_https.rule=Host(`${WEBMAIL_DOMAIN}`)"
                - "traefik.http.routers.mailu_https.entrypoints=websecure"
                - "traefik.http.routers.mailu_https.tls.certresolver=mytlschallenge"
                - "traefik.http.routers.mailu_https.tls=true"
                - "traefik.http.routers.mailu_https.service=mailu_http_and_https"

                - "traefik.http.services.mailu_http_and_https.loadbalancer.server.port=80"
                - "traefik.http.services.mailu_http_and_https.loadbalancer.server.scheme=http"

                - "traefik.tcp.routers.mailu_smtp_relay.rule=HostSNI(`*`)"
                - "traefik.tcp.routers.mailu_smtp_relay.entrypoints=smtp-relay"
                - "traefik.tcp.routers.mailu_smtp_relay.service=mailu_smtp_relay"
                - "traefik.tcp.services.mailu_smtp_relay.loadbalancer.server.port=25"

                - "traefik.tcp.routers.mailu_smtp_ssl.rule=HostSNI(`*`)"
                - "traefik.tcp.routers.mailu_smtp_ssl.entrypoints=smtp-ssl"
                - "traefik.tcp.routers.mailu_smtp_ssl.tls.passthrough=true"
                - "traefik.tcp.routers.mailu_smtp_ssl.service=mailu_smtp_ssl"
                - "traefik.tcp.services.mailu_smtp_ssl.loadbalancer.server.port=465"

                - "traefik.tcp.routers.mailu_smtp.rule=HostSNI(`*`)"
                - "traefik.tcp.routers.mailu_smtp.entrypoints=smtp"
                - "traefik.tcp.routers.mailu_smtp.service=mailu_smtp"
                - "traefik.tcp.services.mailu_smtp.loadbalancer.server.port=587"

                - "traefik.tcp.routers.mailu_pop3.rule=HostSNI(`*`)"
                - "traefik.tcp.routers.mailu_pop3.entrypoints=pop3"
                - "traefik.tcp.routers.mailu_pop3.service=mailu_pop3"
                - "traefik.tcp.services.mailu_pop3.loadbalancer.server.port=110"

                - "traefik.tcp.routers.mailu_pop3_ssl.rule=HostSNI(`*`)"
                - "traefik.tcp.routers.mailu_pop3_ssl.entrypoints=pop3-ssl"
                - "traefik.tcp.routers.mailu_pop3_ssl.tls.passthrough=true"
                - "traefik.tcp.routers.mailu_pop3_ssl.service=mailu_pop3_ssl"
                - "traefik.tcp.services.mailu_pop3_ssl.loadbalancer.server.port=995"

                - "traefik.tcp.routers.mailu_imap.rule=HostSNI(`*`)"
                - "traefik.tcp.routers.mailu_imap.entrypoints=imap"
                - "traefik.tcp.routers.mailu_imap.service=mailu_imap"
                - "traefik.tcp.services.mailu_imap.loadbalancer.server.port=143"

                - "traefik.tcp.routers.mailu_imap_ssl.rule=HostSNI(`*`)"
                - "traefik.tcp.routers.mailu_imap_ssl.entrypoints=imap-ssl"
                - "traefik.tcp.routers.mailu_imap_ssl.tls.passthrough=true"
                - "traefik.tcp.routers.mailu_imap_ssl.service=mailu_imap_ssl"
                - "traefik.tcp.services.mailu_imap_ssl.loadbalancer.server.port=993"
            volumes:
            - "./traefik_cert_dump:/certs"
            - "./overrides/nginx:/overrides"
            networks:
            - web
            - default

Note that I used a new directory for the dumped certs, so as to preserve the previously-generated mailu certs for fallback.

Here's the cert-dumper that I added to the mailu docker-compose.yml:

certdumper:
    restart: always
    image: leolivier/mailu_traefik-certdumper:rpi-armv7
    environment:
    - DOMAIN=${MAIL_SERVER_DOMAIN}
    - TRAEFIK_VERSION=${TRAEFIK_VERSION}
    volumes:
        # Folder, which contains the acme.json
        - "/traefik/letsencrypt:/traefik:ro"
        # Folder, where cert.pem and key.pem will be written
        - "./traefik_cert_dump:/output"

I have a .env file in the mailu root directory, that enables the variables above to be interpolated, e.g.

WEBMAIL_DOMAIN=somewhere.com
MAIL_SERVER_DOMAIN=mail.somewhere.com
TRAEFIK_VERSION=v2

The Traefik docker-compose.yml needed these components added to their respective sections:

    command:
      - "--entrypoints.smtp-relay.address=:25"
      - "--entrypoints.smtp-ssl.address=:465"
      - "--entrypoints.smtp.address=:587"
      - "--entrypoints.pop3.address=:110"
      - "--entrypoints.pop3-ssl.address=:995"
      - "--entrypoints.imap.address=:143"
      - "--entrypoints.imap-ssl.address=:993"
    ports:
      - "${ENTRYPOINT_IP}:25:25"
      - "${ENTRYPOINT_IP}:465:465"
      - "${ENTRYPOINT_IP}:587:587"
      - "${ENTRYPOINT_IP}:110:110"
      - "${ENTRYPOINT_IP}:995:995"
      - "${ENTRYPOINT_IP}:143:143"
      - "${ENTRYPOINT_IP}:993:993"
      - "${ENTRYPOINT_IP}:80:80"
      - "${ENTRYPOINT_IP}:443:443"

Finally, I modified TLS_FLAVOR=letsencrypt to TLS_FLAVOR=mail in mailu.env, so that mailu uses the certs dumped from Traefik instead of fetching its own.

fracture-point picture fracture-point on 30 Oct 2020
👍4 🎉2
Was this page helpful?
0 / 5 - 0 ratings

Related issues

Diman0 picture Diman0  ·  3Comments
v1ru535 picture v1ru535  ·  4Comments
styxlab picture styxlab  ·  4Comments
c-holtermann picture c-holtermann  ·  3Comments
Thorsten1976 picture Thorsten1976  ·  4Comments