Postal: Hide source host via SMTP

[iammattmartin]

For security, when submissions are by SMTP, it would be nice if we can hide the source server in the headers seen by those receiving email.

We already know which app sent the mail and from what IP but just goes some way to protect internals etc.

[deanpcmad]

How do you mean?

Here's the raw message from my postal server to my email account on FastMail. I've changed some sensitive parts.

What areas wouldn't you want hidden? Most of X- headers are from FastMail & their spam system. I believe the only header Postal adds is X-Postal-MsgID

Return-Path: <[email protected]>
Received: from compute3.internal (compute3.nyi.internal [10.202.2.43])
     by sloti36d2t23 (Cyrus fastmail-fmjessie43991-14970-git-fastmail-14970) with LMTPA;
     Wed, 03 May 2017 13:26:34 -0400
X-Cyrus-Session-Id: abc123
X-Sieve: CMU Sieve 3.0
X-Spam-known-sender: no
X-Spam-score: 0.0
X-Spam-hits: BAYES_00 -1.9, HEADER_FROM_DIFFERENT_DOMAINS 0.001, RP_MATCHES_RCVD -0.001,
  SPF_HELO_PASS -0.001, SPF_PASS -0.001, LANGUAGES en, BAYES_USED global,
  SA_VERSION 3.4.0
X-Spam-source: IP='139.162.254.1', Host='my-postal-server.com', Country='GB', FromHeader='com',
  MailFrom='com'
X-Spam-charsets: plain='UTF-8'
X-Resolved-to: [email protected]
X-Delivered-to: [email protected]
X-Mail-from: [email protected]
Received: from mx4 ([10.202.2.203])
  by compute3.internal (LMTPProxy); Wed, 03 May 2017 13:26:34 -0400
Received: from mx4.messagingengine.com (localhost [127.0.0.1])
    by mailmx.nyi.internal (Postfix) with ESMTP id 6FB74C8760
    for <[email protected]>; Wed,  3 May 2017 13:26:34 -0400 (EDT)
Received: from mx4.messagingengine.com (localhost [127.0.0.1])
    by mx4.messagingengine.com (Authentication Milter) with ESMTP
    id ACE0624242A;
    Wed, 3 May 2017 13:26:34 -0400
Authentication-Results: mx4.messagingengine.com;
    dkim=pass (1024-bit rsa key) header.d=voupe.com [email protected] header.b=dVO5hR0N;
    dmarc=none (p=none) header.from=voupe.com;
    spf=pass [email protected] smtp.helo=my-postal-server.com
Received-SPF: pass
    (rp.my-postal-server.com: 139.162.254.1 is authorized to use '[email protected]' in 'mfrom' identity (mechanism 'mx' matched))
    receiver=mx4.messagingengine.com;
    identity=mailfrom;
    envelope-from="[email protected]";
    helo=my-postal-server.com;
    client-ip=139.162.254.1
Received: from my-postal-server.com (my-postal-server.com [139.162.254.1])
    (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
    (No client certificate requested)
    by mx4.messagingengine.com (Postfix) with ESMTPS
    for <[email protected]>; Wed,  3 May 2017 13:26:33 -0400 (EDT)
Resent-Sender: [email protected]
DKIM-Signature: 
X-Postal-MsgID: 123abc
Received: from localhost.localdomain (::ffff:163.172.220.87 [::ffff:163.172.220.87]) by my-postal-server.com with SMTP; Wed, 03 May 2017 17:26:31 +0000
Date: Wed, 03 May 2017 18:26:30 +0100
From: [email protected]
To: [email protected]
Message-ID: <[email protected]>
Subject: Password Reset
Mime-Version: 1.0
Content-Type: text/plain;
 charset=UTF-8
Content-Transfer-Encoding: 7bit

This is a test email

[iammattmartin]

I think it was clear.

hide the source server

The source server that starts the SMTP transaction in to postal.

[deanpcmad]

So let's say server1 sends an SMTP request to my Postal server, you want to hide that it came from server1? The only place that is shown is in the Message-ID

[iammattmartin]

Not for me it ain't - hence the question.

Received: from source.server (source.server [2001:67c:xx::xx]) by postal.server with SMTP; Wed, 03 May 2017 19:08:55 +0100

And as far as your headers show, yours does it too.

Received: from localhost.localdomain (::ffff:163.172.220.87 [::ffff:163.172.220.87]) by my-postal-server.com with SMTP; Wed, 03 May 2017 17:26:31 +0000

163.172.220.87 is your source sending to your postal instance on 139.162.254.1.

[deanpcmad]

Ah so it does. My mistake! I just assumed it was a FastMail IP because it says localhost.localdomain.

Looking through the SMTP client code I think this may be the area where it sets that: https://github.com/atech/postal/blob/2f0155747c5c058afb83856ce5c3c8d5f8cad341/lib/postal/smtp_server/client.rb#L319

[iammattmartin]

Commenting out the below seems to remove the source server to postal line.

# @data << "Received: #{received_header_content}\r\n"

But as we're queueing messages on a local server (in case postal is down etc) we still see that line.

Still reviewing but changing @receiving_headers to false broke SMTP entirely.

[catphish]

Commenting out this line is certainly the simplest solution. Possible enhancements here:

• Add option not to add received header
• Add option to add a valid but anonymous received header
• Add option to strip existing received headers
• Add option to anonymize existing received headers

Received headers can be used to prevent mail loops, so I would favour a combination of points 2 and 4 above, ensuring the headers can be used for loop detection but without revealing sensitive data.

[billyprice1old]

this do not work doing this

[mackinra]

How about these changes? They'll suppress any incoming Received headers, as well as the one Postal adds itself (that's discussed above). https://github.com/atech/postal/pull/282

[iammattmartin]

Does not seem to work for me.

[mackinra]

@mattmartincouk sorry, just added a description to the PR, where you'll need to add the strip_received_headers: true config line to the smtp_server section. And then restart postal.

[iammattmartin]

Did that, didn't work either. Seems to put headers in regardless.

[mackinra]

@mattmartincouk how are you delivering mail to Postal? These changes will only work if you're sending via SMTP. I haven't worked with the API method of sending yet.

[iammattmartin]

SMTP. The headers were no different with and without the patch.

[mackinra]

@mattmartincouk that's odd, and you restarted Postal after applying the changes I assume? It works great for me. Can you provide the smtp_server section of your config file?

[mackinra]

Can anyone else confirm whether or not my change works for them? Would like to have the changes merged so I can more easily stay up with updates.

[catphish]

I've merged this as I can't see any reason it wouldn't work. I'd be interested to know why it didn't work for @mattmartincouk though :(

[mackinra]

Me too. Thanks.

[iammattmartin]

I've just upgraded but no change here.

Using SMTP method, the headers are still there. :(

[catphish]

This definitely works for me. Are you able to run a rails console and confirm the value of Postal.config.smtp_server.strip_received_headers??

[iammattmartin]

irb(main):003:0> Postal.config.smtp_server.strip_received_headers?
=> true

Have double checked but source is still there.

Received: from internalhost.domain.com (::ffff:x.x.x.x. [::ffff:x.x.x.x.]) by outgoing.postal.domain with SMTP; Thu, 10 Aug 2017 01:20:15 +0100
Received: from internalhost.domain.com (localhost [127.0.0.1])
    by internalhost.domain.com (Postfix) with ESMTP id EC5A2800B3
    for <me@[me].com>; Thu, 10 Aug 2017 01:20:14 +0100 (BST)

We send to a local mail server that then authenticates to postal (to avoid any loss when postal is down or busy) not that this should matter.

[catphish]

Indeed that should not matter. Are you 600% sure you have restarted the SMTP server?

[iammattmartin]

Yes yes :(

Even restarted the server to rule everything out.

Will try a test build tomorrow, maybe something set in the config is causing some oddness.

[iammattmartin]

That said, looking at the code, I don't think it would strip out the indented lines that postfix uses for its additional received data. Probably unrelated but will likely mean when I work out what is going on, it will only strip the first two lines leaving the last two.

[mackinra]

It does for us, and we similarly use an external (postfix-based) email server that relays emails into Postal.

[catphish]

This should work. It's not greatly readable, but the below code will bypass the input being appended to the saved email if the current header name is "received". The current header name remains unchanged when processing continuation headers.

if Postal.config.smtp_server.strip_received_headers? && @header_key.downcase == "received"
  next
end

[iammattmartin]

Looks like a dodgy space in the config file was to blame :(

Although it is only stripping some of the postfix lines and actually, a malformed header now appears in the GUI headers.

x-postal-msgid seems to have received header stuff in it now, namely the below.

km7AsgEsvebT by sending.server (Postfix) with ESMTP id 3C7B2800B5 for <[email protected]>; Thu, 10 Aug 2017 01:35:56 +0100 (BST)

Here is it in context:

Resent-Sender: [email protected]
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=my.domain.com; s=appmail-rjjY1g; t=1502325356; bh=xxxxxx/DwxJ171AnmuIQ=; h=date:from:to:message-id:subject; b=uXKQFrptH2WUEd4lmLRnBOv7fgWwGPrC4830io/3TVlFbAcchVqeUUUFhTkW+KMBPmd30IPv7bSBn68FOS2d08zPJJTbI38+6CTxMkWWVAFKVrEWw4yrD8tKx8lstl9JKkcQYtS0sS6ARFyG+sThFzVwu/8xXbJ5CbMejlccUJU=
X-Postal-MsgID: LvnrYEtLOA2w
    by sending.server (Postfix) with ESMTP id 14F10800B3
    for <[email protected]>; Thu, 10 Aug 2017 01:35:56 +0100 (BST)
Date: Thu, 10 Aug 2017 01:35:56 +0100
From: x x <[email protected]>
To: x x <[email protected]>

[mackinra]

Hmm, will have a look at that... may have missed handling the continuation lines in Received headers.

[catphish]

Definitely a bug there. It looked correct to me, but I didn't actively test the continuation headers.

[catphish]

Oh, it doesn't skip on continuation headers at all, I missed that. I'l fix it.

[iammattmartin]

Thanks. :) Will disable for now.

[catphish]

Fixed a63c62ba6276be968a9a16fd6905c1b770cc4e2d

[iammattmartin]

Works perfectly... and back enabled. 👍

[mackinra]

I was about to propose something similar (outside of that if block): https://github.com/mackinra/postal/commit/c184cf6b4945923e8fca517a4504234f8c17a32d

[mackinra]

...but yours looks a bit safer. :) Thanks.

[catphish]

Yours would strip the if data.blank? case if it occurs immediately after a Received header. You don't want what :slightly_smiling_face:

[mackinra]

I was looking at that data.blank? condition, but yeah, excluding the blank line would be bad. :)

[iammattmartin]

Might have found an issue on this one @catphish (sorry to @ you, just this is closed).

It appears to do this to incoming messages too (i.e replies). Not sure if that can be avoided?

[catphish]

The current option will indeed strip headers in both directions. I can see that this is probably not always desirable. Please could you open a new issue to report this as a bug?