K-9: enhance k9 mail identity selection

[SR-gh]

Identity selection could enhanced to make life a lot easier.
Today, it is really painful as it is in most client email readers.

Identity selection should be :
-free to edit and modify when writing a message (more or less as the recipient field currently is)
-automatically set with the best value guessed from the recipient values, when replying.
This would be a great feature to differentiate K9 mail from other rigid email clients.

Why?

I use the catch-all of my domain to handle a lot of different addresses.
Catch-all is really a great feature as it allows me to give anyone a unique email address in order to send me an email.
Consequently, I can, if I receive spam on this address, deduce that this correspondant may have abused my personal information.
I also use it to differentiate between groups of interest, without having to mess with a lot of mailboxes or aliases on the server side, as it is a painful task that can never be done correctly and is completely useless.

The only problem I see is to correctly select the account to associate with the email to be sent.
It could be deduced from the receiving account, when replying. Just use the same.
When not replying, either you are in an account view, and it is this account, or you are in the unified view, and this where creativity lies. Simplest solution would be to use the default account, but in my case, it is not the catch-all, so it's not that simple. I would make it this way : if the identity is the one associated with an account, use that account. If the identity is not the one associated with one account, use the 'default unassociated identity account', which is, in my case, the catch-all. In the end, the list-based feature can be used to force any configured account to be used.

This way, current behaviour does not change, but is extended and a lot more flexible than a fixed list-based feature.
Just imagine such a list-based feature for the recipient field in order to see what I mean.

[odbol]

I also want the second idea "automatically set with the best value guessed from the recipient values, when replying." I think.

However, I think that's almost a separate issue, so I've made it one: issue #1064

Since that's now a separate issue, is this issue closeable?

[johey]

I am also very interested in this feature. Been using k9 on a daily basis for six years and this is the only feature I'm missing.

For Thunderbird, there is a very well working plugin called Virtual Identity. Perhaps one can find some inspiration from that?

[ArchangeGabriel]

@johey For Thunderbird, this is a default feature from some release ago, without the need for an old and abandoned plugin. ;)

Anyway, this feature would be nice indeed for K-9. :)

[saintger]

@ArchangeGabriel Thunderbird doesn't allow to use a sender's email if it has not been previously added as Identity. Whereas Virtual Identity allow to compose/reply without creating Identities.

For instance emails like [email protected] and [email protected] can be forwarded to [email protected] (for tracking purpose). Now if you hit the "Reply", Virtual Identity can be set to use [email protected] or [email protected] as sender's email without having to create the Identity first.

Therefore I agree with @johey, Virtual Identity could be a good starting point.

[ArchangeGabriel]

@saintger No, Thunderbird does allow you to do so since 45.0 (last “New” item from here), and that was precisely the point of my previous comment. Just take a look at the last entry from the drop down menu for the From (or whatever it is in your locale) field.

However, I agree that it doesn’t provide the “Smart Reply” thing from Virtual Identity, which escaped from my previous reading of this issue.

[saintger]

@ArchangeGabriel Indeed you can compose without adding the identity first, I missed that as well.
However 'Virtual Identity' is not enough.

For instance I have several email ([email protected] and [email protected]) all going to the same email account ([email protected]) and I would like to reply only with [email protected].
(it often happen when you migrate from one email provider to another)

In this case I have to use the Thunderbird 'Correct Identity' to associate [email protected] with [email protected].

Such advanced identity selection is then useful in case of email migration and for fighting spam/tracking (see plus addressing and subdomain addressing)

[ArchangeGabriel]

@saintger In this case, you just have to not configure [email protected] identity at all and it would work, wouldn’t it?

[saintger]

@ArchangeGabriel It works if you have only one identity. With several identities, Thunderbird has no way to guess the mapping (for instance I also have the identity [email protected] and so on...)

[ArchangeGabriel]

Ah, yes, didn’t thought of the case of multiple identities for one account + other identities related to this account but not registered in the settings. Doesn’t Thunderbird uses the default identities of the account in this case? This could be enough if you want to answer all non-registered identities with only one other address, but I agree this doesn’t solve all use cases.

[cketti]

@saintger @ArchangeGabriel: Please don't spam this issue by discussing the finer points of another implementation. You've stated your use cases.

[Dash]

Was going to raise this myself, glad to see somebody else has. It seems fairly common practice to have floating @mydomain for signing up to stuff.

The Thunderbird plugin is fairly heavy on the implementation and could probably be simplified.

It's worth noting that things can get confusing if you're using distribution lists or BCC, as the "to" field is sometimes not your address.

I would suggest in the event that the To: field does not match a pre-configured identity, then you have the option of selecting any To: or Cc: address when you press on the From: header in the compose screen.

I think you would need to include a final option for "Enter custom..." or similar if none are applicable.

In the event you have a frequently used situation where you need the same custom entry on a regular basis you can use the existing identity functionality to provide a pre-configured identity.

[danwdart]

Even if the To: does not match your domain, you can still check out another header (was it Delivered-To:? Not sure) which is the reason the list server routed the message to you. Gmail does this effectively with lists.

[Dash]

Indeed. You can have Envelope-To: and Delivered-To: too. I find that Delivered-To:

Delivered-To: is generally the final mailbox, so for me it's @localhost
Envelope-To: is sometimes included, I find that this is usually the single mail-box that my catch-all forwards to.

You can also mine the Received: headers, as sometimes you will have "from X by Y for user@domain".

Seeing the _original & actual_ email destination, is not always feasible. Which is why whatever is done, a free-text entry is a must for this feature to be useful. Address selection is almost a nice-to-have in comparison :)

[johey]

Just being able to manually type in the From: address on sending an email would be good enough for me. Finding out the proper From: on reply would be a nice to have addition that could be saved for later.

[atfrase]

I was about to file this request as well, glad to see it's already been raised.

One suggestion I'd like to add is to simply enhance the "Email address" field of the Identity editor to accept wildcards, such as "*@me.example.com". Then if mail arrives for "[email protected]" replying to it will match that wildcard identity and populate the sender as "[email protected]", and likewise for "[email protected]" or "[email protected]". This same scheme should also cover plus addressing, such as "me+*@gmail.com".

Making the From field editable would be nice as well, but I think simply allowing wildcard matching for existing, configured identities would cover 90% of these use-cases without requiring the user to manually type in their preferred From address based on the received To address.

[deutrino]

The lack of this is by far my biggest pain point with this app. I have multiple identities at multiple domains and I also use plus addressing to sign up on web sites. I have been looking for a privacy conscious Android app with good identity handling for literal years. See also #2155.

[ryancdotorg]

Mutt has this feature via alternates, reverse_name and reverse_realname. See documentation here: https://gitlab.com/muttmua/mutt/wikis/UseCases/MultiAccounts - would be great to have this in k9mail.

Roundcube also has it via a plugin: https://github.com/r3c/custom_from

[ottlinger]

Not sure whether it's a new issue or an add-on for #935 but I'd want a more advanced dialogue box for new identities that allow me to set a separate SMTP server for a given identity.

I have this setup with Thunderbird as I have a "big" IMAP account that is used as a forwarding from multiple addresses. As my hoster changed the spam policy to only allow sending mails from [email protected] only with a valid SMTP-user [email protected]. Thus I'd want to set a SMTP server for each identity to allow me sending from [email protected] (my main entity, configured with its IMAP and SMTP server) and an additional identity [email protected] using a different SMTP user in order to sent mails from that recipient address).

Shall I file new issue or is this the correct place for that? @cketti

Thanks for any comments, ideas, improvements.

[ryancdotorg]

Is there any way I can throw a bounty on this?

[madduck]

@ryancdotorg Have a look at IssueHunt or Bountysource. Throwing a bounty at something isn't that hard. The difficulties usually arise when an issue's solution is the result of multiple people working together, or when the scope of the issue isn't well-defined beforehand, and expectations don't get met. So be careful and as forward and transparent as you can be.

[goddib]

@ottlinger it seems your issue alligns more with https://github.com/k9mail/k-9/issues/1117

[edmundlaugasson]

Using v5.600 and identity selection does not work - always sending out with default identity and not using the one selected while sending the email - recipient receives email always with default identity and not that one what was selected while sending email.
Also when looking in computer with Thunderbird into sent items and opening the sent email then it shows default identity and not what I selected in phone while sending it.
This makes additional identities useless as they do not work. Actually this was the main argument I started to use the app.
Sad to read that since the current issue opened (7 Dec 2015) it has not been fixed....
I would not call current issue enchancement but bug as the main functionality - identity usage does not work.

[ryancdotorg]

Okay, bounty has been fixed and now applies the this github issue.

https://www.bountysource.com/issues/28901565-enhance-k9-mail-identity-selection

[simontb]

This is something I'm interested in as well and I'm considering to implement it.

Had a brief look at the code already and noticed that the "from" address is already filled, if an existing identity in k-9 matches one of the "to" fields of the original mail: https://github.com/k9mail/k-9/blob/b9eba6971fa86b95a4855caddbfa7144e5c23884/app/ui/legacy/src/main/java/com/fsck/k9/activity/MessageCompose.java#L1281-L1287

So to summarize what is missing from my point of view:

• An option to enable editing the "from" field in the compose screen manually. Regarding UX when this option is enabled I propose to add an option to the "send as" menu to every account that has this option enabled and when this new option is selected, the user can edit the "from" field in the compose screen. This way we know which server to use. If we allowed editing in the compose screen from the beginning, this would be complicated to determine in case a user has multiple accounts.
• Advanced matching for the "from" adress. The user can configure an account to be the domain owner in which case any address with that domain in the "to" fields of the original mail can be used as "from" in the reply, regardless of whether it's an identity in k-9 or not.

@cketti is this roughly how it should be implemented?

[Dash]

Mozilla are going through a similar thing at the moment with Thunderbird, might be worth reviewing their change to see the considerations they made:
https://bugzilla.mozilla.org/show_bug.cgi?id=1518025

[cketti]

@simontb: Thanks for volunteering to work on this :+1:

The ability to use a custom "From" address shouldn't be a general setting. Instead, I want users to be able to mark an existing identity as having a catch-all address. Then tie the ability to provide a custom address to the existence of such an identity.

Here's a rough outline of what I think should be done. All of these should probably be separate pull requests:

• Add a way to mark an identity as using a catch-all address (checkbox in EditIdentity?)
• Add a way to edit the sender address when composing a message (maybe add a pen icon to catch-all identities and only allow users to provide the local part [+ name?])
• Add support for catch-all identities when finding the identity to use for replying to a message. Use the address from the message instead of the (default) one from the identity.
• Add support for importing/exporting the "catch-all" identity property

Because this feature involves a couple of changes I think we should add a BuildConfig flag to disable it for release builds while it is in development. It's probably enough to hide the additional UI when the feature is disabled. That way we can continue shipping beta versions without exposing a half-finished feature.

The smaller files this feature touches should probably be converted to Kotlin before making any changes to them (EditIdentity, IdentityAdapter, etc).

~I haven't had time to read through the Thunderbird issue. From the title it appears they use a similar approach. If not we should consider copying their approach (I'd love for someone to summarize the Thunderbird issue). At the very least we should use similar wording for the UI parts.~ I found some time to read the Thunderbird issue. It looks like they've conflated two features and created a very confusing UI in the process. Please disregard everything they did :)

For now I've only considered catch-all addresses like *@domain.example. Eventually, we want to handle "plus addresses" ([email protected]), too. Suggestion on how the user interface for this should look like are welcome.

[ottlinger]

@cketti my personal use case and need for this feature is a bit different.
I have a big IMAP post box that catches mail from various domains:

• [email protected]
• [email protected]

Let's say [email protected] is the IMAP post box that catches mail for [email protected] I would like to be able to have different SMTP settings in order to answer as [email protected] although from a technical perspective the K9-identity is [email protected].

With Thunderbird I'm able to provide SMTP-details for each identity except from just setting the from-address ... does that make the use case clearer? It's not just plus addresses but different domains.

[cketti]

@ottlinger: That is a completely different feature request. It is tracked under #1117.

[tyllmoritz]

I wrote a post in the forum.
Please share your thoughts.

https://forum.k9mail.app/t/suggestion-placeholder-in-sender-use-one-identity-for-many-aliases/283

[dotancohen]

tyllmoritz's forum post describes the use of an unquoted space character, which is illegal in an email address, to be used as a placeholder in the configured From email address. I happen to think that this is a great idea.

For instance, all my outgoing mail will be <something>@dotancohen.com. So if @dotancohen.com were configured as a default template for the outgoing From address, then not only would I save time for each mail, I would avoid typos.

[cketti]

Tying functionality to a space in an email address very much falls into the "confusing UI" category. I don't like that idea.

[tyllmoritz]

Tying functionality to a space in an email address very much falls into the "confusing UI" category. I don't like that idea.

You're right, it would be confusing.

A slightly-less-confusing UI could look like this:
When editing an identity, you can enter an E-Mail address.
On the right side next to the input field for the email-address there could be three "setting-dots".
On click on these "setting-dots, two settings are available:

• "arbitrary sender address"
• "arbitrary sender address (partially)"

the first option disables the E-Mail address input field and allows to freely edit the from-field when composing an email.
the second option inserts <placeholder>into the input field, which is internally represented as an ascii control character, for example the space character or the substitute character.

As the second option is much more work, I'd be more than happy to just have the first option.

[ghost]

Please tell me how to combine identity and autocrypt!

[cketti]

@seppwarum: Please use the forum for support requests. If you want anyone to be able to answer your question you'll have to provide more context.

[ghost]

Yes, identities make life a lot easier, but only if you don't use OpenKeychain to encrypt messages.
Each identity created represents an address and needs its own key in OpenKeychain
(Thunderbird allows it to give each identity its own key).
Please remember this, when enforcing identities!

[dotancohen]

Tying functionality to a space in an email address very much falls into the "confusing UI" category. I don't like that idea.

Though I agree with you in general, in this case it doesn't actually have to be confusing.

The space is not a legal character in an email address, so it should fail validation when trying to send an email. So far so good.

Now, just do not forbid the space character in the verification for setting the default email address. If the user enters a valid email address in the default address field, he will be able to send emails without editing the FROM field. If the user enters a space, then when he tries to send an email he is prevented from doing so by the failed validation, reminding him to update his FROM address.

This would be a power user feature and completely transparent to those who do not need it.