Dataverse: strange formatting in "to" field of Dataverse-sent email

[pameyer]

Noticed while troubleshooting system failure to send email (which was unrelated, and is resolved).

For a user with an email [email protected] and mail.hmdc.harvard.edu as the email host, to field of Dataverse-generated email is "[email protected]"@mail.hmdc.harvard.edu (for reference, these are dummy values - this was not identified with mail.hmdc.harvard.edu).

[pameyer]

Did a little more investigation - judging by my email archive of test emails, this changed between Jun 2017 and Aug 2017. Don't have any specific branch or commit info, but probably reasonable close to develop.

[pdurbin]

Also reported by @shlake at https://groups.google.com/d/msg/dataverse-community/lUyKO6iY6oE/BYVHRV42BAAJ

"The time stamp on one of the emails is during a time I was testing 4.8.1. So not sure if it was 4.8 or 4.8.1 that caused the change in "To:" address. But it is still there in 4.8.3"

[pameyer]

Look a brief look while waiting for VM provisioning. Additional quotes aren't coming from authenticateduser.email in postgres. Potentially related to recipients[i] = new InternetAddress('"' + recipientStrings[i] + '"', "", charset); in sendSystemEmail in MailServiceBean.java; commit messages suggest changes there around a time window matching my test email archive.

[bricas]

We're also seeing this issue with "[email protected]"@host18.example.com as the output. We're running Dataverse 4.8.5.

[pdurbin]

@bricas @pameyer and I have been chatting a bit about this bug this morning at http://irclog.iq.harvard.edu/dataverse/2018-02-12

I'm easily able to replicate this bug on https://demo.dataverse.org running 4.8.4. At first I thought email wasn't being sent at all but the welcome email went to my spam folder and here's the "To:" line

To: <"[email protected]"@ip-172-31-60-234.ec2.internal>

Very strange. At least I received it but I wouldn't be surprised if messages in this odd format are more likely to be flagged as spam. Here's a screenshot:

It's unknown when this bug was introduced.

[pdurbin]

To anyone attempting to fix this bug, I would suggest trying to figure out if the bug was introduced in pull request #4009 for issue #2998 because that pull request was merged as part of 4.8 which is roughly when we starting hearing reports of this bug.

[djbrooke]

Hey @bricas - @pdurbin mentioned this as a possible candidate for backlog grooming today, but I used our regular grooming time to set up a meeting to get feedback from a local scientist that will be using the Provenance features we're building. I put this issue into the backlog so we'll take a look at it during the next estimation session that we have.

[pdurbin]

@qqmyers thanks for making pull request #4491 !

I just dragged this issue to Code Review at https://waffle.io/IQSS/dataverse

[qqmyers]

Glad to help. FWIW- I looked at issue #2998 and pull #4009 - it looks like all the validation is applied to email addresses as they're sent, so having extra quote marks that were applied at send time isn't caught. Likewise, removing them doesn't affect the added validation.

[pdurbin]

@qqmyers thanks. As I just indicated in my code review, it sounds like you've done a good amount of testing. The change is small (deleting a few characters) and makes sense so I moved your pull request (#4491) to QA for testing. I really appreciate you digging in and tracking down the line that needed to be fixed!

@pameyer if you have cycles, maybe you could test the fix as well.

[andrewSC]

For anyone using Mailgun and are seeing the following in their logs when trying to send mail:

  javax.mail.SendFailedException: Invalid Addresses;
  nested exception is:
        com.sun.mail.smtp.SMTPAddressFailedException: 501 Invalid command or cannot parse to address
        ...

This issue is related and you can either wait for the next release or remove the quotes and re-package/re-deploy the war file.