Suitecrm: Sent emails are not copied to the given 'Sent Folder'

[JanHubert]

• [ ] @Dillon-Brown please re-open the issue - updated screenshots and log files are provided below

Issue

E-mails sent by SuiteCRM are not copied to the 'sent folder' on an imap server. This affects composing, replying to and forwarding any email using the E-Mail module. All my users and all imap accounts are affected.

Expected Behavior

I expect the sent email to be copied to the chosen sent folder.

Actual Behavior

No email is copied to the sent folder

Possible Fix

Steps to Reproduce

1. Create an email account (IMAP) in SuiteCRM, choose Inbox and Sent Folder
2. Read your emails within SuiteCRM
3. Reply to any email (or compose a new one)
4. Check your 'Sent Folder' for the sent email using either SuiteCRM's E-Mail module or your favorite imap client.

Context

Your Environment

• SuiteCRM Version used: 7.10.4
• Browser name and version (e.g. Chrome Version 51.0.2704.63 (64-bit)): Version 63.0.3239.132 (Official Build) (64-bit)
• Environment name and version (e.g. MySQL, PHP 7): mysql and php7
• Operating System and version (e.g Ubuntu 16.04): deb9u2

suitecrm.log using DEBUG log level:
log starts exactly before sending the email (screenshot 3) and ends right after clicking the send button (i.e. even before accepting the 'Email sent' pop-up)
mail issue-suitecrm.log

[Mausino]

@JanHubert let add screen of second part of setting.. where is from and reply to...

if your email address is [email protected] and you will add to

FROM: file in email settings [email protected] and you send message... you will not able to find this message into sent folder because send folder is set up that will show you only messages for [email protected]

on this complain also my employees, that their are connect to our info@ and sending emails as name@ and cant find the sent emails in sent folder...

[JanHubert]

@Mausino Thanks a lot for your reply. I checked the folders in each of the mailboxes directly via imap/webmail. The sent email is neither in my mailbox(jan@) nor in any other mailbox (info@, service@, sales@). It is not shown anywhere in SuiteCRM which is probably by design as there is no corresponding email on the imap server.
In my understanding SuiteCRM should copy a sent email to the 'Sent Folder' of the account you are sending the email with. This seems to work for you guys, I cannot verify that behavior on the official demo instance though.

[maatohewetbi]

I have the same problem. Is anybody here that can tell it works?

[JanHubert]

I tried a lot but without any success. Testing it with an untouched re-installation of SuiteCRM is still on my todo list.

[Dillon-Brown]

I'm unable to replicate with 7.9.x or 7.10.x. Please re-open if this is still an issue you can replicate and provide clear steps for. Thanks.

[JanHubert]

Hi, I had the same symptom with a fresh copy of SuiteCRM. As our email and authorization infrastructure is currently changing, I deferred further testing. I will absolutely try to implement email functionality in our SuiteCRM system again and comment accordingly. Thx.

[segro21]

hi, we have the same setup (kopano 8.5.5 gatewa) suitecrm 7.10 and same behavior. sent mails are not copied to "Gesendete Objekte". with the old zarafa-gateway + suite 7.x (very old version) it was working fine... any change to get that running?

[JanHubert]

Hi, we are moving to Kopano next week. A few days later I will upgrade from the latest 7.9.x release to the then latest 7.10.x release. I will provide an update if it works or not.

[labcerouno]

The same here with v 7.10.4. The solution I found was to create a Report with all mails with State = Sent or Responded (these fields an values may have different names. I directly translated from Spanish)

[JanHubert]

Quick update: We are now using the latest version of Kopano and SuiteCRM version 7.10.4. Unfortunately the results are the same. Whoever sents an email, there is no copy stored in any imap folder. No errors in suitecrm.log or in the Kopano gateway log file.
I will provide logs, screenshots and re-open the issue. Maybe we'll jointly find a solution.

[kikobar]

Just installed a fresh SuiteCRM 7.10.7 and I observed the same issue, there is no copy of the sent email stored in the corresponding 'Sent Emails' folder.
I hadn't noted this issue before because I had been using Gmail as email server, which doesn't need a copy of the sent email to be stored because it keeps all the emails in a single folder (All Mail) with a flags system, but now I am using a standard IMAP server and bumped into this problem.

[simong1984]

I am also seeing this problem on 7.10.7 with Office365 IMAP accounts. The emails send correctly but are not stored in the selected Sent folder. We cannot use the CRM to send emails until this is sorted, which is a shame as it would be much more convenient.

[Mausino]

@pgorod @Dillon-Brown @PedroErnst do you thing that i can create same issue, which will referece to this one... becuase this issue is really problem for everyday using of SuiteCRM and as you see... still isn't fixed

[Dillon-Brown]

@Mausino I can just re-open this one, thanks!

[pgioia]

In Bitnami VM distribution with Versión 7.10.4
Sugar Versión 6.5.25 (Compilación 344),
the same erroneous behavior manifests

[axiomgroupe17]

Hello All,
I am having the same email issue it seems as most people on here. I have tried two methods of adding the email instance, once through admin and another through the actual user account. Then inside of these accounts I tried the two different methods of adding the inbound email and outbound email and the sending function does not work. I can clearly see that it is connecting minimally to the imap folder as I can see the emails in the email tab. Unfortunately when i click on the inbox icon to switch to the sent folder or trash all it does is pop up a white window with the spinning balls of doom which I am sure all here are familiar with.

Any help on this topic would be greatly appreciated.
Here is my Suite CRM info
Version 7.10.7
Sugar Version 6.5.25 (Build 344)

[sedatremindme]

Its really important to get this issue solved to keep the whole email communication logged on server for future.
Please fix this issue as soon as possible, its really needed.

My SuiteCRM Info:
Version 7.10.9 and 7.10.10
Sugar Version 6.5.25 (Build 344)

[Dillon-Brown]

Taking a look into this, unfortunately I've yet to replicate the issue. If I have an inbound email that matches my outbound email then any sent mail will appear in the imap sent mail which will then appear in the CRM Sent folder.

If I am sending mail from a different outbound account then the mail won't appear in my sent folder as the mail will be in the sent folder of that outbound accounts imap. Would this not be expected or is there something i'm missing here.

[taneks]

I have the same issue.

The behaviour is correct (to me) when I send a test email from Profile | EmailAddress | Settings | Mail Accounts | Outgoing SMTP Mail Servers. The recipient receives the test email, and a copy of the email is present in the Sent folder of my Gmail Account.

However, when initiating an email by clicking on email link under Contacts | Email Address, the recipient receives the test email, but a copy of the email is NOT in my Gmail Sent Folder.

Was checking my configuration when I saw this thread.
SuiteCRM version 7.10.10

[kikobar]

My observation of this issue is that when you use a standard IMAP folder, the application does not leave a copy of the email in the 'Sent' folder (and this is a bug).
Conversely, when you use a Gmail account, which is a non-standard IMAP folder, where all the emails are kept in the 'All mail' folder with some 'tags' to indicate to which 'folder' they belong, in such a case the system tags the email as 'Sent' and thus shows up in the 'Sent' folder (there is no bug here).
We are using vanilla SuiteCRM [Version 7.10.7, Sugar Version 6.5.25 (Build 344)]

[JanHubert]

Taking a look into this, unfortunately I've yet to replicate the issue. If I have an inbound email that matches my outbound email then any sent mail will appear in the imap sent mail which will then appear in the CRM Sent folder.

If I am sending mail from a different outbound account then the mail won't appear in my sent folder as the mail will be in the sent folder of that outbound accounts imap. Would this not be expected or is there something i'm missing here.

@Dillon-Brown Both scenarios you are describing would be absolutely fine. But in my case (and there seem to be a couple of other people affected as well) there is no mail stored anywhere. It is completely lost after sending. Not in the outbound account's sent folder nor in your personal sent folder.

We are on standard IMAP (Kopano). It may be that Gmail is not affected as described above.

[kikobar]

It may be that Gmail is not affected as described above

Yes, I can confirm that Gmail accounts work. I use multiple accounts and all the Gmail accounts work.
Kindly note that Gmail does not use an standard IMAP setting.

[ebogaard]

When using the Gmail smtp-server when sending e-mails, Gmail itself saves the sent mail to the sent mail folder.

When using any smtp-server that doesn't do this, for example a postfix smtp-server, no sent mails are saved.

So when testing this issue: don't use the Gmail smtp-server, use a local, simple, smtp-server.

[macpczone]

I had this problem as well when I upgraded a client to 7.10.7 and ended up having to write a patch just to stop her from calling me up almost every day to ask if the problem was fixed. What seems to have happened is that the email sending code for the user, seems to be using the campaign SMTP code, which of course does not need to care about putting the emails in a sent box. I had a look at the code to see if I could get the user part to send with the email2send function, but it looked like it would be too hard, so I just copied the email2send code and placed it in the function that the user sending code was using and it worked, so finally the phone calls stopped. I have attached the patch and it applies to 7.10.7. It should also work with 7.10.10, because the diff in git looks exactly the same.

--- modules/Emails/Email.php.old    2018-07-27 09:49:27.000000000 +0100
+++ modules/Emails/Email.php    2018-10-30 15:48:25.000000000 +0000
@@ -2908,6 +2908,32 @@
                 $ieMail->retrieve($_REQUEST['inbound_email_id']);
                 $ieMail->status = 'replied';
                 $ieMail->save();
+            ///////////////////////////////////////////////////////////////////
+            ////   STORE MAIL IN SENT BOX
+            // mark SEEN
+               $ie = new InboundEmail();
+               $ie->retrieve($_REQUEST['inbound_email_id']);
+               if (isset($ie->id) && !$ie->isPop3Protocol() && $mail->oe->mail_smtptype != 'gmail') {
+                   $sentFolder = $ie->get_stored_options("sentFolder");
+                   if (!empty($sentFolder)) {
+                       // Call CreateBody() before CreateHeader() as that is where boundary IDs are generated.
+                       $emailbody = $mail->CreateBody();
+                       $emailheader = $mail->CreateHeader();
+                       $data = $emailheader . "\r\n" . $emailbody . "\r\n";
+                       $ie->mailbox = $sentFolder;
+                       if ($ie->connectMailserver() == 'true') {
+                           $connectString = $ie->getConnectString($ie->getServiceString(), $ie->mailbox);
+                           $returnData = imap_append($ie->conn,$connectString, $data, "\\Seen");
+                           if (!$returnData) {
+                               $GLOBALS['log']->debug("could not copy email to {$ie->mailbox} for {$ie->name}");
+                           } // if
+                       } else {
+                           $GLOBALS['log']->debug("could not connect to mail serve for folder {$ie->mailbox} for {$ie->name}");
+                       } // else
+                   } else {
+                       $GLOBALS['log']->debug("could not copy email to {$ie->mailbox} sent folder as its empty");
+                   } // else
+               } // if
             }
             $GLOBALS['log']->debug(' --------------------- buh bye -- sent successful');
             ////   END INBOUND EMAIL HANDLING

[ebogaard]

@macpczone Works great. Thanks for sharing!

[sedatremindme]

I had this problem as well when I upgraded a client to 7.10.7 and ended up having to write a patch just to stop her from calling me up almost every day to ask if the problem was fixed. What seems to have happened is that the email sending code for the user, seems to be using the campaign SMTP code, which of course does not need to care about putting the emails in a sent box. I had a look at the code to see if I could get the user part to send with the email2send function, but it looked like it would be too hard, so I just copied the email2send code and placed it in the function that the user sending code was using and it worked, so finally the phone calls stopped. I have attached the patch and it applies to 7.10.7. It should also work with 7.10.10, because the diff in git looks exactly the same.

--- modules/Emails/Email.php.old  2018-07-27 09:49:27.000000000 +0100
+++ modules/Emails/Email.php  2018-10-30 15:48:25.000000000 +0000
@@ -2908,6 +2908,32 @@
                 $ieMail->retrieve($_REQUEST['inbound_email_id']);
                 $ieMail->status = 'replied';
                 $ieMail->save();
+            ///////////////////////////////////////////////////////////////////
+            //// STORE MAIL IN SENT BOX
+            // mark SEEN
+             $ie = new InboundEmail();
+             $ie->retrieve($_REQUEST['inbound_email_id']);
+             if (isset($ie->id) && !$ie->isPop3Protocol() && $mail->oe->mail_smtptype != 'gmail') {
+                 $sentFolder = $ie->get_stored_options("sentFolder");
+                 if (!empty($sentFolder)) {
+                     // Call CreateBody() before CreateHeader() as that is where boundary IDs are generated.
+                     $emailbody = $mail->CreateBody();
+                     $emailheader = $mail->CreateHeader();
+                     $data = $emailheader . "\r\n" . $emailbody . "\r\n";
+                     $ie->mailbox = $sentFolder;
+                     if ($ie->connectMailserver() == 'true') {
+                         $connectString = $ie->getConnectString($ie->getServiceString(), $ie->mailbox);
+                         $returnData = imap_append($ie->conn,$connectString, $data, "\\Seen");
+                         if (!$returnData) {
+                             $GLOBALS['log']->debug("could not copy email to {$ie->mailbox} for {$ie->name}");
+                         } // if
+                     } else {
+                         $GLOBALS['log']->debug("could not connect to mail serve for folder {$ie->mailbox} for {$ie->name}");
+                     } // else
+                 } else {
+                     $GLOBALS['log']->debug("could not copy email to {$ie->mailbox} sent folder as its empty");
+                 } // else
+             } // if
             }
             $GLOBALS['log']->debug(' --------------------- buh bye -- sent successful');
             //// END INBOUND EMAIL HANDLING

big thanks for sharing!
Will this be upgrade safe if i create the Emails.php in custom/modules/Emails/ ?

[macpczone]

big thanks for sharing!

> Will this be upgrade safe if i create the Emails.php in custom/modules/Emails/ ?

They made a lot of changes in this file with the last few updates, so it is probably best to just apply the patch, with a dry run first in case they put a fix in which breaks it, so that their bugfixes do not get lost.

[sedatremindme]

ok, thanks.
i tried it on version 7.10.10, but It seems as if it is not working.

[fraromano]

I have used this patch in SuiteCRM 7.10.10 but only works for personal accounts.
For group accounts it doesn't work, messages are not copied in the Sent folder.

[gymad]

Hi @JanHubert
I just made a PR #6600 for fix this issue, can you please confirm if it works as expected?

[JanHubert]

Hi @gymad
Sounds great, I am willing to test your changes. But at the moment I have no test instance. I am quite certain I can create one and do some testing next week though!

[JanHubert]

@gymad I'm afraid I have to ask how to download/patch your PR to my installation? Not a dev, no git setup...

[Dillon-Brown]

Hi @JanHubert, ideally you would want to create a test instance and then pull the PR into it or in this case just pull down hotfix as its already been merged. See Forking SuiteCRM for details.

Otherwise if you didn't want to use Git you would need to copy each of the changed files in that PR and move them to your installation manually.

[JanHubert]

Just as a follow-up: After updating to 7.10.13 I can confirm that copies of sent emails are being saved in the selected sent folder.
Thanks a lot to everyone!