
Two issues:
There should be some kind of dialog box appear to define how you want the email filed.
The email details screen should populate all of the fields (sender/message body/etc)
A blank record is created without any relationship. See the following tail from the sugarcrm.log file:
`Tue May 30 11:13:32 2017 [1665][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Failed to load relationship emails_email_templates while saving Emails
Tue May 30 11:13:33 2017 [3787][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Tue May 30 11:13:33 2017 [3787][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates for emails_email_templates failed to load
Tue May 30 11:13:33 2017 [3787][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Failed to load relationship emails_email_templates while saving Emails
Tue May 30 11:14:37 2017 [1663][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Tue May 30 11:14:38 2017 [1663][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load`
None.
I am simply trying to file emails against contact/opportunity records. The text of the email is completely missing, rendering the imported email useless.
This is a high priority issue.
Did you follow the special steps described in the Release Notes for this upgrade?
https://suitecrm.com/wiki/index.php/Release_notes_7.9.0_User_Guide
I have run through those instructions, but the same problems persist. The 'Sync' operation seemed to fail repeatedly on my primary account (no error displayed, just a blank screen), so I decided to delete and recreate the account.
Using the supplied instructions I tried to recreate my account, but the necessary dialog box does not appear - so I cannot add any new email accounts under the user account settings.
My sugarcrm.log file is still full of exactly the same errors _and_ the email screen is completely broken (see attached screenshot):
Tue May 30 12:25:13 2017 [4255][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Failed to load relationship emails_email_templates while saving Emails
Tue May 30 12:25:13 2017 [4255][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Tue May 30 12:25:13 2017 [4255][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Tue May 30 12:25:13 2017 [4255][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Tue May 30 12:25:13 2017 [4255][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Tue May 30 12:25:13 2017 [4255][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Tue May 30 12:27:05 2017 [4828][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Exception handling in /var/www/crm2015/include/MVC/Controller/SugarController.php:367
Tue May 30 12:27:05 2017 [4828][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Exception in Controller: SugarFolder::retrieveFoldersForProcessing() Cannot Retrieve Folders - Please check the users inbound email settings.
Tue May 30 12:27:05 2017 [4828][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] backtrace:
#0 /var/www/crm2015/include/SugarFolders/SugarFolders.php(745): SugarFolder->retrieveFoldersForProcessing(Object(User), true)
#1 /var/www/crm2015/modules/Emails/EmailUIAjax.php(958): SugarFolder->getUserFolders(Object(ExtNode), false, Object(User), true)
#2 /var/www/crm2015/include/MVC/View/SugarView.php(791): include_once('/var/www/crm201...')
#3 /var/www/crm2015/include/MVC/View/views/view.classic.php(74): SugarView->includeClassicFile('modules/Emails/...')
#4 /var/www/crm2015/include/MVC/View/SugarView.php(199): ViewClassic->display()
#5 /var/www/crm2015/include/MVC/Controller/SugarController.php(399): SugarView->process()
#6 /var/www/crm2015/include/MVC/Controller/SugarController.php(336): SugarController->processView()
#7 /var/www/crm2015/include/MVC/SugarApplication.php(105): SugarController->execute()
#8 /var/www/crm2015/index.php(52): SugarApplication->execute()
#9 {main}
Tue May 30 12:27:57 2017 [4242][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Query Failed: SELECT count(*) c FROM emails JOIN emails_text on emails.id = emails_text.email_idWHERE emails.deleted=0 AND emails.type NOT IN ('out', 'draft')
AND emails.status NOT IN ('sent', 'draft') AND emails.id IN (SELECT eear.email_id FROM emails_email_addr_rel eearJOIN email_addr_bean_rel eabr ON eabr.email_address_id=eear.email_address_id AND eabr.bean_id = '44bc535b-103d-dc6e-00c6-54a3d594bf4d' AND eabr.bean_module = 'Users' WHERE eear.deleted=0) AND emails.status = 'unread': MySQL error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'emails.deleted=0 AND emails.type NOT IN ('out', 'draft')
AND emails.' at line 1
Tue May 30 12:28:08 2017 [4842][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Error: File [modules/Home/metadata/editviewdefs.php] is missing. Unable to create because no corresponding HTML file was found.

I have re-added the primary email account as a "group email" address (it is currently impossible to add "personal" addresses), and re-run the Sync email addresses option.
Email is still completely inaccessible. Here is the latest sugarcrm.log tail:
Tue May 30 12:35:05 2017 [4242][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Tue May 30 12:35:15 2017 [4242][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Error: File [modules/Home/metadata/editviewdefs.php] is missing. Unable to create because no corresponding HTML file was found.
Tue May 30 12:37:44 2017 [4842][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Query Failed: SELECT count(*) c FROM emails JOIN emails_text on emails.id = emails_text.email_idWHERE emails.deleted=0 AND emails.type NOT IN ('out', 'draft')
AND emails.status NOT IN ('sent', 'draft') AND emails.id IN (SELECT eear.email_id FROM emails_email_addr_rel eearJOIN email_addr_bean_rel eabr ON eabr.email_address_id=eear.email_address_id AND eabr.bean_id = '44bc535b-103d-dc6e-00c6-54a3d594bf4d' AND eabr.bean_module = 'Users' WHERE eear.deleted=0) AND emails.status = 'unread': MySQL error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'emails.deleted=0 AND emails.type NOT IN ('out', 'draft')
AND emails.' at line 1
Tue May 30 12:37:44 2017 [4842][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Query Failed: SELECT count(*) c FROM emails JOIN emails_text on emails.id = emails_text.email_idWHERE emails.deleted=0 AND emails.type NOT IN ('out', 'draft')
AND emails.status NOT IN ('sent', 'draft') AND emails.id IN (SELECT eear.email_id FROM emails_email_addr_rel eearJOIN email_addr_bean_rel eabr ON eabr.email_address_id=eear.email_address_id AND eabr.bean_id = '44bc535b-103d-dc6e-00c6-54a3d594bf4d' AND eabr.bean_module = 'Users' WHERE eear.deleted=0) AND emails.status = 'unread': MySQL error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'emails.deleted=0 AND emails.type NOT IN ('out', 'draft')
AND emails.' at line 1
I'm going to try and roll-back. This is a disaster.
Just tried a fresh upgrade, ensuring all email accounts were synced before doing anything else. Clearly something is still broken (see attached screenshot). Is it possible that the sync process is timing out on the primary email account?

Rolling back again. The upgraded system cannot even send a new email.
Ok - a few issues to tackle then.
Samus, I replied in the SuiteCRM forums. However, attached is a screen shot of the add email setting which will not open for me either.

Following is the last part of my logs also. Sorry to chime in on this however I believe we are both having the same issues.
Tue May 30 08:18:19 2017 [81691][1][ERROR] Unable to find relationship emails_email_templates
Tue May 30 08:18:19 2017 [81691][1][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Tue May 30 08:18:19 2017 [81691][1][ERROR] Unable to find relationship emails_email_templates
Tue May 30 08:18:19 2017 [81691][1][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Tue May 30 08:18:19 2017 [81691][1][ERROR] Unable to find relationship emails_email_templates
Tue May 30 08:18:19 2017 [81691][1][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Tue May 30 08:18:19 2017 [81691][1][ERROR] IMAP error detected: SECURITY PROBLEM: insecure server advertised AUTH=PLAIN
Tue May 30 08:18:19 2017 [81691][1][ERROR] IMAP error detected: Client tried to access nonexistent namespace. (Mailbox name should probably be prefixed with: INBOX.) (0.001 + 0.000 secs).
Tue May 30 08:21:02 2017 [91574][1][ERROR] Unable to find relationship emails_email_templates
Tue May 30 08:21:02 2017 [91574][1][FATAL] emails_email_templates for emails_email_templates failed to load
Tue May 30 08:21:02 2017 [91574][1][FATAL] Failed to load relationship emails_email_templates while saving Emails
OK, I have performed another upgrade from 7.8.3 to 7.9.0 and carried out the synchronisation of the email accounts as per the upgrade instructions. All of my accounts synchronise correctly and present completion notices _except_ my main account which results in a blank screen:

As instructed, I cleared my browser cache which has sorted the problems displaying the Inbox correctly:

However, importing emails still does not provide any kind of dialog box allowing me to allocate them to a contact/account/opportunity etc. The routine also only imports a handful of values - From, To and Subject - the actual email body is completely missing:

Here is the latest tail from the sugarcrm.log following the email import attempt:
Tue May 30 19:44:14 2017 [1186][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Failed to load relationship emails_email_templates while saving Emails
Tue May 30 19:44:23 2017 [1738][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Tue May 30 19:44:23 2017 [1738][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates for emails_email_templates failed to load
Tue May 30 19:44:23 2017 [1738][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Failed to load relationship emails_email_templates while saving Emails
Tue May 30 19:47:22 2017 [1402][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Tue May 30 19:47:22 2017 [1402][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
I have successfully deleted and re-added the primary email address at last - as well as deleting the email settings in the Administration applet, you also need to reassign a new primary email address to the user record - otherwise it cannot be re-added. Unfortunately deleting and re-adding the account does not solve the issue of body text being removed from the import, even though the repair sync operation _does_ complete successfully this time round.
Also of interested - attempting to bulk import several messages, including one that has already been imported generates this interesting error:

Sugarcrm.log output remains the same however:
Tue May 30 20:03:54 2017 [5072][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Failed to load relationship emails_email_templates while saving Emails
Tue May 30 20:04:05 2017 [5082][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Tue May 30 20:04:05 2017 [5082][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Tue May 30 20:05:23 2017 [5070][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Tue May 30 20:05:23 2017 [5070][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates for emails_email_templates failed to load
Tue May 30 20:05:23 2017 [5070][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Failed to load relationship emails_email_templates while saving Emails
@blloyd78 how exactly do you "roll back" between attempts?
Do you revert to a snapshot of a virtual machine?
Do you replace files from a backup?
Do you replace files and database from a backup?
I am using a snapshot of the virtual machine @pgorod - I can restore back to a fully operation v7.8.3 installation in a few minutes
UPDATE:
Just tried the 7.8.3 -> 7.8.4 -> 7.9.0 upgrade route and the same problems are still present, as are the errors in the sugarcrm.log:
Tue May 30 21:16:14 2017 [1165][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates for emails_email_templates failed to load
Tue May 30 21:16:14 2017 [1165][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Failed to load relationship emails_email_templates while saving Emails
Tue May 30 21:16:14 2017 [1534][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Tue May 30 21:16:14 2017 [1534][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates for emails_email_templates failed to load
Tue May 30 21:16:14 2017 [1534][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Failed to load relationship emails_email_templates while saving Emails
Hi @blloyd78 i getting same or similary problems as you... after upgrade from SuiteCRM 7.8.3 to 7.9 we are fighting with those things...
If i will have more details from my developer, i put their here.. Sorry if it looks as spam comment...
@Mausino - you may be able to resolve item 2 in the list by clearing your browser cache - delete all cached content and cookies for your SuiteCRM address - that trick worked for me.
@blloyd78 yes, i deleted what i could :D it helps little bit.. maybe on server is something about i don't know... i'm thinking about revert back all process of upgrade.. because now it looks that the emails are unusable for my company... when my employees come to work, will get heart attack :( i will try resolve with developer some things which make me crazy now... thanks for your help
@gymad I have made the code change suggested in your fix and performed a quick repair but the problem remains - the email body is not imported (among other things). The sugarcrm.log indicates the problem still exists:
SET `assigned_user_id`='',`name`='',`date_modified`='',`scheduler_id`='',`execute_time`='',`status`='',`resolution`='',`message`='',`target`='',`data`=NULL,`requeue`=0,`retry_count`=NULL,`failure_count`=128,`job_delay`=0,`client`='',`percent_complete`=NULL
WHERE job_queue.id = '' AND deleted=0: Database Is Not Connected
Wed May 31 10:25:05 2017 [1278][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates for emails_email_templates failed to load
Wed May 31 10:25:05 2017 [1278][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Failed to load relationship emails_email_templates while saving Emails
Wed May 31 10:25:12 2017 [1756][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Wed May 31 10:25:12 2017 [1756][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates for emails_email_templates failed to load
Wed May 31 10:25:12 2017 [1756][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Failed to load relationship emails_email_templates while saving Emails]
Hi @blloyd78
Problem with having multiple issues on one issue is that we apply fixes for different things in different commits.
Can we review what the main issues are again so that when apply changes that you know which one helps progressively?
OK, from your list:
@blloyd78 Lets do some steps if you are able to help us troubleshoot the issues.
Issue 1. When you click onto an already imported email via the Email Module (or another module?) do you get that [FATAL] emails_email_templates_idb for emails_email_templates failed to load error?
Do you get this FATAL error if you click on a email that has not been imported yet?
@samus-aran after upgrade to 7.9 i have those issues
1 still apears
2 I also have not any pop up... it is importing to somewhere..
5 when i upgraded i had this issue also... but i deleted all inbound emails and recreate new and issue disapear but i got new one... that the folders stayed in email settings https://www.screencast.com/t/RSMIjJiVj
@samus-aran
This is what happens when you click on an email that has already been imported:
Wed May 31 10:25:05 2017 [1278][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Failed to load relationship emails_email_templates while saving Emails
Wed May 31 10:25:12 2017 [1756][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Wed May 31 10:25:12 2017 [1756][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates for emails_email_templates failed to load
Wed May 31 10:25:12 2017 [1756][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] Failed to load relationship emails_email_templates while saving Emails
Wed May 31 11:37:22 2017 [1422][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
Wed May 31 11:37:23 2017 [1422][44bc535b-103d-dc6e-00c6-54a3d594bf4d][FATAL] emails_email_templates_idb for emails_email_templates failed to load
An email that has not yet been imported can be accessed and read correctly. There are also no errors generated in the sugarcrm.log
@blloyd78
There is proposed fix for the imported email body issue (issue 1 here).
Would be great if you could test it out - we've asked @Mausino a few questions - same applies to yourself if you are able to answer.
"We just want to double check if the issue occurs when you have imported the emails after the upgrade or before? If it was after then the fix should resolve the issue but will require a re-import of that particular record. You should be able to delete the email (which only deletes the CRM record not the IMAP message) via the Detail View of the Record."
@samus-aran @Mausino - the issue only occurs with emails imported _after_ the upgrade. I have applied the code changes listed in the proposed fix (#3620), and I can import emails - kinda. Here's an original email _before_ import:

And here's that same email _after_ import:

I'm also getting the same [FATAL] emails_email_templates_idb for emails_email_templates failed to load errors
@blloyd78
See the body in the second screenshot - can you reload the page (i.e. just refresh)? Does that still happen - the escaping of the html?
@samus-aran - That worked! We're getting closer...
Ok @blloyd78 Just going over this again.
Pull Requests: https://github.com/salesagility/SuiteCRM/pull/3620 & https://github.com/salesagility/SuiteCRM/pull/3621
So 4 our of 5 we have either tackled or rectifying. The last one is something we haven't reproduced. We may need to work on that more so.
Can we focus on that one in this thread more so?
Ok, when you view this account in the Emails Module, anything interesting in your Logs (SuiteCRM logs) on DEBUG mode? All other email accounts work i.e. don't show blank screen? What is different about that email account to the rest?
When you do the repair tool 'Sync Inbound Email Accounts', and only select your problem account, does it come up with any errors on the screen (should be showing progress at least)?
Updated - May raise a new issue specific to the blank account depending on how extensive it is.
With regards to #5, I assume that this is a bog-standard timeout issue. The Inbox of the account in question holds 607 messages, and a further 4 subfolders; none of the other accounts comes close in terms of numbers.
I should also point out that the blank screen is displayed at the end of the 'Sync Inbound Email Accounts' action - the actual inbox itself loads correctly. It's worth noting that _none_ of the email accounts actually shows a status dialog until _after_ the process has completed.
@samus-aran i had same/similary issue as @blloyd78 and with point 5, my url address after upgrade looks like this and was blank:
but my developer fix something and it looks now like (sorry i wil know more what he fixed when will available tomorrow):
i hope, that it helps localize the issue for point 5. (more about how developer fixed it, i will add tomorrow)
Ok then @blloyd78 - the timeout has been increased for this Inbound Sync Account tool so hopefully a timeout should be minimised but it does sound like that.
But let us know @Mausino the feedback when your developer returns :+1:
@samus-aran he left me skype message now :)
Here is: "I've just removed a custom controller file (custom/modules/Emails/controller.php) which was out of date and produced the issue. I'm not sure if this was added by another plugin installation, or if was added by they and forgot to update/remove when done upgrades."
@Mausino - Thanks for that feedback. Yes that would make sense if you have a custom controller as we can't overwrite that in any upgrade. But that is perhaps something to discuss with your developer to ensure your customisations now match the new structure.
But this raises a point - we should be checking (for often customised files) if we have changes for core modules and people have customisations (using upgrade safe ways which is great!) which would cause conflict. Perhaps just an alert on the upgrade wizard when doing the preflight check that it says "Hey, we are updating these files in core - but we found you have similar named files in the custom folder" so you can prep yourselves of any development changes needed for the upgrade afterwards. Makes sense?
@samus-aran your suggestion is something I had thought of before, and actually meant to do one day.
I think it's absolutely a good idea. Upgrades that replace customized files, or that refrain from replacing them (but leave out necessary new files) must be warned to the users.
I think it should be presented on-screen to the admin doing the upgrade, not just logged. It's something that definitely requires attention. Better yet if was warned before the upgrade, during the pre-flight check, so the issue could be analyzed before any changes broke the system.
@samus-aran very nice idea with warnigs... 馃 or some place where developers could know what they should/need to check :)
The first part of this bug, the part where it shows old data on the page and appears to be severely broken, is proof that the upgrade code should clear out the old cache so that the app doesn't show that outdated stuff to the user. The user shouldn't be stuck with having to go into the Admin page and do that manually.
@samus-aran could you create for your idea "Hey, we are updating these files in core - but we found you have similar named files in the custom folder" the record/suggestion in trello ???, that will not forget by time.
Most helpful comment
@samus-aran could you create for your idea "Hey, we are updating these files in core - but we found you have similar named files in the custom folder" the record/suggestion in trello ???, that will not forget by time.