Mail: Error while loading messages again

Created on 8 Sep 2016 · 17Comments · Source: nextcloud/mail

Tried to solve the issue but no success

Though the symptoms are similar to #50 I think it is different since in the horde log the request seems fine - ie. 20 mails are recieved.
I tried resetting the account according to the README.md (deleted the tables and entries)
The same two directories are not available after reset as before (INBOX (tranlated to Posteingang) and Lindyhop)
restarting dovecot(on the same server) and httpd
how do I install the nextcloud version is it any different from the owncloud version?

Steps to reproduce

Configure imap account
got to mail
click on inbox
no messages but instead a small notification on to "messages could not be loaded"

Expected behaviour

See all the messages and be able to answer and read them.

Actual behaviour

The app fails on certain folders to load the messages.

Mail app

Mail app version: (see apps admin page)
Mail 0.5.3

Mailserver or service: (e.g. Outlook, Yahoo, Gmail, Exchange,...)
dovecot (selfhosted) Thunderbird works well on those directories

Transport security - IMAP: (None, SSL, TLS, STARTTLS)
imap SSL/TLS

Transport security - SMTP: (None, SSL, TLS, STARTTLS)
not important
Number of accounts:
1
Mail app version build date: (only if you are using a Nightly Build)

Server configuration

Operating system:
Fedora 23
Web server:
Apache
Database:
mariadb
PHP version:
PHP 7.0.10
Version: (see admin page)
Nextcloud 10.0 RC1 (stable)
Updated from an older version or fresh install:
updated

List of activated apps:

Enabled:
  - activity: 2.3.2
  - admin_audit: 1.0.0
  - calendar: 1.3.3
  - comments: 1.0.0
  - contacts: 1.4.0.0
  - dav: 1.0.0
  - federatedfilesharing: 1.0.1
  - federation: 1.0.1
  - files: 1.5.2
  - files_pdfviewer: 0.8.1
  - files_sharing: 1.0.0
  - files_texteditor: 2.1
  - files_trashbin: 1.0.0
  - files_versions: 1.3.0
  - gallery: 15.0.0
  - mail: 0.5.3
  - news: 9.0.4
  - notifications: 0.3.0
  - password_policy: 1.0.0
  - provisioning_api: 1.0.0
  - serverinfo: 1.1.1
  - survey_client: 0.1.5
  - systemtags: 1.0.1
  - tasks: true
  - templateeditor: 0.1
  - theming: 1.0.1
  - updatenotification: 1.0.1
  - workflowengine: 1.0.0
Disabled:
  - encryption
  - external
  - files_accesscontrol
  - files_automatedtagging
  - files_external
  - files_retention
  - files_videoplayer
  - firstrunwizard
  - user_external
  - user_ldap
  - user_saml

The content of config/config.php:

{
    "system": {
        "instanceid": "oc4p9uo912ib",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "teilgedanken.de"
        ],
        "datadirectory": "\/var\/nextcloud\/data",
        "overwrite.cli.url": "https:\/\/teilgedanken.de\/nc",
        "htaccess.RewriteBase": "\/nc",
        "dbtype": "mysql",
        "version": "9.1.0.15",
        "dbname": "clouddb",
        "dbhost": "localhost",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "logtimezone": "UTC",
        "installed": true,
        "appstore.experimental.enabled": true,
        "memcache.local": "\\OC\\Memcache\\APC",
        "loglevel": 0,
        "mail_smtpmode": "smtp",
        "mail_smtpsecure": "ssl",
        "mail_smtpauth": 1,
        "mail_smtphost": "wp269.webpack.hosteurope.de",
        "mail_smtpport": "465",
        "mail_from_address": "dietrich",
        "mail_domain": "teilgedanken.de",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "maintenance": false,
        "updater.release.channel": "beta",
        "log_type": "owncloud",
        "customclient_android": "https:\/\/f-droid.org\/repository\/browse\/?fdid=com.nextcloud.client"
    }
}

Client configuration

Browser:
Firefox

Operating system:
Fedora 24

Logs

Web server error log

Insert your webserver log here

Server log (data/nextcloud.log)

``` "reqId":"V9FEvcoTXiFurT8OSe@pSgAAAAE","remoteAddr":"2a02:8071:b180:7f00:221:5cff:fe51:3da5","app":"mail","message":"loading messages 0 to 20 of folder {"reqId":"V9FEvcoTXiFurT8OSe@pSgAAAAE","remoteAddr":"2a02:8071:b180:7f00:221:5cff:fe51:3da5","app":"index","message":"Exception: {\"Exception\":\"TypeError\",\"Message\":\"Argument 1 passed to OCA\\Mail\\Model\\IMAPMessage::hordeToAssoc() must be an instance of Horde_Mail_Rfc822_Address, instance of Horde_Mail_Rfc822_Group given, called in \\/var\\/www\\/deployed.teilgedanken\\/nc\\/apps\\/mail\\/lib\\/model\\/convertaddresses.php on line 73\",\"Code\":0,\"Trace\":\"#0 \\/var\\/www\\/deployed.teilgedanken\\/nc\\/apps\\/mail\\/lib\\/model\\/convertaddresses.php(73): OCA\\Mail\\Model\\IMAPMessage->hordeToAssoc(Object(Horde_Mail_Rfc822_Group))\n#1 \\/var\\/www\\/deployed.teilgedanken\\/nc\\/apps\\/mail\\/lib\\/model\\/imapmessage.php(240): OCA\\Mail\\Model\\IMAPMessage->convertAddressList(Object(Horde_Mail_Rfc822_List))\n#2 \\/var\\/www\\/deployed.teilgedanken\\/nc\\/apps\\/mail\\/lib\\/model\\/imapmessage.php(509): OCA\\Mail\\Model\\IMAPMessage->getToList(true)\n#3 \\/var\\/www\\/deployed.teilgedanken\\/nc\\/apps\\/mail\\/lib\\/mailbox.php(171): OCA\\Mail\\Model\\IMAPMessage->getListArray()\n#4 \\/var\\/www\\/deployed.teilgedanken\\/nc\\/apps\\/mail\\/lib\\/controller\\/messagescontroller.php(133): OCA\\Mail\\Mailbox->getMessages(0, 21, NULL)\n#5 [internal function]: OCA\\Mail\\Controller\\MessagesController->index(2, 'SU5CT1g=', 0, 20, NULL, NULL)\n#6 \\/var\\/www\\/deployed.teilgedanken\\/nc\\/lib\\/private\\/AppFramework\\/Http\\/Dispatcher.php(160): call_user_func_array(Array, Array)\n#7 \\/var\\/www\\/deployed.teilgedanken\\/nc\\/lib\\/private\\/AppFramework\\/Http\\/Dispatcher.php(90): OC\\AppFramework\\Http\\Dispatcher->executeController(Object(OCA\\Mail\\Controller\\MessagesController), 'index')\n#8 \\/var\\/www\\/deployed.teilgedanken\\/nc\\/lib\\/private\\/AppFramework\\/App.php(111): OC\\AppFramework\\Http\\Dispatcher->dispatch(Object(OCA\\Mail\\Controller\\MessagesController), 'index')\n#9 \\/var\\/www\\/deployed.teilgedanken\\/nc\\/lib\\/private\\/AppFramework\\/Routing\\/RouteActionHandler.php(47): OC\\AppFramework\\App::main('OCA\\\\Mail\\\\Contro...', 'index', Object(OC\\AppFramework\\DependencyInjection\\DIContainer), Array)\n#10 [internal function]: OC\\AppFramework\\Routing\\RouteActionHandler->__invoke(Array)\n#11 \\/var\\/www\\/deployed.teilgedanken\\/nc\\/lib\\/private\\/Route\\/Router.php(293): call_user_func(Object(OC\\AppFramework\\Routing\\RouteActionHandler), Array)\n#12 \\/var\\/www\\/deployed.teilgedanken\\/nc\\/lib\\/base.php(967): OC\\Route\\Router->match('\\/apps\\/mail\\/acco...')\n#13 \\/var\\/www\\/deployed.teilgedanken\\/nc\\/index.php(48): OC::handleRequest()\n#14 {main}\",\"File\":\"\\/var\\/www\\/deployed.teilgedanken\\/nc\\/apps\\/mail\\/lib\\/model\\/convertaddresses.php\",\"Line\":33}","level":3,"time":"2016-09-08T11:00:16+00:00","method":"GET","url":"\/nc\/apps\/mail\/accounts\/2\/folders\/SU5CT1g%3D\/messages","user":"enaut"}


</details>
##### Horde IMAP log (data/horde_imap.log)

<details>

Thu, 08 Sep 2016 11:00:15 +0000
Connection to: imap://mx0.teilgedanken.de:993/
Server connection took 0.0754 seconds.
S: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready.
C: 1 AUTHENTICATE PLAIN [INITIAL CLIENT RESPONSE (username: solilith)]
S: 1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SPECIAL-USE] Logged in
Command 1 took 0.2178 seconds.
C: 2 EXAMINE INBOX
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $Forwarded NonJunk Junk)
S: * OK [PERMANENTFLAGS ()] Read-only mailbox.
S: * 992 EXISTS
S: * 0 RECENT
S: * OK [UIDVALIDITY 1428757386] UIDs valid
S: * OK [UIDNEXT 1528] Predicted next UID
S: * OK [HIGHESTMODSEQ 7035] Highest
S: 2 OK [READ-ONLY] Examine completed (0.000 + 0.000 secs).
Command 2 took 0.0035 seconds.
C: 3 UID SORT RETURN (ALL COUNT) (DATE) US-ASCII UNDELETED
S: * ESEARCH (TAG "3") UID ALL 582,855,446:448,450:460,466,470,476:477,485:489,492:493,499,504,503,507,522,516:517,519,525:526,539,544:545,552:553,321,346,335,555,347:348,362,364,556:557,374,560,562,377,388,392:393,563,567,566,565,408,415,568,417,424,440,569,443,461:464,570,465,467,571,468:469,471,572,576,472,577:578,473,579:580,480,581,481:482,583,483,586,584,592,591,484,590,490,494:495,497:498,502,506,508:510,515,518,520,529,521,523:524,528,530:532,535:537,540:543,546,607:608,547:548,609,549,610,550,603,611,604,612:614,606,616:617,619:624,630,632:636,639,638,640,642:655,659,657,661:662,664,663,665:675,677:684,686:691,1:10,694,693,697,11:16,20,26,25,27:28,701,30:31,64,88:93,702:703,142,145,704,148,150:151,153,155,706:708,710:711,713,716,164,174,718,175:176,721,178,722,181:182,724:725,183,726,728,193:196,198,200:204,206:209,211:212,222:223,231:233,735:738,250,739:741,252:254,258:261,289:307,310,308:309,311,313,312,314:317,757,784,786:788,790:803,825,804:806,808:820,823:824,821:822,826:850,861,851:853,862:863,854,860,857,864:867,876,868:870,875,874,871,877:878,872:873,879,881:900,902:904,906:913,916:928,930:943,945,947:958,960:961,963:979,981:983,986,988:993,995,997:1001,1003,1005:1009,1012:1024,1027:1029,1031:1032,1044,1033:1043,1045:1049,1051:1064,1066:1070,1074:1081,1083:1084,1086:1100,1102,1104:1106,1116:1119,1107:1111,1120:1124,1115,1125:1131,1134:1139,1141:1142,1146:1148,1150:1152,1154,1156,1159,1161:1165,1169:1183,1185:1187,1189:1199,1213,1212,1201:1202,1214:1233,1236:1244,1246,1245,1247,1249:1261,1268,1262:1263,1265,1267,1266,1269:1280,1282:1285,1288:1292,1294:1296,1298,1297,1299:1305,1307,1306,1308:1310,1312:1316,1319,1321,1320,1322:1332,1352,1333:1351,1354:1355,1357:1387,1389:1450,1452:1463,1467:1478,1480:1519,1521,1523:1525 COUNT 992
S: 3 OK Sort completed (0.023 + 0.000 + 0.022 secs).
Command 3 took 0.0265 seconds.
C: 4 UID FETCH 1503:1519,1521,1523:1525 (ENVELOPE FLAGS RFC822.SIZE INTERNALDATE BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (IMPORTANCE LIST-POST X-PRIORITY CONTENT-TYPE)])

Personal stuff I won't post here

S: 4 OK Fetch completed (0.004 + 0.000 + 0.003 secs).

Command 4 took 0.2259 seconds.
C: 5 LOGOUT
S: * BYE Logging out
S: 5 OK Logout completed (0.000 + 0.000 secs).
Command 5 took 0.0043 seconds.


</details>
##### Browser log

<details>

Javascript log:
Content Security Policy: Die Direktive 'frame-src' sollte nicht mehr verwendet werden. Bitte verwenden Sie stattdessen die Direktive 'child-src'.(unbekannt)
Content Security Policy: Die Einstellungen der Seite haben das Laden einer Ressource auf self blockiert ("script-src https://teilgedanken.de 'unsafe-eval'").mail:1
JQMIGRATE: Migrate is installed, version 1.4.0jquery-migrate.min.js:2:542
Content Security Policy: Die Einstellungen der Seite haben das Laden einer Ressource auf self blockiert ("script-src https://teilgedanken.de").html:1
```

1. to develop bug

Source

enaut

👍1

Most helpful comment

Got this too, happens when you receive an email with a hidden recipient list.
For me there is a Group "Verborgene_Empfaenger" on that mail.

The following fix works:

diff --git a/lib/Model/ConvertAddresses.php b/lib/Model/ConvertAddresses.php
index f723826c..78f5c554 100644
--- a/lib/Model/ConvertAddresses.php
+++ b/lib/Model/ConvertAddresses.php
@@ -71,7 +71,9 @@ trait ConvertAddresses {
        protected function convertAddressList($envelope) {
                $list = [];
                foreach ($envelope as $t) {
-                       $list[] = $this->hordeToAssoc($t);
+                       if ($t instanceof Horde_Mail_Rfc822_Address) {
+                               $list[] = $this->hordeToAssoc($t);
+                       }
                }
                return $list;
        }

That at least makes the list load again, not sure if the groups should be shown somehow or not.

nickvergessen on 19 Dec 2016

🎉3

All 17 comments

So some update...

I just discovered that its some specific messages that cause the trouble. When the messages are not on the first page the folder loads fine. However as soon as I scroll down far enough the additional messages are not loaded. So I created a test folder with thunderbird and moved some messages to this folder. By doing so I found one malicious message that stops the folder from loading.

It's a mail with several umlauts and é and an attachment. Also it is forwarded and sent through a mailinglist... so all possible reasons to fail.

If I forward the mails to myself the problem is gone!
If I click 'edit as new' and send the mail to myself the problem persists!

What should I do next? I could send someone the problematic email but I don't want it posted here!

Any help is appreciated

enaut on 13 Sep 2016

Horde_Mail_Rfc822_Group

@irgendwie :wink:

ChristophWurst on 13 Sep 2016

Got this too, happens when you receive an email with a hidden recipient list.
For me there is a Group "Verborgene_Empfaenger" on that mail.

The following fix works:

diff --git a/lib/Model/ConvertAddresses.php b/lib/Model/ConvertAddresses.php
index f723826c..78f5c554 100644
--- a/lib/Model/ConvertAddresses.php
+++ b/lib/Model/ConvertAddresses.php
@@ -71,7 +71,9 @@ trait ConvertAddresses {
        protected function convertAddressList($envelope) {
                $list = [];
                foreach ($envelope as $t) {
-                       $list[] = $this->hordeToAssoc($t);
+                       if ($t instanceof Horde_Mail_Rfc822_Address) {
+                               $list[] = $this->hordeToAssoc($t);
+                       }
                }
                return $list;
        }

That at least makes the list load again, not sure if the groups should be shown somehow or not.

nickvergessen on 19 Dec 2016

🎉3

Awesome. Yeah we'll need a solution like that because there is no common base class for group addresses and regular ones. We should show groups too somehow. Does that class provide anything printable?

ChristophWurst on 19 Dec 2016

There is $t->groupname, $t->groupname_encoded and $t->label but all 3 are Verborgene_Empfaenger for me, so not really a nice printed version....

But I guess that is caused by the sender, so you can use either one of them.

nickvergessen on 19 Dec 2016

I had the same problem, the fix from https://github.com/nextcloud/mail/issues/53#issuecomment-267920014 works for me. Thanks @nickvergessen

matzex on 20 Dec 2016

So just to note you as I reported this issue: #53 is the "solution". So after applying this patch it works even with the hidden recipients mails. There is no marker, that there are additional recipients though!

enaut on 21 Dec 2016

Have same Problem and comment #53 help me. Just as info ... bevore upgrade from php 5.6 to php 7 it have working ... maybe php 5.6 don´t see this error and php 7 see it? WHatever, please fix it :-) Thx