Server: [NC 14] Cannot sync calendars since updating to RC3

[aproposnix]

First, sorry for xposting this from the forums. I'm not sure where I should upload support requests. Please close this issue report if I violate some policy.

The Issue:
Since NC 14rc3 I no longer can sync calendar events to Davdroid or any other caldav client (tried with Thunderbird/Lightning on desktop as well). I updated today to 14rc4 but it didn't resolve the issue.

Expected behavior:
I should be able to sync calendars with mobile and desktop calendars.

==========================
Specs:
Nextcloud 14rc4 (original upgrade from 13 through updater)

Ubuntu 16.04

Apache
Server version: Apache/2.4.18 (Ubuntu)
Server built: 2018-06-07T19:43:03

===============
PHP
PHP 7.0.30-0ubuntu0.16.04.1 (cli) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
with Zend OPcache v7.0.30-0ubuntu0.16.04.1, Copyright (c) 1999-2017, by Zend Technologies

=====
Database
mysql Ver 15.1 Distrib 10.0.34-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

In my logs, I see this whenever I try to connect:

LogicException: This is not supposed to happen

/var/www/nextcloud/3rdparty/sabre/dav/lib/CalDAV/Plugin.php - line 345:

OCA\DAV\CalDAV\Plugin->getCalendarHomeForPrincipal("principals/groups/admin")

/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/PropFind.php - line 98:

Sabre\CalDAV\Plugin->Sabre\CalDAV\{closure}("*** sensiti ... *")

/var/www/nextcloud/3rdparty/sabre/dav/lib/CalDAV/Plugin.php - line 349:

Sabre\DAV\PropFind->handle("{urn:ietf:p ... t", Closure {})

Sabre\CalDAV\Plugin->propFind(Sabre\DAV\PropFind {}, OCA\DAV\CalD ... {})

/var/www/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php - line 105:

call_user_func_array([ OCA\DAV\Ca ... "], [ Sabre\DAV\ ... }])

/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 1059:

Sabre\Event\EventEmitter->emit("propFind", [ Sabre\DAV\ ... }])

/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 981:

Sabre\DAV\Server->getPropertiesByNode(Sabre\DAV\PropFind {}, OCA\DAV\CalD ... {})

/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 1666:

Sabre\DAV\Server->getPropertiesIteratorForPath("principals/groups/admin", [ "{urn:ietf ... "], 0)

/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php - line 359:

Sabre\DAV\Server->generateMultiStatus(Generator {}, false)

Sabre\DAV\CorePlugin->httpPropFind(Sabre\HTTP\R ... "}, Sabre\HTTP\Response {})

/var/www/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php - line 105:

call_user_func_array([ Sabre\DAV\ ... "], [ Sabre\HTTP ... }])

/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 479:

Sabre\Event\EventEmitter->emit("method:PROPFIND", [ Sabre\HTTP ... }])

/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 254:

Sabre\DAV\Server->invokeMethod(Sabre\HTTP\R ... "}, Sabre\HTTP\Response {})

/var/www/nextcloud/apps/dav/lib/Server.php - line 293:

Sabre\DAV\Server->exec()

/var/www/nextcloud/apps/dav/appinfo/v2/remote.php - line 35:

OCA\DAV\Server->exec()

/var/www/nextcloud/remote.php - line 163:

require_once("/var/www/ne ... p")

========================

Does anyone have any idea how to fix this? I was stupid enough to accidentally switch to the beta channel in the updater which is why I upgraded. The server is my production environment :/

[nextcloud-bot]

GitMate.io thinks possibly related issues are https://github.com/nextcloud/server/issues/5086 (Cannot sync NC Calendar with Google Calendar), https://github.com/nextcloud/server/issues/2889 (NC 11 - Cannot share files with users after update), https://github.com/nextcloud/server/issues/2862 (Root directories not listed after update to NC 11), and https://github.com/nextcloud/server/issues/10665 (DAV Error in Nextcloud 14).

[Co0p3r21]

Duplicate to #10665 (DAV Error in Nextcloud 14). But good to know that I am not the only one.

[aproposnix]

Just to be clear (from the bot feedback above), this is not about syncing the nextcloud server with external calendars like Google Calendar. All of my calendars are local (on the server) but do not sync with any other device.

Also, I should note that DAVdroid continues to sync the contacts, just not the calendars.

[Ilareguy]

I'm having the same issue on my server. I just found out that DAVdroid (or any other client really) fails on PROPFIND whenever you're part of a group on your Nextcloud server. That being said, it'll work fine if you're not part of any group. A potential temporary fix, if you absolutely need to sync your calendars with DAVdroid, is to remove yourself from any group on your Nextcloud server (including the "admin" group, in which case you'll have to create an alternate admin account since you can't remove yourself from the group), then try the PROPFIND again (tap "Refresh calendar list" in DAVdroid), and your calendars should appear. Once they're there, you can put yourself back into groups on your server. At this point, DAVdroid seems to be able to sync my calendars without issues.

[aproposnix]

@Octalized Thanks for this. I can confirm that the workaround you describe works.

[timm2k]

Corresponding error from DAVdroid:

--- BEGIN DEBUG INFO ---
Account name: Nextcloud

HTTP REQUEST:
Request{method=PROPFIND, url=https://nextcloud.host/remote.php/dav/principals/groups/group1/, tags={}}
<?xml version='1.0' encoding='UTF-8' ?><propfind xmlns="DAV:" xmlns:CAL="urn:ietf:params:xml:ns:caldav" xmlns:CARD="urn:ietf:params:xml:ns:carddav"><prop><CAL:calendar-home-set /><n0:calendar-proxy-read-for xmlns:n0="http://calendarserver.org/ns/" /><n1:calendar-proxy-write-for xmlns:n1="http://calendarserver.org/ns/" /><group-membership /></prop></propfind>

HTTP RESPONSE:
Response{protocol=h2, code=500, message=, url=https://nextcloud.host/remote.php/dav/principals/groups/group1/}
<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:exception>LogicException</s:exception>
  <s:message>This is not supposed to happen</s:message>
</d:error>



EXCEPTION:
at.bitfire.dav4android.exception.HttpException: HTTP 500 
        at at.bitfire.dav4android.DavResource.checkStatus(DavResource.kt:283)
        at at.bitfire.dav4android.DavResource.checkStatus(DavResource.kt:259)
        at at.bitfire.dav4android.DavResource.processMultiStatus(DavResource.kt:351)
        at at.bitfire.dav4android.DavResource.propfind(DavResource.kt:246)
        at at.bitfire.davdroid.DavService$refreshCollections$1$2.invoke(DavService.kt:232)
        at at.bitfire.davdroid.DavService$refreshCollections$1$2.invoke(DavService.kt:245)
        at at.bitfire.davdroid.DavService$refreshCollections$1$2.invoke$default(DavService.kt:186)
        at at.bitfire.davdroid.DavService.refreshCollections(DavService.kt:282)
        at at.bitfire.davdroid.DavService.access$refreshCollections(DavService.kt:44)
        at at.bitfire.davdroid.DavService$onStartCommand$$inlined$let$lambda$1.invoke(DavService.kt:68)
        at at.bitfire.davdroid.DavService$onStartCommand$$inlined$let$lambda$1.invoke(DavService.kt:44)
        at kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30)

SOFTWARE INFORMATION
Package: at.bitfire.davdroid
Version: 2.0.2-ose (243) from 23.08.2018
Installed from: com.android.packageinstaller
JB Workaround installed: no

CONNECTIVITY (at the moment)
Active connection: WiFi, CONNECTED

CONFIGURATION
Power saving disabled: yes
android.permission.READ_CONTACTS permission: granted
android.permission.WRITE_CONTACTS permission: granted
android.permission.READ_CALENDAR permission: granted
android.permission.WRITE_CALENDAR permission: granted
org.dmfs.permission.READ_TASKS permission: granted
org.dmfs.permission.WRITE_TASKS permission: granted
System-wide synchronization: automatically
Account: Nextcloud
  Address book sync. interval: 240 min
  Calendar     sync. interval: 240 min
  OpenTasks    sync. interval: 240 min
  WiFi only: false
  [CardDAV] Contact group method: GROUP_VCARDS
  [CalDAV] Time range (past days): 90
           Manage calendar colors: true
Address book account: Kontakte (Nextcloud Rw)
  Main account: Account {name=Nextcloud, type=bitfire.at.davdroid}
  URL: https://nextcloud.host/remote.php/dav/addressbooks/users/user1/kontakte/
  Sync automatically: true
Address book account: Kontakte (user2) (Nextcloud Hw)
  Main account: Account {name=Nextcloud, type=bitfire.at.davdroid}
  URL: https://nextcloud.host/remote.php/dav/addressbooks/users/user1/kontakte_shared_by_user2/
  Sync automatically: true

SQLITE DUMP
android_metadata
        |  locale |
        |  de_DE |
----------
services
        |  _id | accountName | service | principal |
        |  1 | Nextcloud | carddav | https://nextcloud.host/remote.php/dav/principals/users/user1/ |
        |  2 | Nextcloud | caldav | https://nextcloud.host/remote.php/dav/principals/users/user1/ |
----------
sqlite_sequence
        |  name | seq |
        |  services | 2 |
        |  homesets | 2 |
        |  collections | 7 |
----------
homesets
        |  _id | serviceID | url |
        |  1 | 2 | https://nextcloud.host/remote.php/dav/calendars/user1/ |
        |  2 | 1 | https://nextcloud.host/remote.php/dav/addressbooks/users/user1/ |
----------
collections
        |  _id | serviceID | type | url | readOnly | forceReadOnly | displayName | description | color | timezone | supportsVEVENT | supportsVTODO | source | sync |
        |  1 | 2 | CALENDAR | https://nextcloud.host/remote.php/dav/calendars/user1/pers%c3%b6nlich/ | 0 | 0 | user1 | Persönlich | -13513604 | <null> | 1 | 1 | <null> | 1 |
        |  2 | 2 | CALENDAR | https://nextcloud.host/remote.php/dav/calendars/user1/geburtstage/ | 0 | 0 | Geburtstage | Geburtstage | -256 | <null> | 1 | 1 | <null> | 1 |
        |  3 | 2 | CALENDAR | https://nextcloud.host/remote.php/dav/calendars/user1/contact_birthdays/ | 1 | 0 | Geburtstage von Kontakten | <null> | -54 | <null> | 1 | 0 | <null> | 0 |
        |  4 | 2 | CALENDAR | https://nextcloud.host/remote.php/dav/calendars/user1/default_shared_by_user3/ | 0 | 0 | user3 (user3) | <null> | -13534004 | <null> | 1 | 1 | <null> | 1 |
        |  5 | 2 | CALENDAR | https://nextcloud.host/remote.php/dav/calendars/user1/pers%c3%b6nlich_shared_by_user2/ | 0 | 0 | user2 (user2) | Persönlich | -1146130 | <null> | 1 | 1 | <null> | 1 |
        |  6 | 1 | ADDRESS_BOOK | https://nextcloud.host/remote.php/dav/addressbooks/users/user1/kontakte/ | 0 | 0 | Kontakte | <null> | <null> | <null> | 0 | 0 | <null> | 1 |
        |  7 | 1 | ADDRESS_BOOK | https://nextcloud.host/remote.php/dav/addressbooks/users/user1/kontakte_shared_by_user2/ | 0 | 0 | Kontakte (user2) | <null> | <null> | <null> | 0 | 0 | <null> | 1 |
----------

SYSTEM INFORMATION
Android version: 7.1.2 (lineage_athene-userdebug 7.1.2 NJH47F 835130b1a0)
Device: Motorola Moto G4 (athene)

--- END DEBUG INFO ---

[wahlis]

I have the same issue. The problem remains after upgrading to RC1

[FunctionalHacker]

Just a me too comment. The weird thing is, it worked just fine on RC1 until I decided (for an unrelated reason) delete and add the account again on DAVdroid. After that, calendar sync stopped working

EDIT: It seems that when you do the workaround and delete yourself from the groups, you can then add the groups back after setting up your sync clients, and it continues to work. It seems that the DAV calendar discovery does not work when the user is in a group

[criwe]

Maybe its related to #10134

[pixelipo]

Also affected by this bug. I'd like to point out that actual synchronization works without a glitch as long as you have a local list of calendars from before upgrade. What is not working is fetching list of available calendars from the server.

[pixelipo]

@georgehrke do you have any idea what could be going on here?

[georgehrke]

@MorrisJobke jfyi for release planning.

Will look into this tomorrow

[wahlis]

Managed to get sync working again by
1 - remove myself from any groups as @Octalized suggested
2 - replace Davdroid from Fdroid (fos-version) with Davdroid from Google Play (gplay-version)

[FunctionalHacker]

@wahlis No need to change your Davdroid to the gplay version. This is a regression in Nextcloud, not in Davdroid.

[georgehrke]

Can you please confirm that https://github.com/nextcloud/server/pull/10958 fixes the issue? Thx!

[timm2k]

Unfurtunatly this doesn't fix the error for me. DAVDroid still gets a
500 error while requesting group membership.

Only the exception message thrown by Nextcloud disappeart. Therfore the
website souce now appears in the log.

{"reqId":"BnjuCzKArTP1QdhPdpn9","level":3,"time":"2018-09-02T09:10:58+02:00","remoteAddr":"93.204.197.95","user":"timm","app":"remote","method":"PROPFIND","url":"\/remote.php\/dav\/principals\/groups\/group1\/","message":{"Exception":"TypeError","Message":"Return
value of OCA\\DAV\\CalDAV\\Plugin::getCalendarHomeForPrincipal() must be
of the type string, none
returned","Code":0,"Trace":[{"file":"\/var\/www\/data\/html\/nextcloud\/3rdparty\/sabre\/dav\/lib\/CalDAV\/Plugin.php","line":345,"function":"getCalendarHomeForPrincipal","class":"OCA\\DAV\\CalDAV\\Plugin","type":"->","args":["principals\/groups\/group1"]},{"file":"\/var\/www\/data\/html\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/PropFind.php","line":98,"function":"Sabre\\CalDAV\\{closure}","class":"Sabre\\CalDAV\\Plugin","type":"->","args":["***
sensitive parameters replaced
***"]},{"file":"\/var\/www\/data\/html\/nextcloud\/3rdparty\/sabre\/dav\/lib\/CalDAV\/Plugin.php","line":349,"function":"handle","class":"Sabre\\DAV\\PropFind","type":"->","args":["{urn:ietf:params:xml:ns:caldav}calendar-home-set",{"__class__":"Closure"}]},{"function":"propFind","class":"Sabre\\CalDAV\\Plugin","type":"->","args":[{"__class__":"Sabre\\DAV\\PropFind"},{"__class__":"OCA\\DAV\\CalDAV\\Principal\\User"}]},{"file":"\/var\/www\/data\/html\/nextcloud\/3rdparty\/sabre\/event\/lib\/EventEmitterTrait.php","line":105,"function":"call_user_func_array","args":[[{"__class__":"OCA\\DAV\\CalDAV\\Plugin"},"propFind"],[{"__class__":"Sabre\\DAV\\PropFind"},{"__class__":"OCA\\DAV\\CalDAV\\Principal\\User"}]]},{"file":"\/var\/www\/data\/html\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":1059,"function":"emit","class":"Sabre\\Event\\EventEmitter","type":"->","args":["propFind",[{"__class__":"Sabre\\DAV\\PropFind"},{"__class__":"OCA\\DAV\\CalDAV\\Principal\\User"}]]},{"file":"\/var\/www\/data\/html\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":981,"function":"getPropertiesByNode","class":"Sabre\\DAV\\Server","type":"->","args":[{"__class__":"Sabre\\DAV\\PropFind"},{"__class__":"OCA\\DAV\\CalDAV\\Principal\\User"}]},{"file":"\/var\/www\/data\/html\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":1666,"function":"getPropertiesIteratorForPath","class":"Sabre\\DAV\\Server","type":"->","args":["principals\/groups\/group1",["{urn:ietf:params:xml:ns:caldav}calendar-home-set","{http:\/\/calendarserver.org\/ns\/}calendar-proxy-read-for","{http:\/\/calendarserver.org\/ns\/}calendar-proxy-write-for","{DAV:}group-membership"],0]},{"file":"\/var\/www\/data\/html\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/CorePlugin.php","line":359,"function":"generateMultiStatus","class":"Sabre\\DAV\\Server","type":"->","args":[{"__class__":"Generator"},false]},{"function":"httpPropFind","class":"Sabre\\DAV\\CorePlugin","type":"->","args":[{"absoluteUrl":"https:\/\/jfaez3atkqnvimki.myfritz.net\/remote.php\/dav\/principals\/groups\/group1\/","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"\/var\/www\/data\/html\/nextcloud\/3rdparty\/sabre\/event\/lib\/EventEmitterTrait.php","line":105,"function":"call_user_func_array","args":[[{"__class__":"Sabre\\DAV\\CorePlugin"},"httpPropFind"],[{"absoluteUrl":"https:\/\/jfaez3atkqnvimki.myfritz.net\/remote.php\/dav\/principals\/groups\/group1\/","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"\/var\/www\/data\/html\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":479,"function":"emit","class":"Sabre\\Event\\EventEmitter","type":"->","args":["method:PROPFIND",[{"absoluteUrl":"https:\/\/jfaez3atkqnvimki.myfritz.net\/remote.php\/dav\/principals\/groups\/group1\/","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"\/var\/www\/data\/html\/nextcloud\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php","line":254,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[{"absoluteUrl":"https:\/\/jfaez3atkqnvimki.myfritz.net\/remote.php\/dav\/principals\/groups\/group1\/","__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"\/var\/www\/data\/html\/nextcloud\/apps\/dav\/lib\/Server.php","line":293,"function":"exec","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"\/var\/www\/data\/html\/nextcloud\/apps\/dav\/appinfo\/v2\/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->","args":[]},{"file":"\/var\/www\/data\/html\/nextcloud\/remote.php","line":163,"args":["\/var\/www\/data\/html\/nextcloud\/apps\/dav\/appinfo\/v2\/remote.php"],"function":"require_once"}],"File":"\/var\/www\/data\/html\/nextcloud\/apps\/dav\/lib\/CalDAV\/Plugin.php","Line":42,"CustomMessage":"--"},"userAgent":"DAVdroid\/2.0.3-ose
(2018\/08\/30; dav4android; okhttp\/3.11.0)
Android\/7.0","version":"14.0.0.17"}

[timm2k]

Same error with NC 14 RC2 + #10958

[aproposnix]

I'm not sure if it's relevant for the issue but when i upgraded to 14.0.0 RC 2, I had to apply the workaround again.

[georgehrke]

Did anybody else try #10958 ?

@timm2k Do you use something like OPCache? How did you apply the patch? Did you make sure the :string in the function getCalendarHomeForPrincipal($principalUrl):string { is removed?

[timm2k]

Yes, opcache is active but I've restarted php-fpm after replacing Plugin.php files.
I took https://github.com/nextcloud/server/blob/83639a9898581919a84f1e2b05e64b236c8268e2/apps/dav/lib/CalDAV/Plugin.php and https://github.com/nextcloud/server/blob/83639a9898581919a84f1e2b05e64b236c8268e2/apps/dav/lib/CardDAV/Plugin.php

I can confirm that my files are correct. I've just wget the raw files from your PR and now it works. I may forget a final at the end of the file by copy&paste in the first time...

So now I can confirm that calendars shared to a group are working with NC14 RC2 + #10958!

Thanks @georgehrke :-)

[aproposnix]

Thanks to everyone for making Nextcloud so great! The issue has been resolved for me with the new 14 release.

[ChristophWurst]

Thanks to everyone for making Nextcloud so great! The issue has been resolved for me with the new 14 release.

Thank you very much for this feedback, we appreciate this a lot :v:

[pixelipo]

Thanks guys, awesome work!

P.S. Sorry for spamming, but this was a really annoying bug and you did a great job fixing it :)