Calendar: Not all events from a Google Calendar subscription are shown

[vandenheuvel]

Steps to reproduce

Add a Google Calendar by clicking:

1. "New Calendar""Subscription from link (read-only)"
2. Copy-pasting the "Secret address in iCal format"
3. Hitting "Enter"

Expected behaviour

See all events from that calendar

Actual behaviour

Most events are displayed, some aren't. It is unclear what makes that difference.

Calendar app

Calendar app version: 2.0.3

Client configuration

Browser: Firefox 76.0.1
Operating system: Arch Linux

Server configuration

Operating system: (e.g. Debian 8)
Web server: Nginx
Database: MariaDB
PHP version: 7.4.5
Nextcloud Version: 18.0.4-1
Updated from an older installed version or fresh install: Continuously updating for about two years
List of activated apps:

• accessibility: 1.4.0
• activity: 2.11.0
• calendar: 2.0.3
• cloud_federation_api: 1.1.0
• comments: 1.8.0
• contacts: 3.3.0
• dav: 1.14.0
• encryption: 2.6.0
• federatedfilesharing: 1.8.0
• federation: 1.8.0
• files: 1.13.1
• files_external: 1.9.0
• files_pdfviewer: 1.7.0
• files_rightclick: 0.15.2
• files_sharing: 1.10.1
• files_trashbin: 1.8.0
• files_versions: 1.11.0
• files_videoplayer: 1.7.0
• firstrunwizard: 2.7.0
• logreader: 2.3.0
• lookup_server_connector: 1.6.0
• nextcloud_announcements: 1.7.0
• notifications: 2.6.0
• oauth2: 1.6.0
• password_policy: 1.8.0
• photos: 1.0.0
• privacy: 1.2.0
• provisioning_api: 1.8.0
• recommendations: 0.6.0
• serverinfo: 1.8.0
• settings: 1.0.0
• sharebymail: 1.8.0
• spreed: 8.0.8
• support: 1.1.0
• survey_client: 1.6.0
• systemtags: 1.8.0
• tasks: 0.12.2
• text: 2.0.0
• theming: 1.9.0
• twofactor_backupcodes: 1.7.0
• updatenotification: 1.8.0
• viewer: 1.2.0
• workflowengine: 2.0.0

Nextcloud configuration:

        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***"
        ],
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "18.0.4.2",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "maintenance": false,
        "theme": "",
        "loglevel": 2,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "app_install_overwrite": [
            "tasks"
        ]

Log file (data/nextcloud.log)

This is probably on the browser side, I excluded this for privacy.

Browser log

The javascript console log:

Content Security Policy: The page’s settings blocked the loading of a resource at eval (“script-src”).
JQMIGRATE: Migrate is installed, version 1.4.1 main.js:566:531
Proxying an event bus of version 1.1.2 with 0.2.1 notifications.js:21:13435
Handlebars is deprecated: please ship your own, this will be removed in Nextcloud 20 main.js:1082:57
Notifications permissions not yet requested notifications.js:21:52482
Initial settings: 
Object { appVersion: "2.0.3", eventLimit: true, firstRun: false, showWeekends: true, showWeekNumbers: true, skipPopover: true, slotDuration: "00:30:00", talkEnabled: true, timezone: "Europe/Berlin" }
calendar.js:2:2964568
[DEBUG] calendar: The calendar app is using version 2.2019c of the timezone database 
Object { app: "calendar", uid: "authenticated:user" }
calendar.js:2:2029245
Fallback to locale en calendar.js:2:2653880
This page uses the non standard property “zoom”. Consider using calc() in the relevant property values, or using “transform” along with “transform-origin: 0 0”. main.js:230:11228
Source map error: Error: JSON.parse: unexpected character at line 1 column 1 of the JSON data
Resource URL: https://domain.com/core/js/dist/main.js?v=c836bf7a-0
Source Map URL: main.js.map

Source map error: Error: JSON.parse: unexpected character at line 1 column 1 of the JSON data
Resource URL: https://domain.com/apps/notifications/js/notifications.js?v=c836bf7a-0
Source Map URL: notifications.js.map

Source map error: Error: JSON.parse: unexpected character at line 1 column 1 of the JSON data
Resource URL: https://domain.com/apps/calendar/js/calendar.js?v=c836bf7a-0
Source Map URL: calendar.js.map

[DEBUG] calendar: Current user principal is /remote.php/dav/principals/users/user/ 
Object { app: "calendar", uid: "authenticated:user" }
calendar.js:2:2029245

[georgehrke]

This is probably on the browser side, I excluded this for privacy.

Please include the nextcloud.log. Subscriptions are not directly fetched from the remote server, but cached in Nextcloud. So this is most likely a server side issue.

[vandenheuvel]

I readded the calendar once more while recreating the log. Naturally, the bug stopped manifesting itself. I'll close this for now.

[vandenheuvel]

Sometime over the last two weeks, some newly added events again did not show up in the calendar. I extracted the log file (starting at a reboot after which I readded the calendar).

[tcitworld]

{"reqId":"ocKrAa9JReDNFB9eDemY","level":4,"time":"2020-05-15T18:59:55+00:00","remoteAddr":"185.159.157.10","user":"user","app":"webdav","method":"MKCOL","url":"/remote.php/dav/calendars/user/calendargooglecom","message":{"Exception":"Sabre\\VObject\\Recur\\NoInstancesException","Message":"This recurrence rule does not generate any valid instances","Code":0,"Trace":[{"file":"/usr/share/webapps/nextcloud/apps/dav/lib/CalDAV/CalDavBackend.php","line":2261,"function":"__construct","class":"Sabre\\VObject\\Recur\\EventIterator","type":"->"},{"file":"/usr/share/webapps/nextcloud/apps/dav/lib/CalDAV/CalDavBackend.php","line":1055,"function":"getDenormalizedData","class":"OCA\\DAV\\CalDAV\\CalDavBackend","type":"->"},{"file":"/usr/share/webapps/nextcloud/apps/dav/lib/CalDAV/WebcalCaching/RefreshWebcalService.php","line":141,"function":"createCalendarObject","class":"OCA\\DAV\\CalDAV\\CalDavBackend","type":"->"},{"file":"/usr/share/webapps/nextcloud/apps/dav/appinfo/app.php","line":70,"function":"refreshSubscription","class":"OCA\\DAV\\CalDAV\\WebcalCaching\\RefreshWebcalService","type":"->"},{"file":"/usr/share/webapps/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php","line":260,"function":"{closure}","class":"OC_App","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/usr/share/webapps/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php","line":235,"function":"doDispatch","class":"Symfony\\Component\\EventDispatcher\\EventDispatcher","type":"->"},{"file":"/usr/share/webapps/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php","line":73,"function":"callListeners","class":"Symfony\\Component\\EventDispatcher\\EventDispatcher","type":"->"},{"file":"/usr/share/webapps/nextcloud/lib/private/EventDispatcher/SymfonyAdapter.php","line":70,"function":"dispatch","class":"Symfony\\Component\\EventDispatcher\\EventDispatcher","type":"->"},{"file":"/usr/share/webapps/nextcloud/apps/dav/lib/CalDAV/CalDavBackend.php","line":1965,"function":"dispatch","class":"OC\\EventDispatcher\\SymfonyAdapter","type":"->"},{"file":"/usr/share/webapps/nextcloud/3rdparty/sabre/dav/lib/CalDAV/CalendarHome.php","line":262,"function":"createSubscription","class":"OCA\\DAV\\CalDAV\\CalDavBackend","type":"->"},{"file":"/usr/share/webapps/nextcloud/apps/dav/lib/CalDAV/CalendarHome.php","line":73,"function":"createExtendedCollection","class":"Sabre\\CalDAV\\CalendarHome","type":"->"},{"file":"/usr/share/webapps/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1197,"function":"createExtendedCollection","class":"OCA\\DAV\\CalDAV\\CalendarHome","type":"->"},{"file":"/usr/share/webapps/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":590,"function":"createCollection","class":"Sabre\\DAV\\Server","type":"->"},{"function":"httpMkcol","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/usr/share/webapps/nextcloud/3rdparty/sabre/event/lib/EventEmitterTrait.php","line":105,"function":"call_user_func_array"},{"file":"/usr/share/webapps/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":479,"function":"emit","class":"Sabre\\Event\\EventEmitter","type":"->"},{"file":"/usr/share/webapps/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":254,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/usr/share/webapps/nextcloud/apps/dav/lib/Server.php","line":319,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/usr/share/webapps/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/usr/share/webapps/nextcloud/remote.php","line":165,"args":["/usr/share/webapps/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/usr/share/webapps/nextcloud/3rdparty/sabre/vobject/lib/Recur/EventIterator.php","Line":203,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0","version":"18.0.4.2"}

Are any of those events recurring events ?

[vandenheuvel]

I am not sure what you are asking, but yes, there are recurrent meetings in the google calendar that fails to sync. The meeting that fail to sync are not necessarily recurring, though.

[gjgd]

I'm having the same issue. Since the latest version, my Google calendar events don't show up in the Nextcloud calendar anymore

[tcitworld]

The issue seems to be in @vandenheuvel's case :

Sabre\VObject\Recur\NoInstancesException
This recurrence rule does not generate any valid instances

@gjgd Do you have recurring events in this calendar as well?

[gjgd]

I do

[georgehrke]

@gjgd And do you see the same Sabre\VObject\Recur\NoInstancesException error message in the Nextcloud.log?

[gjgd]

Yes! Here is the error when I import my calendar.

{
  "Exception": "Sabre\\VObject\\Recur\\NoInstancesException",
  "Message": "This recurrence rule does not generate any valid instances",
  "Code": 0,
  "Trace": ...
}

I tried it again on version v1.7.2 and it worked so it's a regression. I'll try to produce a minimal example of a calendar that doesn't work this week-end

[georgehrke]

I tried it again on version v1.7.2 and it worked so it's a regression

The way we handle WebCal calendars changed a lot in 2.0. While it might work in v1.x, it do not consider it a regression, because that error indicates invalid calendar-data.

The NoInstancesException is thrown when there is recurring event that does not contain any instances. (e.g. because you deleted all of them). This is prohibited by the calendar-standard.

[georgehrke]

If you can provide us with the ics that is causing this NoInstancesException, we can check whether there might be an issue in the parser that mistakenly marks it as invalid, but if the calendar-data is actually invalid, there is not much we can do.

[gjgd]

Ah yes, indeed I did delete a recurring event. I guess it's Google calendar that didn't clean things up properly.

I'll post an example soon. Is there any way to ignore the invalid recurring event so that the rest of the calendar is still displayed ?

[gjgd]

This is an example of an event that throws the error:

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Sabre//Sabre VObject 4.1.6//EN
CALSCALE:GREGORIAN
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VTIMEZONE
TZID:Asia/Tokyo
X-LIC-LOCATION:Asia/Tokyo
BEGIN:STANDARD
TZOFFSETFROM:+0900
TZOFFSETTO:+0900
TZNAME:JST
DTSTART:19700101T000000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/Chicago:20200313T110000
DTEND;TZID=America/Chicago:20200313T120000
RRULE:FREQ=WEEKLY;UNTIL=20200410T045959Z;INTERVAL=2;BYDAY=FR
EXDATE;TZID=America/Chicago:20200327T110000
EXDATE;TZID=America/Chicago:20200313T110000
DTSTAMP:20200606T113658Z
ORGANIZER;[email protected]:mailto:[email protected]
UID:[email protected]
CREATED:20190108T165200Z
DESCRIPTION: redacted
LAST-MODIFIED:20200606T034539Z
LOCATION:
SEQUENCE:1
STATUS:CONFIRMED
SUMMARY:redacted
TRANSP:OPAQUE
END:VEVENT
END:VCALENDAR

So I tried to import this event in v1.7.2 and in v2.0.3 and in both cases nextcloud returns this error:

<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:exception>Sabre\VObject\Recur\NoInstancesException</s:exception>
  <s:message>This recurrence rule does not generate any valid instances</s:message>
</d:error>

Note that in v1.7.2 the error is ignored and other events in the calendar are displayed, whereas in v2.0.3 the error prevents other events from being displayed, which is the real issue here.

I should also note that this bug only happens when creating a subscription to a calendar with a link. This bug will not happen if you import an ICS file (in this case the invalid event is ignored, but other events are still displayed).

[eleith]

i can confirm that new subscriptions to calendars no longer show any events if this error This recurrence rule does not generate any valid instances is hit.

are there any workarounds in the meantime?

[novadeviator]

I have a same/similar issue when subscribing to a private google calendar (i think the fact that it is private might not be relevant).

Nextcloud log reports:

Fatal | webdav | Sabre\VObject\Recur\NoInstancesException: This recurrence rule does not generate any valid instances

It seems to import all events, but when new events are added or changes occur in google calendar, they don't sync to nextcloud.

[ashkitten]

i believe i'm affected by this issue. seeing An error occurred, unable to create the calendar. pop up when i try to subscribe via an ics link (either public or private). downloading the ics file and importing it to a new calendar works as a stopgap, but won't be updated with changes.

[balaramadurai]

i believe i'm affected by this issue. seeing An error occurred, unable to create the calendar. pop up when i try to subscribe via an ics link (either public or private). downloading the ics file and importing it to a new calendar works as a stopgap, but won't be updated with changes.

Hello @ashkitten, I found a way to work this out. Even though, you do see an error message. Click on another page in your Nextcloud setup and click back on the "calendar" and you will see your calendar.google.com in your Nextcloud.

I haven't tested to find if the subscribed calendar is updated when you change something on Google.

Hope that helps

[gjgd]

This works, but the problem is that all events after the broken event that creates the error will not be added to your calendar

[eleith]

submitted https://github.com/nextcloud/server/pull/22969 to address the issue

the exception handling during webcal refresh leads to an exception on invalid recurrance events

however, the Sabre\VObject\Recur\NoInstancesException is of type Exception but the try/catch clause only catches Sabre\Dav\BadRequest. by expanding the types of exceptions, we can properly handle the exception and further webcal events can be iterated and added

there are a few other approaches that could be taken and i'm open to implementing them

[novadeviator]

I continue have problem with adding a google calendar. It imports most events but does not update anymore when events are added - I believe.

here's the log output:
https://gist.github.com/novadeviator/39e1519000a1390e5f57563257193d31