Server: 404 error on OPTION requests breaks CalDAV sync after upgrade to Nextcloud 15

[mp1409]

Steps to reproduce

1. Update to Nextcloud 15
2. Synchronise calendar with iOS calendar and GNOME Evolution

Expected behaviour

Synchronisation should work, no 404 erros in log.

Nginx log with Nextcloud 14 and iOS device:

IP_ADRESS_REDACTED - - [27/Dec/2018:00:34:28 +0100] "OPTIONS /remote.php/dav/principals/users/USER_NAME_REDACTED/ HTTP/1.1" 401 569 "-" "iOS/12.1 (16B92) dataaccessd/1.0"
IP_ADRESS_REDACTED - - [27/Dec/2018:00:34:28 +0100] "OPTIONS /remote.php/dav/principals/users/USER_NAME_REDACTED/ HTTP/1.1" 200 0 "-" "iOS/12.1 (16B92) dataaccessd/1.0"
IP_ADRESS_REDACTED - USER_NAME_REDACTED [27/Dec/2018:00:34:28 +0100] "OPTIONS /remote.php/dav/principals/users/USER_NAME_REDACTED/ HTTP/1.1" 200 0 "-" "iOS/12.1 (16B92) dataaccessd/1.0"
IP_ADRESS_REDACTED - - [27/Dec/2018:00:34:28 +0100] "REPORT /remote.php/dav/addressbooks/users/USER_NAME_REDACTED/contacts/ HTTP/1.1" 401 569 "-" "iOS/12.1 (16B92) dataaccessd/1.0"
IP_ADRESS_REDACTED - - [27/Dec/2018:00:34:28 +0100] "PROPFIND /remote.php/dav/calendars/USER_NAME_REDACTED/ HTTP/1.1" 207 48311 "-" "iOS/12.1 (16B92) dataaccessd/1.0"
IP_ADRESS_REDACTED - USER_NAME_REDACTED [27/Dec/2018:00:34:28 +0100] "REPORT /remote.php/dav/addressbooks/users/USER_NAME_REDACTED/contacts/ HTTP/1.1" 207 286 "-" "iOS/12.1 (16B92) dataaccessd/1.0"
IP_ADRESS_REDACTED - - [27/Dec/2018:00:34:28 +0100] "PROPFIND /remote.php/dav/calendars/USER_NAME_REDACTED/inbox/ HTTP/1.1" 207 427 "-" "iOS/12.1 (16B92) dataaccessd/1.0"
IP_ADRESS_REDACTED - - [27/Dec/2018:00:34:28 +0100] "PROPFIND /remote.php/dav/calendars/USER_NAME_REDACTED/inbox/ HTTP/1.1" 207 427 "-" "iOS/12.1 (16B92) dataaccessd/1.0"
...

Nginx log with Nextcloud 14 and GNOME Evolution:

IP_ADDRESS_REDACTED - - [27/Dec/2018:21:00:10 +0100] "OPTIONS /remote.php/dav/addressbooks/users/USERNAME_REDACTED/Adressbuch/ HTTP/1.1" 401 569 "-" "Evolution/3.30.3"
IP_ADDRESS_REDACTED - - [27/Dec/2018:21:00:10 +0100] "OPTIONS /remote.php/dav/calendars/USERNAME_REDACTED/privat/ HTTP/1.1" 401 569 "-" "Evolution/3.30.3"
IP_ADDRESS_REDACTED - - [27/Dec/2018:21:00:10 +0100] "OPTIONS /remote.php/dav/addressbooks/users/USERNAME_REDACTED/Adressbuch/ HTTP/1.1" 401 569 "-" "Evolution/3.30.3"
IP_ADDRESS_REDACTED - - [27/Dec/2018:21:00:10 +0100] "OPTIONS /remote.php/dav/calendars/USERNAME_REDACTED/privat/ HTTP/1.1" 401 569 "-" "Evolution/3.30.3"
IP_ADDRESS_REDACTED - USERNAME_REDACTED [27/Dec/2018:21:00:10 +0100] "OPTIONS /remote.php/dav/addressbooks/users/USERNAME_REDACTED/Adressbuch/ HTTP/1.1" 200 0 "-" "Evolution/3.30.3"
IP_ADDRESS_REDACTED - USERNAME_REDACTED [27/Dec/2018:21:00:10 +0100] "OPTIONS /remote.php/dav/calendars/USERNAME_REDACTED/privat/ HTTP/1.1" 200 0 "-" "Evolution/3.30.3"
IP_ADDRESS_REDACTED - USERNAME_REDACTED [27/Dec/2018:21:00:11 +0100] "PROPFIND /remote.php/dav/addressbooks/users/USERNAME_REDACTED/Adressbuch/ HTTP/1.1" 207 883 "-" "Evolution/3.30.3"
IP_ADDRESS_REDACTED - USERNAME_REDACTED [27/Dec/2018:21:00:11 +0100] "PROPFIND /remote.php/dav/calendars/USERNAME_REDACTED/privat/ HTTP/1.1" 207 957 "-" "Evolution/3.30.3"
IP_ADDRESS_REDACTED - USERNAME_REDACTED [27/Dec/2018:21:00:11 +0100] "PROPFIND /remote.php/dav/addressbooks/users/USERNAME_REDACTED/Adressbuch/ HTTP/1.1" 207 469 "-" "Evolution/3.30.3"
...

Actual behaviour

After upgrade to 15, a lot of 404s in the log for the options request. At least with Evolution, the calendar synchronisation fails (cannot confirm the iOS device at the moment). Adressbooks & Android devices appear to work, though. There are no errors in the Nextcloud log.

Ngnix log with Nextcloud 15 and iOS device:

IP_ADRESS_REDACTED - - [28/Dec/2018:17:39:48 +0100] "OPTIONS /remote.php/dav/principals/users/USER_NAME_REDACTED/ HTTP/1.1" 404 5 "-" "iOS/12.1 (16B92) dataaccessd/1.0"
IP_ADRESS_REDACTED - - [28/Dec/2018:17:39:48 +0100] "OPTIONS /remote.php/dav/principals/users/USER_NAME_REDACTED/ HTTP/1.1" 404 5 "-" "iOS/12.1 (16B92) dataaccessd/1.0"
IP_ADRESS_REDACTED - - [28/Dec/2018:17:39:48 +0100] "REPORT /remote.php/dav/addressbooks/users/USER_NAME_REDACTED/contacts/ HTTP/1.1" 401 569 "-" "iOS/12.1 (16B92) dataaccessd/1.0"
IP_ADRESS_REDACTED - - [28/Dec/2018:17:39:48 +0100] "PROPFIND /remote.php/dav/calendars/USER_NAME_REDACTED/ HTTP/1.1" 401 569 "-" "iOS/12.1 (16B92) dataaccessd/1.0"
IP_ADRESS_REDACTED - USER_NAME_REDACTED [28/Dec/2018:17:39:48 +0100] "PROPFIND /remote.php/dav/calendars/USER_NAME_REDACTED/ HTTP/1.1" 207 48311 "-" "iOS/12.1 (16B92) dataaccessd/1.0"
IP_ADRESS_REDACTED - USER_NAME_REDACTED [28/Dec/2018:17:39:48 +0100] "REPORT /remote.php/dav/addressbooks/users/USER_NAME_REDACTED/contacts/ HTTP/1.1" 207 286 "-" "iOS/12.1 (16B92) dataaccessd/1.0"
IP_ADRESS_REDACTED - USER_NAME_REDACTED [28/Dec/2018:17:39:49 +0100] "PROPFIND /remote.php/dav/calendars/USER_NAME_REDACTED/inbox/ HTTP/1.1" 207 427 "-" "iOS/12.1 (16B92) dataaccessd/1.0"
IP_ADRESS_REDACTED - USER_NAME_REDACTED [28/Dec/2018:17:39:49 +0100] "PROPFIND /remote.php/dav/calendars/USER_NAME_REDACTED/inbox/ HTTP/1.1" 207 427 "-" "iOS/12.1 (16B92) dataaccessd/1.0"
...

Nginx log with Nextcloud 15 and GNOME Evolution:

IP_ADDRESS_REDACTED - - [29/Dec/2018:08:34:43 +0100] "OPTIONS /remote.php/dav/addressbooks/users/USERNAME_REDACTED/Adressbuch/ HTTP/1.1" 200 0 "-" "Evolution/3.30.3"
IP_ADDRESS_REDACTED - - [29/Dec/2018:08:34:43 +0100] "OPTIONS /remote.php/dav/calendars/USERNAME_REDACTED/privat/ HTTP/1.1" 404 5 "-" "Evolution/3.30.3"
IP_ADDRESS_REDACTED - - [29/Dec/2018:08:34:44 +0100] "PROPFIND /remote.php/dav/addressbooks/users/USERNAME_REDACTED/Adressbuch/ HTTP/1.1" 401 569 "-" "Evolution/3.30.3"
IP_ADDRESS_REDACTED - USERNAME_REDACTED [29/Dec/2018:08:34:44 +0100] "PROPFIND /remote.php/dav/addressbooks/users/USERNAME_REDACTED/Adressbuch/ HTTP/1.1" 207 883 "-" "Evolution/3.30.3"
IP_ADDRESS_REDACTED - USERNAME_REDACTED [29/Dec/2018:08:34:44 +0100] "PROPFIND /remote.php/dav/addressbooks/users/USERNAME_REDACTED/Adressbuch/ HTTP/1.1" 207 469 "-" "Evolution/3.30.3"
IP_ADDRESS_REDACTED - USERNAME_REDACTED [29/Dec/2018:08:34:44 +0100] "PROPFIND /remote.php/dav/addressbooks/users/USERNAME_REDACTED/Adressbuch/ HTTP/1.1" 207 469 "-" "Evolution/3.30.3"
IP_ADDRESS_REDACTED - - [29/Dec/2018:08:35:04 +0100] "OPTIONS /remote.php/dav/calendars/USERNAME_REDACTED/privat/ HTTP/1.1" 404 5 "-" "Evolution/3.30.3"
IP_ADDRESS_REDACTED - - [29/Dec/2018:09:03:39 +0100] "OPTIONS /remote.php/dav/calendars/USERNAME_REDACTED/privat/ HTTP/1.1" 404 5 "-" "Evolution/3.30.3"
IP_ADDRESS_REDACTED - - [29/Dec/2018:09:10:21 +0100] "OPTIONS /remote.php/dav/calendars/USERNAME_REDACTED/privat/ HTTP/1.1" 404 5 "-" "Evolution/3.30.3"
...

Maybe this is related to #12774? However, my installation seems to contain the fix from #12808.

Server configuration

Operating system: FreeBSD 11.2-RELEASE-p7

Web server: Nginx 1.15.8

Database: PostgreSQL 9.6.11

PHP version: PHP 7.1.25

Nextcloud version: 15.0.0

Updated from an older Nextcloud/ownCloud or fresh install: Updated

Where did you install Nextcloud from: FreeBSD ports

Signing status:

Signing status

Integrity checker has been disabled. Integrity cannot be verified.

List of activated apps:

App list

Enabled:
  - accessibility: 1.1.0
  - activity: 2.8.2
  - admin_audit: 1.5.0
  - bruteforcesettings: 1.3.0
  - calendar: 1.6.4
  - cloud_federation_api: 0.1.0
  - contacts: 3.0.1
  - dav: 1.8.0
  - federatedfilesharing: 1.5.0
  - federation: 1.5.0
  - files: 1.10.0
  - files_pdfviewer: 1.4.0
  - files_sharing: 1.7.0
  - files_texteditor: 2.7.0
  - files_trashbin: 1.5.0
  - files_versions: 1.8.0
  - files_videoplayer: 1.4.0
  - gallery: 18.2.0
  - logreader: 2.0.0
  - lookup_server_connector: 1.3.0
  - nextcloud_announcements: 1.4.0
  - notifications: 2.3.0
  - oauth2: 1.3.0
  - password_policy: 1.5.0
  - provisioning_api: 1.5.0
  - serverinfo: 1.5.0
  - sharebymail: 1.5.0
  - support: 1.0.0
  - survey_client: 1.3.0
  - twofactor_backupcodes: 1.4.1
  - updatenotification: 1.5.0
  - workflowengine: 1.5.0
Disabled:
  - comments
  - encryption
  - files_external
  - firstrunwizard
  - systemtags
  - theming
  - user_external
  - user_ldap

Nextcloud configuration:

Config report

<?php
$CONFIG = array (
  'instanceid' => 'REDACTED',
  'passwordsalt' => 'REDACTED',
  'secret' => 'REDACTED',
  'trusted_domains' => array (
    0 => 'REDACTED',
  ),
  'datadirectory' => '/usr/local/www/nextcloud/data',
  'overwrite.cli.url' => 'REDACTED',
  'dbtype' => 'pgsql',
  'version' => '15.0.0.10',
  'dbname' => 'REDACTED',
  'dbhost' => 'REDACTED',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'REDACTED',
  'dbpassword' => 'REDACTED',
  'installed' => true,
  'loglevel' => 2,
  'logtimezone' => 'UTC',
  'mail_smtpmode' => 'smtp',
  'mail_from_address' => 'REDACTED',
  'mail_domain' => 'REDACTED',
  'mail_smtphost' => 'REDACTED',
  'mail_smtpport' => '25',
  'mail_smtpsecure' => 'tls',
  'maintenance' => false,
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'theme' => '',
  'auth.bruteforce.protection.enabled' => false,
  'integrity.check.disabled' => true,
  'apps_paths' => array (
    0 => array (
      'path' => '/usr/local/www/nextcloud/apps',
      'url' => '/apps',
      'writable' => true,
    ),
    1 => array (
      'path' => '/usr/local/www/nextcloud/apps-pkg',
      'url' => '/apps-pkg',
      'writable' => false,
    ),
  ),
);

Are you using external storage, if yes which one: No

Are you using encryption: No

Are you using an external user-backend, if yes which one: No

[koppd]

The Android app CalDAV-Sync is also affected by this NC 15 change. However, there is a workaround in that app: enable preemtive authentification (in the extended setting).

[sven3]

I use Z-Push to offer ActiveSync services. It is configured to access NextCloud's CardDAV/CalDAV for contacts and calendar, which worked like a charm. It stopped working with NextCloud 15 due to 404s mentioned above.

[acidicX]

I can reproduce this with Evolution, however it only appears to be on one of my two nextcloud 15 instances. One is running on docker behind an nginx reverse proxy, the other directly with php-fpm behind nginx. The one on docker is producing the error. Might be a configuration / header issue (although I haven't changed anything from v14 onwards)?
Both were updated from v14.

[sven3]

My Nextcloud is upgraded from latest 14.x release. It runs under Apache 2.4.25 on amd64. I also didn't change anything regarding the headers. It worked with NC14 and refuses to work under NC15.

[mp1409]

My (faulty) Nextcloud is a plain Nginx and PHP-FPM setup. With @sven3 seeing the same problems under Apache & changes in the relevant code (#12774), I would be surprised if this is a simple configuration error.

@acidicX: Could you share the Nginx config of your working Nextcloud? Maybe we can spot the difference to mine and find a config workaround.

[acidicX]

I just checked again - I can reproduce the error now on the second instance as well.
It seems to be related to sharing permissions. My user "A" on the other nextcloud instance (the "working" one) had no shared calenders subscribed, only calenders shared for others by himself. I just tested it with another user "B", tried to get the calendar working in Evolution (subscribe with user A to calendar_shared_by_user_B) and now I'm getting the same error there as well :boom:

So @mp1409 you are right, it's probably not a config issue, but rather some bug regarding how the permissions are checked on the option requests by NC.

[BastienDurel]

I have the same error, nexcloud should return 401 and not 404 when unauthenticated access is requested (I have another server with plain old sabredav, and it does so)

[BastienDurel]

This behavior is related to commit ffd77780406532964bf617f105ff7bcf2ee9c8f7; which allows unauthenticated OPTIONS requests on any URL. But as (if comments are correct) it uses a fake tree, it cannot find the full path requested (it can only handle a single filename)

I've implemented a fix on my server, and filled a PR https://github.com/nextcloud/server/pull/13354 (disallowing anonymous OPTIONS on sub-directories)

[BastienDurel]

The PR Is merged, and the fix will be in 15.0.1