Core: Calendar and Contacts app empty after upgrade to 9.0.0.19

[Sp1l]

Steps to reproduce

1. Working installation of 8.2.2 with Contacts and Calendar app
2. Upgrade ownCloud to 9.0.0.19
3. Upgrade database via Web-UI
4. Download new Calendar and Contacts app
5. Replace app/calendar and app/contacts with the new 9.0 versions
6. Enable Calendar app, update database via Web-UI
7. Enable Contacts app, update database via Web-UI
   

   Expected behaviour

All calendar and contacts items visible

Actual behaviour

No calendars, calendar items, contacts, contact groups visible

Server configuration

FreeBSD 10.3-RC1

Apache 2.4.18 / PHP 5.6.19 / LibreSSL 2.3.2 / mod_h2 1.27 / mod_nghttp2 1.8.0

MariaDB 10.1.11

PHP 5.6.19 ZTS (from FreeBSD ports)

9.0.0.19

Upgraded from 8.2.2

Installed from FreeBSD ownCloud port

Web-server runs in a jail, firewalled to allow only inbound connections.

Signing status (ownCloud 9.0 and above):

No errors have been found.

List of activated apps:

Enabled:
  - activity: 2.2.1
  - calendar: 1.0
  - comments: 0.2
  - contacts: 1.0.0.0
  - dav: 0.1.5
  - federatedfilesharing: 0.1.0
  - federation: 0.0.4
  - files: 1.4.4
  - files_pdfviewer: 0.8
  - files_sharing: 0.9.1
  - files_texteditor: 2.1
  - files_trashbin: 0.8.0
  - files_versions: 1.2.0
  - files_videoplayer: 0.9.8
  - firstrunwizard: 1.1
  - gallery: 14.5.0
  - notifications: 0.2.3
  - provisioning_api: 0.4.1
  - systemtags: 0.2
  - templateeditor: 0.1
  - updatenotification: 0.1.0
  - user_external: 0.4
Disabled:
  - encryption
  - external
  - files_external
  - ownnote
  - user_ldap

The content of config/config.php:

{
    "system": {
        "instanceid": "532d79850c74c",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "datadirectory": "\/home\/owncloud",
        "dbtype": "mysql",
        "version": "9.0.0.19",
        "installed": true,
        "maintenance": false,
        "theme": "",
        "trusted_domains": [
            "brnrd.eu",
            "cloud.example.net"
        ],
        "dbname": "owncloud",
        "dbhost": "db.example.net",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "forcessl": true,
        "ldapIgnoreNamingRules": false,
        "loglevel": "1",
        "user_backends": [
            {
                "class": "OC_User_IMAP",
                "arguments": [
                    "{imap.example.org:144\/imap}"
                ]
            }
        ],
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trashbin_retention_obligation": "auto"
    }
}

Are you using external storage, if yes which one: None

Are you using encryption: no

Are you using an external user-backend, if yes which one: IMAP

There are no commands defined in the "ldap" namespace.

Client configuration

Firefox 45.0

Windows 10 Pro

Logs

Web server error log

[Wed Mar 09 16:43:09.446812 2016] [auth_digest:notice] [pid 99209:tid 34393318400] AH01757: generating secret for digest authentication ...
[Wed Mar 09 16:43:09.450851 2016] [http2:info] [pid 99209:tid 34393318400] mod_http2 (v1.2.7, nghttp2 1.8.0), initializing...
[Wed Mar 09 16:43:09.555939 2016] [mpm_event:notice] [pid 99209:tid 34393318400] AH00489: Apache/2.4.18 (FreeBSD) LibreSSL/2.3.2 PHP/5.6.19 configured -- resuming normal operations
[Wed Mar 09 16:43:09.556047 2016] [core:notice] [pid 99209:tid 34393318400] AH00094: Command line: '/usr/local/sbin/httpd -D NOHTTPACCEPT'
[Wed Mar 09 20:49:25.217988 2016] [:error] [pid 99233:tid 34393338880] [client 172.17.2.172:54279] PHP Fatal error:  Call to a member function getRequest() on null in /usr/local/www/owncloud/lib/private/response.php on line 77
[Wed Mar 09 20:57:24.935239 2016] [:error] [pid 99233:tid 34393342976] [client 172.17.2.172:54336] PHP Fatal error:  require_once(): Failed opening required '/appinfo/v2/remote.php' (include_path='/usr/local/www/owncloud/3rdparty/phpseclib/phpseclib/phpseclib:/usr/local/www/owncloud/3rdparty/pear/console_getopt:/usr/local/www/owncloud/3rdparty/pear/archive_tar:/usr/local/www/owncloud/3rdparty/pear/pear_exception:/usr/local/www/owncloud/3rdparty/pear/pear-core-minimal/src:/usr/local/www/owncloud/lib/private:/usr/local/www/owncloud/config:/usr/local/www/owncloud/3rdparty:/usr/local/www/owncloud/apps:/usr/local/www/owncloud/lib:.:/usr/local/share/pear:/usr/local/www/owncloud') in /usr/local/www/owncloud/remote.php on line 137
[Wed Mar 09 21:15:19.891064 2016] [negotiation:error] [pid 99210:tid 34393351168] [client 172.17.2.172:54483] AH00687: Negotiation: discovered file(s) matching request: /usr/local/www/owncloud/settings/admin (None could be negotiated).
[Wed Mar 09 21:15:47.015568 2016] [negotiation:error] [pid 99233:tid 34393333760] [client 172.17.2.172:54489] AH00687: Negotiation: discovered file(s) matching request: /usr/local/www/owncloud/settings/admin (None could be negotiated).
[Wed Mar 09 21:18:24.400925 2016] [negotiation:error] [pid 99212:tid 34393352192] [client 172.17.2.172:54513] AH00687: Negotiation: discovered file(s) matching request: /usr/local/www/owncloud/settings/admin (None could be negotiated).

ownCloud log (data/owncloud.log)

{"reqId":"rzZmVB6wTcYJSFMxc3N\/","remoteAddr":"172.17.2.172","app":"core","message":"Could not get application: cURL error 7: ","level":3,"time":"2016-03-09T19:22:38+00:00"}
{"reqId":"78a9eLAfXEzZdJ+JFpGU","remoteAddr":"172.17.2.172","app":"PHP","message":"require_once(\/appinfo\/v2\/remote.php): failed to open stream: No such file or directory at \/usr\/local\/www\/owncloud\/remote.php#137","level":3,"time":"2016-03-09T19:57:24+00:00"}
{"reqId":"78a9eLAfXEzZdJ+JFpGU","remoteAddr":"172.17.2.172","app":"PHP","message":"require_once(): Failed opening required '\/appinfo\/v2\/remote.php' (include_path='\/usr\/local\/www\/owncloud\/3rdparty\/phpseclib\/phpseclib\/phpseclib:\/usr\/local\/www\/owncloud\/3rdparty\/pear\/console_getopt:\/usr\/local\/www\/owncloud\/3rdparty\/pear\/archive_tar:\/usr\/local\/www\/owncloud\/3rdparty\/pear\/pear_exception:\/usr\/local\/www\/owncloud\/3rdparty\/pear\/pear-core-minimal\/src:\/usr\/local\/www\/owncloud\/lib\/private:\/usr\/local\/www\/owncloud\/config:\/usr\/local\/www\/owncloud\/3rdparty:\/usr\/local\/www\/owncloud\/apps:\/usr\/local\/www\/owncloud\/lib:.:\/usr\/local\/share\/pear:\/usr\/local\/www\/owncloud') at \/usr\/local\/www\/owncloud\/remote.php#137","level":3,"time":"2016-03-09T19:57:24+00:00"}
{"reqId":"z6YLL5Jix2ubPWgs+6AO","remoteAddr":"172.17.2.172","app":"core","message":"Could not get categories: cURL error 7: ","level":3,"time":"2016-03-09T20:15:14+00:00"}

Browser log

n.a.

[LukasReschke]

@Sp1l Could you open your browers' network console when opening the contacts and calendar app and posting the request data as well as the response here? Thanks a lot.

Please remove all "cookies" and other sensitive data from the output :)

[LukasReschke]

cc @DeepDiver1975

[Sp1l]

Couldn't attach it, so here's output from the network console

[m4tz3]

same Problem here, no calendar and contacts any more...

[Sp1l]

Issue solved!

./occ dav:migrate-addressbooks
# for every user
./occ dav:migrate-calendars <username>
./occ dav:sync-birthday-calendar

All my calendars, contacts (incl groups) have returned.

[laurentperrinet]

Running the command above reports that migration is done:

# php occ  dav:migrate-calendars user
Start migration for user

still, all calendar data is gone on the web UI (using firefox).

I am running owncloud on a QNAP server with a mysql DB, I can see the calendar tables using phpMyAdmin.

[ghost]

If only calendar is missing make sure that you're not using IE/Edge: https://github.com/owncloud/calendar/issues/309

[j4mb4l4j4]

Hi, I had the same Problem that my Calendar was gone.

"php occ dav:migrate-calendars user "

didn't worked.

I had to play in a backup from 8.2.2 to get my Calendar back... Thats not so good.
I was using Google Chrome.

Is there a way to export the calendar and reimport it after the Update ? Or is there a workaround, so that I may should wait for the next owncloud Version?

I am using:

• PHP 5.5.9-1ubuntu4.14 (cli) (built: Oct 28 2015 01:34:46)
• MariaDB Version 10.0.24-MariaDB-1~trusty
• innodb_version 5.6.28-76.1

[laurentperrinet]

similarly, I could retrieve my data by rolling back to a backup from 8.2.2.

[Sp1l]

My calendar items were still in oc_clndr_objects. After running the command they appeared in oc_calendarobjects

[MorrisJobke]

@DeepDiver1975 I reopen this as this is the same IMAP user backend issue we talked about today. The IMAP user backend doesn't seem to be loaded during migration.

cc @nickvergessen because you fixed something similar.

@ltsavar noticed this.

[DeepDiver1975]

@DeepDiver1975 I reopen this as this is the same IMAP user backend issue we talked about today. The IMAP user backend doesn't seem to be loaded during migration.

During the regular migration the imap user backend is not loaded and therefore the users are not know.
But if you run the occ command afterwards this should work out from my understanding - right?

The overall fix for this is the long discussed user accounts table ..... :disappointed: it will not be done in 9.1

[DeepDiver1975]

basically we need this https://github.com/owncloud/core/issues/23558

[PVince81]

@DeepDiver1975 any update ? Not possible without the accounts table ?

[DeepDiver1975]

The only Alternative in the future (9.1+) would be to use a live Migration step ....

[PVince81]

@DeepDiver1975 so should we move this to 9.2 then ? Didn't you add a live migration in 9.1 recently ?

[PVince81]

also: is there a possible workaround ?

[DeepDiver1975]

also: is there a possible workaround ?

he workaround it to run the command on cli.
live migration steps are not available in 9.0 - only in 9.1 but 9.1 no longer requires calendar and contacts migration.

I vote for close - workaround is given

[Flachzange]

It seems that I have the same issue as described but the given workaround using the cli command does not work as the old database tables for calendars and addressbooks are gone.

I was running 8.2.7 when I attempted the upgrade to 9.0.4. The update itself went fine besides the usual remarks on disabled apps.

After the upgrade, all contacts and calendars were missing. I did some research and found this thread amongst others. I use the IMAP user backend as well.

Starting the migration manually using the cli commands resulted in "no tables found". Looking at the database, it seems that all old tables from 8.x were gone.

I had to restore from an backup to get contacts and calendars back.

[DeepDiver1975]

The old tables are not touched during the upgrade from 8.2 to 9.0. this will happen with the upgrade to 9.1.

No idea why the tables are gone for you. It wasn't us!

[Flachzange]

Well, then this is the reason. I did another upgrade from 9.0.4 to 9 (without knowing that calendars/addressbooks needs to be migrated).

How is the migration from 8.x to 9.1 intended to work with the IMAP user backend as long as

1. the migration does work automatically during the upgrade,
2. dav migration cli commands are removed in 9.1, and
3. old tables are removed in 9.1?

Thank you.

[DeepDiver1975]

You cannot move from 8.2 to 9.1. you have to move to 9.0 first.

The Migration has to be execution manually in your case using imap auth.

[Flachzange]

Alright, thanks. I hope this also made clear to all others using IMAP user backend.

[ghost]

Probably something which should be documented? (https://github.com/owncloud/documentation/issues)

[Flachzange]

Done.

https://github.com/owncloud/documentation/issues/2533

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.