Contacts: Large addressbook makes browser slow/unresponsive

Created on 3 Nov 2017 · 18Comments · Source: nextcloud/contacts

A customer has a large address book with more than 13'000 contacts. When opening the contacts app it takes more than 3 minutes to load the contacts and chromium shows the page-is-unresponsive message several times. When the contacts are finally loaded, every further action takes at least a half a minute to be performed. The whole tab the uses more than 700 MB of RAM. There should be a better memory handling as the actual setting makes the contacts app nearly unusable with many contacts.

Steps to reproduce

Have more than 13'000 contacts in one address book.
Open contacts app.

Expected behaviour

Contacts get loaded fast. Only the first contacts should load and as you scroll down, the rest should get downloaded.

Actual behaviour

Contacts load very slow, and browser nearly crashes.

Server configuration

Operating system:
Debian 8.9
Web server:
Nginx
Database:
Mysql
PHP version:
5.6
Nextcloud version: (see Nextcloud admin page)
12.0.2
Contacts version: (see Nextcloud apps page)
1.5.3
Updated from an older Nextcloud or fresh install:
Fresh install
Signing status:
No errors have been found.
List of activated apps:

Enabled:
  - activity: 2.5.2
  - comments: 1.2.0
  - contacts: 1.5.3
  - dav: 1.3.0
  - federatedfilesharing: 1.2.0
  - federation: 1.2.0
  - files: 1.7.2
  - files_external: 1.3.0
  - files_pdfviewer: 1.1.1
  - files_sharing: 1.4.0
  - files_texteditor: 2.4.1
  - files_trashbin: 1.2.0
  - files_versions: 1.5.0
  - files_videoplayer: 1.1.0
  - firstrunwizard: 2.1
  - gallery: 17.0.0
  - logreader: 2.0.0
  - lookup_server_connector: 1.0.0
  - nextcloud_announcements: 1.1
  - notifications: 2.0.0
  - oauth2: 1.0.5
  - password_policy: 1.2.2
  - provisioning_api: 1.2.0
  - serverinfo: 1.2.0
  - sharebymail: 1.2.0
  - survey_client: 1.0.0
  - systemtags: 1.2.0
  - theming: 1.3.0
  - twofactor_backupcodes: 1.1.1
  - updatenotification: 1.2.0
  - user_external: 0.4
  - workflowengine: 1.2.0
Disabled:
  - admin_audit
  - encryption
  - user_ldap

Nextcloud configuration:

{
    "system": {
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "http:\/\/localhost",
        "dbtype": "mysql",
        "version": "12.0.2.0",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "localhost",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "ocsfx00acnxt",
        "loglevel": 0
    }
}

Are you using external storage, if yes which one: local/smb/sftp/...
no
Are you using encryption: yes/no
no
Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
no

Client configuration

Browser:

Chromium Version 57.0.2987.98

Operating system:

Debian 8.9

CardDAV-clients:

Logs

Web server error log

no errors

Nextcloud log (data/nextcloud.log)

nothing interesting

Browser log

no messages in console

needs info

Source

djermann

👍1

All 18 comments

@djermann The contacts app by now handles large amounts of contacts in a bad way, there is no question about it. The problem for me is, that I have no real idea on how to improve this situation. Maybe you can share the use case of such a high number of contacts and how we should improve the handling? E.g.

How to choose which contacts to initially load?
Completely filter out specific contacts?
What information is valuable of each contact?
etc.

irgendwie on 3 Nov 2017

I can think of different possible approaches to solve this.

When a user opens the contacts app, it only shows the first 10-20 contacts in alphabetical order. If there was a way to detect which contacts are visible in the webinterface, then the contacts app would only download these plus maybe 100 more. Later the user maybe decides to scroll down, so the already downloaded 100 more contacts would be displayed, while in the mean time it already downloads another 100 contacts.
When opening the contacts app, it could just download the first say 100 contacts and when the user scrolls down to the 100. contact, a button appears that says something like 'load more contacts'. Then it would download another 100 contacts.
One could also split the contacts in pages of 100.
One could split the contacts into pages of the first letter. So all contacts whose last name starts with an A would appear on the page named 'A'. Although, this behavior would only make sense when having more than a certain amount of contacts (e.g. 200).

The same behavior should also apply for the results of a search.

djermann on 3 Nov 2017

👍1

Let me point your attention to the CardDAV plugin for Roundcube (https://github.com/blind-coder/rcmcarddav), which is a CardDAV client, programmed in PHP, which works perfectly well with my Nextcloud server. I must admit, that I only have 600 contacts in my address book, but it might give you an idea how it could look like.
Right after a CardDAV address book has been set-up an initial address synchronization takes place, which will not be necessary for Nexctloud because you can directly work on the database. The address book is presented in a three column view, the first column shows the address book and the groups, the second column shows the contacts and the third column shows the address details if an address is selected. So it more or less uses the same layout as the contacts app in Nextcloud. This is how it looks like:

20171108_addressbook

It seems that the contact groups, the first and last names, and the email address, are loaded at once (or maybe in chunks) and the address details are only loaded when a contact is selected. (Loading ... is displayed for a fraction of a second). This reduces the amount of data which need to be kept in memory and makes usage really fast.

j-ed on 8 Nov 2017

@j-ed aside from the chunk loading, we're actually loading our info like that :)

skjnldsv on 8 Nov 2017

@skjnldsv Ok, that's new to me. I wonder where the huge performance differences are coming from if I compare both applications 😕

j-ed on 8 Nov 2017

We have also encountered a problem after upgrading the contacts app to 2.0.1. On the Firefox browser version 52 on Linux, large contacts (in our case 398 and 4908) are not showing and only the spinning wheel appears. I had a look on the log files, as well on the browser console but I did not find anything related to this.

I tested this on different browsers and it worked for the browsers below. I have 4908 contacts in my Nextcloud installation but it still takes more than 5 minutes to load on different browsers, which makes the app not usable for me or any users who has many contacts.

Chrome (version 57.0.2987.98 on Linux)
Vivaldi (version 1.11.917.43 on Linux)
Firefox (version 42 and version 56 on Windows)
Internet Explorer (version 11 on Windows)

CamZie on 10 Nov 2017

👍1

Hello everyone! Is this still a big issue? Or are our latest optimisation good enough? :)

skjnldsv on 28 Feb 2018

Latest 3.0.0 should be fast enough :)

skjnldsv on 28 Sep 2018

The app/contacts page still puts "too much" stress on my browser (FF 66), which warns me about slow script and asks me whether I'd prefer to shut it up. That is with NC 15.0.5 and Contacts 3.0.5, ~ 8k Contacts in a adress book shared with me. It is certainly not technically unusable and synchronisation with Thunderbird/Cardbook is pretty fast; but the Web-Ui takes a minute to load and is laggy afterwards, with a warning about a slow script more or less every time interacting with it.

fwolfst on 26 Mar 2019

but the Web-Ui takes a minute to load and is laggy afterwards, with a warning about a slow script more or less every time interacting with it.

Which make sense. Browsers cannot handle that many things at once.
We did some pretty intense optimisations, but it still is what it is: a javascript application.
We will never be as fast and reactive as a desktop app as cardbook or something else and we'll never be :(

Though 8k users seems pretty crazy. I'm wondering what is the bottleneck here, the loading request? The display of all of those users? :thinking:

skjnldsv on 26 Mar 2019

We will never be as fast and reactive as a desktop app as cardbook or something else and we'll never be :(

Well - yes and no. One could put the load on the server, like in "traditional" web-applications and only render what is needed, paginate etc. But I assume that the architecture wants to have most of the accessible data already at client side to do the work there (in my oppinion that is the "crazy" part, not the amout of contacts). By the way, you wrote "Users", but its really about contacts. Although there is ~100 at least potential Users involved too.

And I don't want to blame. Thanks for the optimiziations! I guess the the choice of using a heavy (?) JavaScript-Client-Framework also has had good consequences.

fwolfst on 26 Mar 2019

By the way, you wrote "Users", but its really about contacts. Although there is ~100 at least potential Users involved too.

I indeed wanted to write contacts :)

And I don't want to blame. Thanks for the optimiziations! I guess the the choice of using a heavy (?) JavaScript-Client-Framework also has had good consequences.

Don't worry, I was not taking it like that :)
It's not really a choice here. Nextcloud is just a php environment. You're talking about server-side rendering, but that would require a whole rework of the architecture we're working on. There are some options our there, yes, but it would complicate things far more than it would help us being able to properly maintain such applications :wink:

I will try to check if there are some other optimisations to be done here :)

skjnldsv on 26 Mar 2019

I'm wondering what is the bottleneck here, the loading request? The display of all of those users? thinking

Just from looking at the "Network" tab of Dev Tools in Firefox: Loading the 5MB data of the larger adressbook takes a while, but then comes a long period of no network activity until a PROPFIND to a specific .vcf path is done (I guess that is the currently selected/first contact).

fwolfst on 26 Mar 2019

but then comes a long period of no network activity until a PROPFIND to a specific .vcf path is done (I guess that is the currently selected/first contact).

how long (roughly)? :)

skjnldsv on 26 Mar 2019

A minute. My machine is not top notch, but imho sufficiently powered (5gb
ram, 4x 2.6ghz core).

On Tue, Mar 26, 2019 at 4:10 PM John Molakvoæ notifications@github.com
wrote:

but then comes a long period of no network activity until a PROPFIND to a
specific .vcf path is done (I guess that is the currently selected/first
contact).

how long (roughly)? :)

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/nextcloud/contacts/issues/428#issuecomment-476694015,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AANY7WoYmndkx9c17JpD3KP7mgpxslu8ks5vajh5gaJpZM4QRF9p
.