Photoprism: Indexing freezes the whole host server then the process gets killed by system

[nicooo7875]

Hi,

First of all : you guys are doing a great job on photoprism ! keep it up, it's an awesome project !

Context :

• Raspberry pi 4 w/ 4GB RAM & a 4TB usb hdd
• Photoprism running in docker, alongside other containers (nextcloud, bitwarden, searx, traefik)
• photoprism arm64 image is frehly pulled from yesterday evening
• photos are stored on usb hdd, in a nextcloud directory
• nextcloud is used for mobile upload (i know i know)
• photoprism originals folder is bound to the nextcloud photos directory
• currently ~42500 photos in photoprism library

The issue :

1. i drop some pictures in a new directory in the nextcloud photos directory
2. once finished i switch to photoprism UI, library, index
3. photoprism finds the new directory, starts indexing
4. after 20/30 seconds, everything is frozen (photoprism, other nextcloud webpages, even ssh terminal sessions to the host server)
5. after some time (could be minutes, could be hours), i can access the server again

i do not get anything obvious when typing docker container logs photoprism, but i do see in dmesg on the host server that photoprism process was killed because the server ran out of memory.

indexing used to work (i have indexed thousands of pictures before) but i run into this error each time now (trying to index the same directory, that is true).

i would be grateful if you could tell me where to look to try to solve this ! i can provide logs if needed.

thanks.

[lastzero]

Thank you! :) Please add swap, so that spikes don't kill the process. You may also reduce the number of workers or disable TensorFlow.

Running photoprism next to all the other services requires more than 4 GB of memory for smooth operation. See system requirements in our docs.

[lastzero]

See https://docs.photoprism.org/getting-started/:

We recommend a modern computer with at least 2 cores and 4 GB of memory for running PhotoPrism. Indexing large photo and video collections significantly benefits from fast, local SSD storage and enough memory for caching.

If you're running out of memory, you can reduce the number of workers in the configuration. As a measure of last resort, you can additionally disable image classification using TensorFlow.

Number of workers can be limited with the PHOTOPRISM_WORKERS environment variable, see https://docs.photoprism.org/getting-started/config-options/.

[lastzero]

Is there anything more we can do?

[nicooo7875]

Sorry, busy times this weekend, I did not get a chance to test with less
workers yet. I will test with 1 or 2 I think and report back, hopefully
within 72 hours. Thanks.

Le mar. 15 sept. 2020 à 11:17, Michael Mayer [e-mail] a
écrit :

Is there anything more we can do?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/photoprism/photoprism/issues/497#issuecomment-692585730,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AP63DSITKIPFPVK4BJK6CBTSF4WJLANCNFSM4RHZSBHQ
.

[nicooo7875]

alright so i run a quick test with only 2 workers and it indexed everything smoothly, UI stayed responsive, and server stayed reachable. the indexing speed is acceptable for me with 2 workers. by checking with top on host raspberry pi i still have around 300MB of free RAM when indexing, so i am going to keep it that way for now and monitor what's going on the next time i index.

and everything was explained in the doc.... sorry for that !! and thanks again.