Photos: External local storage is not picked up

[rajil]

Hello,
I have an local external storage where i am exposing my digikam organized photos. Unfortunately, the Photos app does not pick up this folder. Is there a setting any where to select folders for the Photo app?

[skjnldsv]

Hello!
This app is indeed not compatible with external storage for performances reasons.
I'm still not sure thi sis something we'll be able to do or if we even want to dive into this yet :)

cc @rullzer @icewind1991 & @jancborchardt

[SimplyCorbett]

That's disappointing. Why not allow users to choose which external storage will work on photos instead? It doesn't have to be a universal setting.

For example I might keep photos on another server and connect through NAS. Since my primary storage is S3 I'm not able to mount or symbolic link the folder into my nextcloud data directory.

Or maybe I'm keeping them on wasabi because API requests and download is free.

[S74HK9hV]

I upgraded one of my instances to NC18 already and didn't realize that external shares aren't working.

I'm glad I saw this post before upgrading my main NC server. This basically breaks Nextcloud for me in terms of photos.

There should be a big red flag in the NC18 changelog about this. The fact that there is no option to use the old gallery app, and the new one basically won't work for a lot of us

I imagine there are lots of people who share their photo collection via external storage. Photo collections are generally quite large and better suited for storage on a dedicated NAS device. I would also imagine there are a lot of other people with inexpensive NAS that aren't capable of running nextcloud on the same hardware.

@skjnldsv what are the performance issues with external storage? the old gallery app seemed to work just fine. admittedly, it wasn't the fastest but it worked well enough to make it a compelling alternative to uploading/sharing things on google

[lgrawet]

In my case I use "external storage" to connect local samba storage. I need to access the storage booth ways (nextcloud and samba) and I also need to preserve user permissions at OS level.

[SimplyCorbett]

Hello!
This app is indeed not compatible with external storage for performances reasons.
I'm still not sure thi sis something we'll be able to do or if we even want to dive into this yet :)

cc @rullzer @icewind1991 & @jancborchardt

@skjnldsv I would appreciate it if you expanded on this.

[rajil]

My nextcloud data directory and the local photos/digikam folder share the same file system. If I do a symlink my external local directory somewhere in the data directory, will photos index it?

[skjnldsv]

My nextcloud data directory and the local photos/digikam folder share the same file system. If I do a symlink my external local directory somewhere in the data directory, will photos index it?

Hey, if your data directory is just hard linked, yes, this is the job of your underlying file system, same goes for any external storage mounted in the os level.
Only the external storage app is not used here :)

[SimplyCorbett]

Can we get clarification on the performance impact external storage has on this application? I can live with it if there is a legitimate performance concern but if the external storage is on the local filesystem/LAN or low latency I can't see it being a problem.

Is the performance concern related to external storage or the method in which nextcloud handles the external storage app? Is it a privacy issue? Gallery was slow at generating previews but once they were generated I never noticed a performance impact on external storage.

Thank you.

[alarmix]

Reading this too late, is there a painless way to revert to NC17? For me, as presumably for many others, the new photo app is absolutely useless without external storage support!
Agree that there SHOULD have been a VERY big red notice about it in the changelog!!

[alarmix]

FYI: the "old" NC17 Gallery app IS working OK with NC18!
Instructions in this thread:
https://help.nextcloud.com/t/new-photo-app-in-nextcloud-18/69949/20

[skjnldsv]

Can we get clarification on the performance impact external storage has on this application? I can live with it if there is a legitimate performance concern but if the external storage is on the local filesystem/LAN or low latency I can't see it being a problem.

Is the performance concern related to external storage or the method in which nextcloud handles the external storage app? Is it a privacy issue? Gallery was slow at generating previews but once they were generated I never noticed a performance impact on external storage.

Hey, sorry about the delay :)
So, I'm not the best expert, but as far as I understood, this is about setting up the full storage backend when initializing. Currently only the timeline view doesn't include the external storage. The my albums section have it.

Let me ping our expert on this! :wink:

[jospoortvliet]

FYI, we are trying to fix regressions asap.

In this case, the new Timeline view does not show external storage for performance reasons as @skjnldsv mentions, but the My Albums view should (which is one of the reasons why it is much slower... But still a bit faster than Gallery was).

See https://help.nextcloud.com/t/new-photo-app-in-nextcloud-18/69949/32 for a more complete response.

If the My Albums view does not show external storage, please let us know - that would be a bug.

Long term, we'd like to improve performance to the point that we can always show external storage and shared images, without getting slow like the Gallery did, but this is a big task and won't be done soon.

(and no, I'm not the 'expert' @skjnldsv referred to, that would be @icewind1991 whom I talked to)

[jospoortvliet]

I think this means we can close this issue - not showing external storage in the timeline is, for now, a known limitation. Pls report a new issue if your images on external storage don't show in the Albums view.

[dassio]

how can you close an issue if it is not fixed, it is not an limitation, it is an bug .

[rullzer]

You should see external files in album overviews.
However on the 'your photos' it is indeed a known limitation. External storages don't allow us to query this efficiently.

[dhrunia]

Album overview is not showing photos from external storage. It shows the external storage as a folder but when I click on that folder it says No photos in here

My configuration:
External storage: An account on Nextcloud 18.0.3
My Nextcloud version: 18.0.4

[dassio]

Album overview is not showing photos from external storage. It shows the external storage as a folder but when I click on that folder it says _No photos in here_

My configuration:
External storage: An account on Nextcloud 18.0.3
My Nextcloud version: 18.0.4

you maybe facing this issue: https://github.com/nextcloud/photos/issues/267
the fetching or preview generating is taking too long and before anything is sent back to your browser, the photo app tell you there is nothing there

[mailinglists35]

This app is indeed not compatible with external storage for performances reasons.

is this the only limitation? if yes, could you please let users enable it?

I have nextcloud in a vm, but the access to the storage is as fast as the bare metal hosting hypervisor (linux host, linux guest, virtio networking to bare metal samba server).

this is very frustrating.

however, this only seem to happen on web interface - the ios client actually does seem to trigger generating previews as you browse the "Media" tab and sort them in a single chronologically timeline from all over the places they are found on external storage

[skjnldsv]

is this the only limitation? if yes, could you please let users enable it?

yes, that is the only limitation :/
The issue is not for the user, but for the admin. Some ext storage backends like you mentioned can be super reactive, but some are awful and can lead to complications really quick.

[mailinglists35]

is this the only limitation? if yes, could you please let users enable it?

yes, that is the only limitation :/

The issue is not for the user, but for the admin. Some ext storage backends like you mentioned can be super reactive, but some are awful and can lead to complications really quick.

sorry, when I said "user" I was thinking of the admin of the instance as the nextcloud "user" :)

do you think it would technically be possible to let admins enable a configuration parameter?

[skjnldsv]

do you think it would technically be possible to let admins enable a configuration parameter?

Yes, let's see how we can add this.
Let's try add the external storage and see how it goes, if it's that bad, we'll add a config to disable it by admins.

[knuxify]

Alternatively, we could query only certain types of storage by default (for example local storage or fast storages on the local network) and leave it off by default on the rest.

[mailinglists35]

Alternatively, we could query only certain types of storage by default (for example local storage or fast storages on the local network) and leave it off by default on the rest.

I would be happier to be able to control the option instead of letting software decide :D

[knuxify]

Alternatively, we could query only certain types of storage by default (for example local storage or fast storages on the local network) and leave it off by default on the rest.

I would be happier to be able to control the option instead of letting software decide :D

I'm talking about defaults, of course the user (or admin) can set everything themselves :)

Also, I thought about this a bit more after making that initial reply, and I think the best option would be to let admins choose which external storages they want to enable and which ones they want to keep turned off, but we'd need to add a new category for this in the admin settings and I'm not sure if it's possible to get the external storages from within the code for this app (but I'd assume there's an API for this).

[mailinglists35]

until this issue is properly solved, is it safe in current release version to manually bind mount the external storage to the user folder (I being the only user of NC)?

[dassio]

until this issue is properly solved, is it safe in current release version to manually bind mount the external storage to the user folder (I being the only user of NC)?

this issue is only related to photos apps, of course you can mount external to user folder, it works with the server core app, i have been doing this for a very long time

[quytx]

Could we at least add support for External Storage of "Local" type? Performance wouldn't be an issue in that case I assume.

[PeeBeerBeach]

I also have Fotos on an external storage of the type "local" and am a bit disapointed that i cannot properly use that within the photo app.
Bind mounting the dir seems to be a solution, but i need to make it accessible to at least 2 users. Do i have to bind mount it 2 times then into the respective user folders or is there a way to mount it to a globally accessible path?

[ioneng]

I just noticed that photos app in Android client is able to load all my photos on local external storage. I'm not sure whether this is new to the latest Android client or it has always been so. I'm on server version 20.0.1

[xppx99]

I agree with the @quytx suggestion, at least allow for the storage "Local" type, as performance surely wont be an issue.
Thanks

[RoL0NL]

I also have Fotos on an external storage of the type "local" and am a bit disapointed that i cannot properly use that within the photo app.
Bind mounting the dir seems to be a solution, but i need to make it accessible to at least 2 users. Do i have to bind mount it 2 times then into the respective user folders or is there a way to mount it to a globally accessible path?

I can't get Nextcloud to pick up the contents of these binded directories (docker bind to -fpm container). They just appear as empty directories. Do I have to run any command to 'refresh' the database? Nevermind, just run occ files:scan --all

OT: This is the biggest issue for me keeping me from committing to Nextcloud. My large photo collection is stored in a ZFS pool and I don't want to 'embed' it into the Nextcloud infrastructure.

[hmlendea]

This is a pretty big issue.
I started setting up Nextcloud 2 days ago and I'm glad I found this early since it is a bit of a deal breaker for me.

"A bit" beacause as @RoL0NL mentioned, the Android app does show everything in the Photos tab, including external storage photos. So that's ok. But I really want them in the web interface also. Not to mention that I am very worried this feature will be removed from Android as well.

[rubbo898]

Could we at least add support for External Storage of "Local" type? Performance wouldn't be an issue in that case I assume.

That's the POINT, imho "performance issues" is a joke if we are talking about a Local external storage...

So this it's not even a WIP? If so there is no way for me to fit AS IS nextcloud to my needs...what a shame