Server: Configuring S3 and OpenStack Swift as Primary Storage

Created on 12 Jun 2016 · 31Comments · Source: nextcloud/server

Currently it is only possible to mount S3, Google Drive, Dropbox etc. as external storage in a subfolder.
Mounting S3 as primary storage is implanted in the Enterprise Edition. This could be interesting for the community too, so I think we should rewrite it. What do you think?

Reference: https://doc.owncloud.org/server/8.0/admin_manual/enterprise_external_storage/s3_swift_as_primary_object_store_configuration.html

enhancement

Source

MariusBluem

👍7

Most helpful comment

It would be great to be able to mount Amazon S3 as primary storage !

lsteamgeo on 13 Jun 2016

👍10

All 31 comments

You could use a normal object store as primary storage (like Swift, but not S3) https://doc.owncloud.org/server/9.0/admin_manual/configuration_server/config_sample_php_parameters.html#using-object-store-with-owncloud

MorrisJobke on 13 Jun 2016

It would be great to be able to mount Amazon S3 as primary storage !

lsteamgeo on 13 Jun 2016

👍10

S3 is still generating lots of overhead connections. https://github.com/owncloud/core/issues/13147

despens on 1 Jul 2016

Ref: https://github.com/nextcloud/files_primary_s3

MariusBluem on 16 Jul 2016

Ref: https://github.com/nextcloud/files_primary_s3

Yes. I'm working on this. I have a working prototype locally. Will try to get this up this week.

MorrisJobke on 18 Jul 2016

👍5

also a migration mechanism would be _really_ nice then. I am very much looking forward to this feature if it's coming!

flypenguin on 7 Sep 2016

@MorrisJobke did this get anywhere? I'm just setting up NextCloud and would really really like this feature!

markseddon on 13 Sep 2016

@MorrisJobke did this get anywhere? I'm just setting up NextCloud and would really really like this feature!

This was taken over by @icewind1991 who did some code in https://github.com/nextcloud/server/pull/768

MorrisJobke on 13 Sep 2016

I would also be excited to see this come in the next release.

oquidave on 17 Sep 2016

...maybe we could also introduce a function in the external_files which allows admins to tick a box called "use as primary storage" for "group/user/all" 🤔

cc @icewind1991 @MorrisJobke

MariusBluem on 4 Oct 2016

So, the idea here is to have the external storage as the root storage of a user, meaning it's the only storage/directory he can write to, right? if I'm wrong ignore me.

The occ command seems to be capable of setting the mount point of external storage to / (root) with:
./occ files_external:config <Mount_ID> mount_point /
meaning you can mount for example a sftp share on / instead of needing a subfolder like /SFTP.

I tested this with external storage configured by the admin and set to one user being able to user it. Haven't tested other configurations.

As far as I can see only the web ui is blocking this. The above thing also seems to be resetted if you go into the configuration tab as admin.

matrss on 12 Oct 2016

👍1

the use case for _me_ would be that I use one s3 bucket for all data stored in owncloud. basically S3 replaces the data directory.

I don't know if I understand you correctly, you would want to do this on a per-user basis, right?

flypenguin on 12 Oct 2016

👍2

Agreed.

My use case is that I would like S3 (or whichever cloud storage is your poison!) to be the only storage for all the users, the same as @flypenguin. Admittedly I could just use something like s3fs-fuse but I'd prefer to use an actual plugin / functionality if it was offered.

markseddon on 12 Oct 2016

👍1

Yes, my approach would be on a per user basis. Also have not tested if this breaks other external storage because you would mount those inside of an external storage itself.
If I wanted the whole data directory inside of an external storage I would personaly mount it in the OS level.

matrss on 17 Oct 2016

@Risebell Performance and consistency-wise this seems like a pretty bad idea. S3 mounting might be a good idea locally, but not on 100+ user servers.

leinelissen on 17 Oct 2016

768 is merged

MorrisJobke on 24 Nov 2016

2315 now open :) . thx!

flypenguin on 24 Nov 2016

From what I can tell looking at the merge, this only covered known Amazon S3 endpoints with hard-coded regions. Is there plan to support S3 Compatible (i.e. Ceph) backends as primary storage. This is an OC Enterprise feature keeping me from going to NextCloud at the moment.

Also I should mention the lack of Swift Auth V1 support in OC Enterprise was a big problem, since object storage deployments using only Swift, and not all of OpenStack, use only the V1 Auth and do not have V2 (keystone) implemented.

jgallucci32 on 26 Nov 2016

From what I can tell looking at the merge, this only covered known Amazon S3 endpoints with hard-coded regions. Is there plan to support S3 Compatible (i.e. Ceph) backends as primary storage. This is an OC Enterprise feature keeping me from going to NextCloud at the moment.

This should work. We tested also with different local implementations of the S3 APIs.

Also I should mention the lack of Swift Auth V1 support in OC Enterprise was a big problem, since object storage deployments using only Swift, and not all of OpenStack, use only the V1 Auth and do not have V2 (keystone) implemented.

We plan to also update the underlying library to support new Auth levels. But I don't know what is currently supported. Maybe @icewind1991 knows more here.

MorrisJobke on 26 Nov 2016

Any news about this feature? I would also like to connect NextCloud with SwiftStack or OpenIO via S3 API.

@markseddon any idea if NextCloud + s3fs-fuse is a good idea in production? Never tried it.

Razva on 6 Feb 2017

Implemented! @Razva

https://docs.nextcloud.com/server/11/admin_manual/configuration_files/primary_storage.html

MariusBluem on 6 Feb 2017

🎉3

@MariusBluem is it production ready, 100+ users?

Razva on 6 Feb 2017

Feel free to test it in a staging-instance and report us about your experience - I think it is as production-ready as every other part of the Nextcloud server ...

Of course, if you find bugs - please report back!

You may also want to ask in our forum (https://help.nextcloud.com) about the experiences of other administrators - but, to be honest: I think it has not such a large user-base as maybe the calendar-app has.

If you need enterprise support or custom consulting about S3 implementation or Nextcloud in general (especially for larger instances in a critical eviroment) you are welcome to request a quote by the Nextcloud GmbH team over here: https://nextcloud.com/enterprise

MariusBluem on 6 Feb 2017

👍1

@MorrisJobke
Is S3 as primary storage working. https://github.com/nextcloud/files_primary_s3 is not found. what is the correct git url for this S3 implementation.

sumanyam on 3 Dec 2017

S3 as primary storage is part of the server and has no own repository. You will find instructions how to set this up in our documentation: https://docs.nextcloud.com/server/12/admin_manual/configuration_files/primary_storage.html

MariusBluem on 3 Dec 2017

Hello, Im wondering if would be possible to make the same thing with Google Drive. Thnx

sohighsoquiet on 27 Jan 2018

@MariusBluem That documentation doesn't appear to exist anymore and running a search for 'primary', 'primary storage' etc does not give any useful results :<

iJebus on 9 Mar 2018

Same for me....

zoic21 on 21 Mar 2018

@iJebus @zoic21

The instructions are still available for version 11, I've tested with 13 and you can still mount S3 as primary storage.

https://docs.nextcloud.com/server/11/admin_manual/configuration_files/primary_storage.html

My config file looks similar to Dan_Smith's file from here:

https://help.nextcloud.com/t/external-storage-set-amazon-s3-region/13254/7

'objectstore' => array(
    'class' => 'OC\\Files\\ObjectStore\\S3',
    'arguments' => array(
      'bucket' => 'bucket-name-in-here',
      'autocreate' => true,
      'key'    => 'awskeyinhere',
      'secret' => 'awssecretinhere',
      'use_ssl' => true,
      'region' => 'aws-region-in-here'
    ),
  ),

jdenly on 6 May 2018

how about mount the bucket as /var/www/nextcloud/data ? I am using the oss object storage offered by aliyun in China, and can mount it as a disk by using the ossfs forked from s3fs. And which performance concerns I should pay attention to?

Or better to mount the buckets follow the instruction https://docs.nextcloud.com/server/11/admin_manual/configuration_files/primary_storage.html and as

@jdenly did? But oss is a bit different as Amazon s3.

@iJebus @zoic21

The instructions are still available for version 11, I've tested with 13 and you can still mount S3 as primary storage.

https://docs.nextcloud.com/server/11/admin_manual/configuration_files/primary_storage.html

My config file looks similar to Dan_Smith's file from here:

https://help.nextcloud.com/t/external-storage-set-amazon-s3-region/13254/7
'objectstore' => array(
    'class' => 'OC\\Files\\ObjectStore\\S3',
    'arguments' => array(
      'bucket' => 'bucket-name-in-here',
      'autocreate' => true,
      'key'    => 'awskeyinhere',
      'secret' => 'awssecretinhere',
      'use_ssl' => true,
      'region' => 'aws-region-in-here'
    ),
  ),