Meteor-files: Change storagePath onBeforeUpload

[HugoHeneault]

Hi there,

Your Meteor-Files is gorgeous, but we need some extra config and found nothing which fits our needs.

We're building a CMS for hundreds (and more) websites and need to save files on different folders based on the client.
Different collection is not an option for us.

Would it be possible to change the storagePath in onBeforeUpload, depending on our client storage folder?
We've seen some interesting things here: https://github.com/VeliovGroup/Meteor-Files/issues/99
But this is not allowing us to call a promise returning the folder of the uploading client.

How would you achieve that?

Thanks a lot for your help, and your great tool!

[dr-dimitru]

Hi @HugoHeneault ,

I'm looking forward this feature too.
Although this feature is wanted by many, no one yet suggested a way to do so in trusted (_Server_) code without security breach.

If you can describe how would you configure storagePath (_how you will construct it_) when client initiates upload - I'll implement it.

[dr-dimitru]

Different collection is not an option for us.

Why? If each user has its very own collection for files, sounds not bad.

[HugoHeneault]

Hi!

Thanks for your quick reply! And for your help.

We have a static folder outside of our app, and we put all our files into it.
Files would be stored in static/website_id.

Then we'll have a Meteor route reading files if the client is allowed and for public files, an nginx proxy pointing on every client static folder.

Do you need more explanation?

[HugoHeneault]

Different collection is not an option for us.

Why? If each user has its very own collection for files, sounds not bad.

Having hundreds (or thousands) of collections sounds quite bad for us. But we don't have any experience with huge number of collections. Maybe it's not a problem?

[dr-dimitru]

You need to dig into it. But queries will be much faster on collection with less records.

Back to your request, static/website_id my question was how would you get this website_id?

[HugoHeneault]

Have you any experience with thousands of collections?

We'll call a Meteor method that returns the website_id from the Meteor
user.

[jonathanredford]

I'm in a similar situation to Hugo.

I'm wanting to change the storagePath based on the file size (ie. if > 5GB upload to folderA, else upload to folderB)

Is this possible?

[dr-dimitru]

@jonathanredford currently not

@HugoHeneault I'll look how we can implement it is secure and non-breaking way
Current problem user and even file is not available on every steps of upload

[HugoHeneault]

Cool! Maybe a way to edit the folderPath onBeforeUpload?

[dr-dimitru]

every chunk is needs a path to file
onBeforeUpload is only for meta-data
I'll think this way, as onBeforeUpload called on every-incoming chunk

[HugoHeneault]

Ok. So how would you do it? Can I help?

[dr-dimitru]

Now - no idea, requires major changes

[HugoHeneault]

Oh..
How long would it take?
Do I have to figure out something with another module?

[dr-dimitru]

Usually I proceed all feature requests within two weeks

[HugoHeneault]

Cool! Waiting for your update!!

[HugoHeneault]

I will use this solution until you proceed this request: https://github.com/VeliovGroup/Meteor-Files/issues/228#issuecomment-245929344

[ck23onGithub]

Hello.

I also encountered the need to save files in different folders.

What i did was to use the namingFunction in the construction of the collections.
The path of the your file can be determined by putting a folder name before your file name.

example:
file to be uploaded: sample.txt
target folder to upload: filesystem/sampleFiles/

therefore, what you do is use the namingFunction to rename your file from sample.txt to /sampleFiles/sample.txt

this way you are actually directing the file to be saved somewhere inside your server.

i hope this helps. and perhaps this maybe the solution for the dynamic storage path requested by many.

[oeway]

This feature is something I would like to have too, for example each user have a folder.

[dr-dimitru]

Working on it

[dr-dimitru]

@oeway lake a look on https://github.com/noris666/Meteor-Files-POST-Example/blob/master/avatars.example.js for cURL uploads

[dr-dimitru]

Implemented in v1.7.4
Please use storagePath passed as _Function_ to change path on _FS_ dynamically

Please check on your end

[valorloff]

Hi! Big thanx for storagePath function!
my admin say's, that for performance reasons, in folders max count of files should not exceed ~10k (images for website), and i want use your new function for chunking portions of files between folders for uploads,
is it right way? your opinion?

[dr-dimitru]

Hello @valorloff ,

I'm glad you found this useful. Please see this comment where I show how to generate very deep (16^17 of variations) folder structure. You're free to optimise it