Uppy: Dropped files that are named same in sub-folders skipped?

[genachka]

Using dashboard plugin, I see that it parses the sub-folders if I drop a folder (is there a limit to how many sub-folders deep it will do??) I noticed that it won't show the files in the UI of sub-folders if it finds the names are the same.

I do notice that on the file-added event it does know about that file but UI never shows it.

[goto-bus-stop]

Are only the names the same or are they the same file (same content, date etc) in two separate folders?

Files are assigned IDs by Uppy based on their name, modified date, and size. These IDs must be unique, if two files have the same name/date/size they only one of them will be added. This is due to internal limitations that are hard to change.

We recently discussed changing the ID generation to include the folder names when you drop a folder into Uppy, so that you can add multiple identical files if they are in different folders. This should be coming up quite soon after some of the existing PRs fixing .relativePath properties are merged.

[genachka]

@goto-bus-stop

It can be both. And it would be very confusing for our users when files are discarded when the user doesn't expect that. In our case we can have several hundred files selected when there are sub-folders used. And it would be very difficult in the dashboard plugin to know which files were dropped for a user in that case.

Are only the names the same or are they the same file (same content, date etc) in two separate folders?

That would be a helpful change. I hope it will work on both identical file names as well as identical files.

We recently discussed changing the ID generation to include the folder names when you drop a folder into Uppy, so that you can add multiple identical files if they are in different folders. This should be coming up quite soon after some of the existing PRs fixing .relativePath properties are merged.

Is there a limitation on how many levels deep of sub-folders is supported?

[goto-bus-stop]

Is there a limitation on how many levels deep of sub-folders is supported?

We recurse into subfolders, so there should be no practical limit. (unless you manage to blow the stack by going thousands of layers deep :laughing: )

[genachka]

BTW, I do see a use case for some others that do want to have unique only. So maybe it could be a flag that is toggled. And if unique, and files are excluded, it would be helpful to show in the Dashboard UI any files skipped because of not unique and also have ability to retrieve those names and their path via API.

That would be a helpful change. I hope it will work on both identical file names as well as identical files.

We recently discussed changing the ID generation to include the folder names when you drop a folder into Uppy, so that you can add multiple identical files if they are in different folders. This should be coming up quite soon after some of the existing PRs fixing .relativePath properties are merged.

[kvz]

Hi there, thanks for reporting this! We've talked about this with the team and agree we need to handle this in a better way. We've added it to our backlog for the next release as such:

• [ ] core: avoid overwriting duplicate files by a) throwing a warning instead and b) adding the relative-path of files to a new tus fingerprint function (we might use file.id as a fingerprint instead) (#754, #1606)

Some more details in issue #754:

Discussed this with the team, here's what we want to do, if you drag the same file again, we would abort overriding, and instead throw a warning with Informer, and ask the user to clear first. The exception would be if the file is a ghost file (golden retriever knows about it but could not restore it), then we'd replace the ghost.

For people dragging folder a and b onto Uppy where both have an index.html with the same dateModified and size, we're going to change the fingerprinting function to include the relative-path (which we think is available). This would allow duplicates across folders to be uploaded.

We're adding this to our roadmap, and closing the issue while we're not actively working on it yet.

We'll open issue #1606 when we start working on this (so you might want to follow that one), closing the related issues in the meantime. Thanks again for your feedback!