October: Feature request: File upload widget, spare the duplicates

[tschallacka]

Expected behavior

If a file is already uploaded, don't upload it a second time, re-use the existing file to create new database entry. The second database entry can be used to modify tile and description for seperate uses.
This saves a lot of disk space on places where a lot of the same images are used with seperate uses.

Actual behavior

It simply creates a new file

Reproduce steps

Use the FileUpload widget to upload a file to \System\Models\File

October build

419

My proposed solution includes an assumption, and the assumption could be the dealbreaker. But maybe it could be included as an configurable option in the formwidget.

The assumption
If, filename, mimetype, filesize and public status is the same as a present file, we can safely assume it's the same file. is acceptable then if the content in FormWidget:392-395 can be changed to

        $file = $fileModel;
        $existingFile = $file->newQuery()
                                ->where('file_name', $uploadedFile->getClientOriginalName())
                                ->where('file_size', $uploadedFile->getSize())
                                ->where('content_type', $uploadedFile->getMimeType())
                                ->where('is_public', $fileRelation->isPublic())
                                ->limit(1)
                                ->get()
                                ->first();
        if ($existingFile) {
            $file->disk_name = $existingFile->disk_name;
            $file->file_size = $existingFile->file_size;
            $file->file_name = $existingFile->file_name;
            $file->content_type = $existingFile->content_type;
            $file->is_public = $existingFile->is_public;
        }
        else {
            $file->data = $uploadedFile;
            $file->is_public = $fileRelation->isPublic();
        }
        $file->save();

And in https://github.com/octobercms/october/blob/master/modules/system/models/File.php one could add

   public function afterDelete() {
          if($this->newQuery()->where('disk_name', $this->disk_name)->where('is_public',$this->is_public)->where($this->getKeyName(),'!=',$this->getKey())->count() == 0) {
              parent::afterDelete();
          }
    }

here only a check is done on diskname and is_public, since that is a unique value, and no two files can occupy the same filename space in the same subdirectory(public/private).

[LukeTowers]

While this is an effective implementation of the proposed solution, I'd like to counter propose something for you:

Why not use the mediamanager instead? File uploads / attachments are meant to be for use cases where files are not going to be used in multiple places. The media library is meant for use cases where files are going to be used in multiple places.

What is currently preventing you from using the media library in its intended role?

[tschallacka]

It returns a string to the relative path to the file, which means I have to make file wrappers all the time to get any useful data or to make thumbnails. The fileupload widget is much more practical.

Add an attachone system/models/file, add a fileupload widget, and then at $model->file I can call get thumb, file_name, description, filesize, all the things I need, without having to code for every model filewrappers and methods.

Guess i'm lazy, but I like the easy of the fileupload more than the extra work of the media manager.
Also, productivity usually suffers under media manager. People usually find stuff faster in windows explorer and then drag a file, than in the media manager(we have 50.000 + images). It saves a lot of strain on our office infrastructure(when having to display too many files in browsers(load a folder with 1000 files, it takes forever, takes up network capacity, office server bandwidth, server cpu time) by just re-uploading the same file, they found via more efficient methods.

[kg-bot]

And what will happen if user A decides to delete file that he uploaded, and that file is same file that user B is using?

[LukeTowers]

    public function afterDelete() {
          if($this->newQuery()->where('disk_name', $this->disk_name)->where('is_public',$this->is_public)->where($this->getKeyName(),'!=',$this->getKey())->count() == 0) {
              parent::afterDelete();
          }
    }

@kg-bot this is the code that handles that. Normally, afterDelete() would remove the file from the filesystem after deleting the db record, this code adds an additional check to make sure that there aren't any other db records referencing the same file in the filesystem before actually performing that deletion of the file in the filesystem.

[kg-bot]

@LukeTowers never done too much with Fileupload deleting files, nice to know that, thanks.

[LukeTowers]

@tschallacka do you still want this feature?

[tschallacka]

Yes i do.

[LukeTowers]

@tschallacka could you make a pull request then?

[github-actions[bot]]

This issue will be closed and archived in 3 days, as there has been no activity in the last 30 days. If this issue is still relevant or you would like to see action on it, please respond and we will get the ball rolling.