Multer: Modify files before storing them

[jacobtomlinson]

It would be useful to have the ability to modify files as they are uploaded. A simple example would be resizing an image using gd.

My initial thoughts are the fileFilter function could return the file as the second argument rather than a boolean. That way it could be manipulated in the fileFilter function and then be passed on to be stored, or dropped if false was passed instead of the file.

https://github.com/expressjs/multer/blob/master/lib/make-middleware.js#L94

This would be more useful when using an external storage engine such as S3 as you wouldn't have to upload -> download -> edit -> re-upload.

[jacobtomlinson]

I see now after looking into it that the file object doesn't actually have the fileStream associated with it at the time that it goes into fileFilter. So perhaps it would be better to have another function which allows you to modify the stream.

[LinusU]

Hmm this is an interesting one, it could be cool to have a transformFile which gets a stream and returns a new stream. I'm not sure if this falls inside the scope of multer thought.

gd will require you to have the entire image in memory anyhow so I think that the easiest solution is to use the memory storage and the passing the buffer to resize.

[jacobtomlinson]

Yes that is the solution I ended up going with.

Sadly it meant implementing my own code to store the output files in S3.

[LinusU]

Ah, didn't even think of that. The storage api is documented but it's also considered internal. One possible solution would be to call _handleFile yourself...

[wesleytodd]

I can add this right now in a PR if you are still interested. This was my thought for the api:

var storage = multer.memoryStorage({
  transformFile: [through2(/* ... */)]
})
var upload = multer({
  storage: storage
})

Add that to both of the built in storage drivers. It think would be like a 5 line PR using pump (excluding tests).

Our use case for this is similar to the OP where we want to preform some transforms on images and then save to Google Cloud Storage. We may end up writing our own engine, but it would be nice to do simple stuff like modifying the image before writing to disk with the core engines.

[LinusU]

@wesleytodd Maybe it's better to make the transformFile option belong on the global multer instance? That way we don't have to implement it in every storage module.

Also, I think that transformFile should be a function that takes (req, file, cb) that calls the cb with a replacement stream

[wesleytodd]

Yeah, sounds like great ideas. I will take a look at this later today, and those changes should make this an even smaller PR.

[wesleytodd]

Noticed that it didn't link the PR here, so here it is: #356

[jonchurch]

Current PR that would allow this is https://github.com/expressjs/multer/pull/399