Node-restify: Streaming multipart parser without tmpfiles

Created on 31 Oct 2013 · 9Comments · Source: restify/node-restify

The multipart parser
(https://github.com/mcavage/node-restify/blob/master/lib/plugins/multipart_parser.js) uses formidable currently. Formidable creates tempfiles for each file upload, and never clears those. Even with a tmpreaper in place, this can be exploited for a DOS by sending requests with a lot of file uploads in a short time.

https://github.com/mscdex/busboy looks like a promising alternative to formidable. It supports streaming and dropping file uploads. The interface for streaming would need to be worked out- maybe by passing in something like req.form?

Source

gwicke

👍3

Most helpful comment

Some of us do contribute. However, I don't think a simple PR can provide this feature unless there is some refactoring effort along with unit tests. I see the following issues with the current design.

bodyParser.js loads all the parsers statically from the same directory. Instead there should be way to register the parser that way people can override based on the mime or content type or url or some other value
multipartBodyParser.js should be like a base class with generic api methods. The default implementation class could use formidable and then the community can create their own parser with their favourite library
If possible please adopt ES6 and babelify the code to support older node. Cant bear to see function within functions within functions with return (false) and return next() as well as return (object)

Given the popularity of this framework there should be some kind of agreement regards to the changes even before someone could invest their time for a PR.

prabhu on 18 Aug 2016

👍2

All 9 comments

Any word on this? I'm looking to use multipart streaming to allow uploading a file to AWS S3 through the API.

synth3tk on 23 Oct 2014

+1 for file streaming support :)

epegzz on 1 Feb 2015

Is this ever likely to happen?

cxreg on 12 Mar 2016

FWIW, we have been using https://github.com/mscdex/busboy in other non-restify projects at Wikimedia since, and is has worked well for us (apart from some bugs early on).

gwicke on 13 Mar 2016

👍1

👍

bluesockets on 30 Jun 2016

I can't bear formidable as well. Busboy seems stable at least for me.

prabhu on 18 Aug 2016

If anyone is interested, we would welcome a PR to the plugins repository to either change the implementation or swap out the module that it uses.

micahr on 18 Aug 2016

bodyParser.js loads all the parsers statically from the same directory. Instead there should be way to register the parser that way people can override based on the mime or content type or url or some other value
multipartBodyParser.js should be like a base class with generic api methods. The default implementation class could use formidable and then the community can create their own parser with their favourite library
If possible please adopt ES6 and babelify the code to support older node. Cant bear to see function within functions within functions with return (false) and return next() as well as return (object)

Given the popularity of this framework there should be some kind of agreement regards to the changes even before someone could invest their time for a PR.

prabhu on 18 Aug 2016

👍2

@prabhu All of those suggestions sound great - feel free to kick off a discussion in the plugins repo, as plugin development has stopped in the main repo here.