Multer: File still uploads even when it exceeds fileSize limit

[pribanacek]

For testing purposes, I'm uploading a 102kB file with limits setup like so:

...
const limits = {
    fileSize: 5,
};

const upload = multer({storage: storage, fileFilter: fileFilter, limits: limits});
...

Clearly that exceeds the 5 byte limit, but I'm finding that the file still gets saved without any errors, except that the saved file is 5 bytes in size, so it seems something just writes the first part of the file and then gives up.
I'm assuming this isn't an intended feature, but do correct me if I'm wrong.

[LinusU]

Hmm, it seems like this test case should cover that, and it passes 🤔

https://github.com/expressjs/multer/blob/master/test/issue-232.js

Would you be able to submit a failing test case? Then I can take a look at fixing it ☺️

[karandutt01]

@pribanacek
Try this one:
app.use(function (err, req, res, next) {
// ye error multer ki documentation se liya ('LIMIT_FILE_SIZE')
if (err.code === 'LIMIT_FILE_SIZE') {
res.send({ result: 'fail', error: { code: 1001, message: 'File is too big' } })
}
});

[adf0001]

Hmm, it seems like this test case should cover that, and it passes

https://github.com/expressjs/multer/blob/master/test/issue-232.js

Would you be able to submit a failing test case? Then I can take a look at fixing it

line changed:
form.append('file', util.file('huge.rar')) //huge.rar is a file of 2600M (2.6G)

mocha --timeout 120000 issue-232.js

Issue #232
√ should report limit errors (24565ms)

1 passing (25s) //<------ too long

When I test in Chrome locally, I get 0% to 100% uploading status report from Chrome, and at 100%, get a "MulterError: File too large". The error emit after all the file has uploaded.

windows 7, node v6.5, multer 1.4.1

[munawarb]

Hi,
I get the same problem. For testing I'm uploading a 1 gigabyte file with an upload limit of 2.5 mB (2.510241024). I only see any errors after the file has fully uploaded. Shouldn't the middleware stop the upload and send an error as soon as the cap in the limits object is reached instead of waiting for the full file to upload?

[LinusU]

I don't think that there is a way to not read the data from the socket. Since HTTP is a simple request/response protocol there is no way to tell the browser to stop sending the file.

If there is any technical way to implement this I'd be happy to reopen! Please let me know

[FINDarkside]

I'm not sure if I'm missing something, but surely there's a way to close the connection instead of reading the file completely? What if I send 50GB file, is multer going to write it all to disk before discarding, or did I misinterpret the problem?

[munawarb]

Hi @FINDarkside,
It reads the entire file before giving an error message. I've since switched to Formidable which errors out as soon as the file size is exceeded. The problem here is that the client isn't being notified so eventually the client just times out. But this is fine for me since I've got client side validation on file sizes and things, so anyone uploading more than the acceptable amount is bypassing the client side checks and deserves to be timed out anyway 🗡

[FINDarkside]

Thanks for the reply, I'll look into Formidable!

IIRC the client not being notified is because browsers don't follow the spec properly, so they keep uploading despite server responding the body is too large. So the only way to stop the client is to close the connection, but then the client doesn't get any error messages. Not sure where I read this from anymore and if browsers still work that way.

[munawarb]

Hi @FINDarkside,
You're welcome :). Last I tested which was a couple months ago with Firefox, it timed out when Formidable closed the connection on a large file upload. So yup it looks like browsers are still not following the spec.
Here's some Formidable middleware for Express that works well for me. It's better than using express-formidable because you get more control by implementing it yourself:

// This middleware will add body to the request object and files will be contained in the files field.
// The upload directory is also defined here.
app.use((req, res, next) => {
    const form = new formidable.IncomingForm({keepExtension: true, maxFileSize: 2.5*1024*1024, uploadDir: uploadDirectory});
    form.parse(req, (err, body, files) => {
        if (err)
            return next(err);
        Object.assign(req, {body, files});
        next();
    });
});

[FINDarkside]

Figured out I don't really need multipart forms, I simply need to accept a single file. So I did it manually by streaming the data from request. Didn't find better way to handle too big files than to call req.destroy which means the upload fails with ERR_CONNECTION_RESET instead of relevant error message.

Same strategy might work with formidable as well.

[FINDarkside]

Shouldn't this be pretty big issue? It'd be trivial to DoS server by using its all download bandwidth if it's unable to refuse files. Simply send couple 50GB files from different machines.