Sails: EMAXBUFFER:

[MuslimMunir]

My requirement is to validate file type before uploading a file
when i run this
req.file('avatar') to check if file exists
this return this error and server crash is there any way to verify file presence in request ?

Error: EMAXBUFFER: An Upstream (NOOP_avatar) timed out before it was plugged into a receiver. It was still unused after waiting 4500ms. You can configure this timeout by changing the maxTimeToBuffer option.
at null. (../node_modules/skipper/standalone/Upstream/Upstream.js:86:15)
at Timer.listOnTimeout (timers.js:92:15)
[ERROR] 14:59:32 Error

[boshido]

+1
I got the same problem

Error: EMAXBUFFER: An Upstream (`NOOP_file`) timed out before it was plugged into a receiver. It was still unused after waiting 4500ms. You can configure this timeout by changing the `maxTimeToBuffer` option.
    at null.<anonymous> (/var/source/node_modules/skipper/standalone/Upstream/Upstream.js:86:15)
    at null.<anonymous> (/var/source/node_modules/async-listener/glue.js:188:31)
    at Timer.listOnTimeout [as ontimeout] (timers.js:121:15)

[mikermcneil]

@MuslimMunir @boshido Hey guys, would you please set up a repo which reproduces the issue? Here's an example repo you can fork to save some time.

Thanks!

[MuslimMunir]

@mikermcneil It too much easy to reproduce just call the upload file function without sending file in request.
the issue is when this line executes req.file('file')
and the parameter bfile has no file in it this error is thrown

[mikermcneil]

It too much easy to reproduce just call the upload file function without sending file in request.

@MuslimMunir I understand that it's easy to reproduce for you locally, but I can't reproduce it over here :/ If you don't have time to put together a repo that reproduces this, can you help me out by providing your Node, NPM, Skipper version and framework (Sails or Express) version? If you aren't sure what version of skipper you're using, please just run node-v && npm -v && npm ls and post the output here. Thanks!

[emudojo]

@mikermcneil Actually I was hitting the same error, and is somewhat easy to reproduce.

if the bodyparser finds a file, EMAXBUFFER starts ticking, if for instance I decide that I NO LONGER want to do req.file('name').upload({..}) because .. reasons

then there's NO way to stop the timeouts.untilMaxBufferTimer in the Upstream class, I was looking for a way to do req.file('name').discart|delete|ignore() but no, if you do upload or if the payload is empty then this won't happen

this triggers the error for example

    uploadImage: function (req, res) {
        //if this is empty or missing, then the error never happens
        var type = req.param('type');
        var currentUser = req.param('id');
        var file = req.file('user_image');
        sails.log(type);
        //If the type of file is not yet defined, return error message
        if (type === 'undefined' || _.indexOf(['avatar', 'background'], type) < 0) {
            return res.badRequest({'error': 'Before you can upload a file it\'s type has to be either \'avatar\' or \'background\''})
        }
        //Now we need to find whether if the requesting user has permission to change the object
        if (currentUser === req.session.passport.user) {
            try {
                if (file !== 'undefined') {
                    //Does the actual file.upload, when the code gets here skipper is happy
                    return this._upload(file, currentUser, {'type': type}, req, res);
                } else {
                    return res.badRequest({'error': 'File is invalid'})
                }
            } catch (error) {
                console.log(error);
            }
        } else {
            return res.badRequest({'error': 'Operation not permitted by the current user'});
        }
    }

if the current user validation fails, then I don't want to upload the image, as it might be an attack or attempt to tamper data, but because file.upload never happens I get the error right after I do the Operation is not permitted, and then the server hangs and the page never gets the response back

Update

After reading the whole thing (module), which is something most people neglect to do, I found a non documented option that solved my problem and hence this error, I'll leave it here as a reference for future time travelers

in order to cancel a file upload with an actual file (not empty, not missing), you need to do something like

var file = req.file('user_image');
if (!this._checkBeforeUpload(file)) {
  //Let's pretend this is an actual function somewhere that validates the file, but it fails
  //Setting noop to true, cancels the stream operation as seen here     
  //skipper/standalone/Upstream/prototype._read.js:25:3
  file.upload({noop: true});
  return res.badRequest({'error': 'Image not uploaded we don\'t like you, go away!'});
}

Hopefully it will help others :D

[sailsbot]

@MuslimMunir,@boshido,@mikermcneil,@emudojo: Hello, I'm a repo bot-- nice to meet you!

It has been 30 days since there have been any updates or new comments on this page. If this issue has been resolved, feel free to disregard the rest of this message and simply close the issue if possible. On the other hand, if you are still waiting on a patch, please post a comment to keep the thread alive (with any new information you can provide).

If no further activity occurs on this thread within the next 3 days, the issue will automatically be closed.

Thanks so much for your help!

[jamilnyc]

This is the exact issue I'm having. I have a form with a file input and if the file input is not given a file, it throws this error and crashes the app. In my situation, there are valid use cases where the user may not upload a file, but still submit the form.

I've tried a bunch of different approaches, without any luck. @MuslimMunir @boshido @emudojo have you found a solution?

[emudojo]

if you do NOT Need to process the file, then omit its reference in the controller, starting skipper with an empty file will trigger the error, another option is to just cancel the stream operation as I showed

[jamilnyc]

@emudojo I'll give canceling the stream a shot and follow up. For the first option, how can I omit it's reference if I need to refer to it to check if anything was attached to the form submission?

[jamilnyc]

@emudojo Cancelling the stream with a No Op, requires the user actually submits a file, correct? It can't be empty or missing, as you say. So it doesn't work when the user simply doesn't submit any file at all. It can be reproduced with the following snippet of code.

Here's a simple form (in some view EJS file):

<form method="post" action="/file/upload" enctype="multipart/form-data">
    <input type="file" name="media" />
    <input type="submit" value="Submit" />
</form>

And the controller to handle it.

module.exports = {
  upload: function (req, res) {
    // Check if any files were uploaded
    if (!req.file('media')._files[0]) {
        sails.log.warn('No file uploaded');
        req.file('media').upload({noop: true});
        return res.send('no file given!');
    }

    req.file('media').upload({
      dirname: '/tmp/uploads'
    },function whenDone(err, uploadedFiles) {
      if (err) {
        sails.log.error('Error uploading file', err);
      }
      res.send('thanks for your file');
    });
  }
};

[iyogeshjoshi]

as @emudojo said usually people neglect reading module, I noticed that the author has done a really good job of assigning all the timeouts with Upstream object, following is the snippet of the same:

...
this.timeouts.untilMaxBufferTimer = setTimeout(function() {
    debug('maxTimeToBuffer timer fired- upstream is %s As of now there are %d file uploads', self._connected?'connected to a receiver (so we\'re good).':'NOT CONNECTED TO A RECEIVER!!',self._files.length);
    if (!self._connected) {
      var e = new Error();
      e.code = 'EMAXBUFFER';
      e.message =
        e.code + ': ' +
        'An Upstream (`' + self.fieldName + '`) timed out before it was plugged into a receiver. ' +
        'It was still unused after waiting ' + opts.maxTimeToBuffer + 'ms. ' +
        'You can configure this timeout by changing the `maxTimeToBuffer` option.';
      self.fatalIncomingError(e);
    }
  }, opts.maxTimeToBuffer);
...

So it seems if you clear those timers it won't throw any error, but I'm not sure if this is the right way to do this. So in case where you're NOT sending any file and it throws error, following code will take care of it.

module.exports = {
  upload: function (req, res) {
    // Check if any files were uploaded
    const mediaFile = req.file('media')
    if (!mediaFile._files[0]) {
        sails.log.warn('No file uploaded')
        // Either loop through all the mediaFile.timeouts OR 
        // clear selected ones only
        clearTimeout(mediaFile.timeouts.untilMaxBufferTimer)
        clearTimeout(mediaFile.timeouts.untilFirstFileTimer)
        // Do the following if the file is uploaded but you want to cancel the upload
        // req.file('media').upload({noop: true})
        return res.send('no file given!')
    }

    req.file('media').upload({
      dirname: '/tmp/uploads'
    },function whenDone(err, uploadedFiles) {
      if (err) {
        sails.log.error('Error uploading file', err)
      }
      res.send('thanks for your file')
    })
  }
}

This worked for me perfectly but I think the timeouts has to be taken care of themselves when there is no upload at all, I think that would have been an ideal behavior instead of clearing all timeouts manually.

[emudojo]

having to deal with those timeouts by hand is not ideal, but as of now there's no other way

[ELIDJEAKA]

it does not work for me, how to solve this problem that has embroiled me for several days

let skipper = require("skipper")
 module.exports = {
    uploadFile: function(req,res){
      req.file('image').upload(function(err,file){
          if (err) console.log(err)
          res.json({"status":"file upload successfully","file" :file})
      })
    }
 };

this's my error log :+1:
silly: Subscribed to the Product with id=7 (room :: sails_model_product_7:update)
silly: Subscribed to the Product with id=7 (room :: sails_model_product_7:destroy)
silly: Subscribed to the Product with id=7 (room :: sails_model_product_7:message)
silly: res.created() :: Sending 201 ("CREATED") response
silly: res.ok() :: Sending 200 ("OK") response
verbose: Error: EMAXBUFFER: An Upstream (image) timed out before it was plugged into a receiver. It was still unused after waiting 10000ms. You can configure this timeout by changing the maxTimeToBuffer option.