Aws-sdk-js: s3.getObject(params).createReadStream().pipe(file) throws no error, but transfers no data to file

[ghost]

Hi - thanks for looking into this. This is a Meteor app, using Node 0.12.7

The following seems to work exactly as it should and returns file data:

let params = {Bucket: Meteor.settings.awsBucket, Key: documentInfo.fileKey};
s3.getObject(params, function(err, data) {
            if (err){
                console.log(err, err.stack);
            } else{
                console.log(data);
            }
        });

However, the following code throws no errors, but also does not transfer any data into the file on disk:

let params = {Bucket: Meteor.settings.awsBucket, Key: documentInfo.fileKey};
let file = Npm.require('fs').createWriteStream('image.png');
s3.getObject(params).createReadStream().pipe(file);

I did notice that when I console.log(s3.getObject(params)), it says that the in the response, data is null:

response: 
I20151002-09:55:25.417(-4)?    { request: [Circular],
I20151002-09:55:25.417(-4)?      data: null,
I20151002-09:55:25.417(-4)?      error: null,

I can't seem to locate where the problem is - is it that s3.getObject is not returning any data when used in conjunction with streams, or is the error with the streams themselves? By including the following, I never get an error message, but I do always get a 'done' message:

file.on('close', function(){
            console.log('done');
        })

file.on('error', function(err) {
             console.log(err);
         });

[chrisradek]

@Rich17
What version of the SDK are you using?
I tried the following code using SDK v2.2.8 and Nodejs 0.12.7, and was able to successfully write a ~250MB file from S3 to a file on my local computer.

var AWS = require('aws-sdk');
var fs = require('fs');

var params = {
  Bucket: 'VALID_BUCKET',
  Key: 'BIG_FILE.mp4'
};
var s3 = new AWS.S3();
var file = fs.createWriteStream('test.mp4');
file.on('close', function(){
    console.log('done');  //prints, file created
});
s3.getObject(params).createReadStream().on('error', function(err){
    console.log(err);
}).pipe(file);

Let me know if the above example doesn't work for you.

[NMEsolutionz]

‎Thank you so much chrisradek  Natalie Ellis From: chrisradekSent: Tuesday, 20 October 2015 00:38To: aws/aws-sdk-jsReply To: aws/aws-sdk-jsSubject: Re: [aws-sdk-js] s3.getObject(params).createReadStream().pipe(file) throws no error, but transfers no data to file (#744)@Rich17
What version of the SDK are you using?
I tried the following code using SDK v2.2.8 and Nodejs 0.12.7, and was able to successfully write a ~250MB file from S3 to a file on my local computer.

var AWS = require('aws-sdk');
var fs = require('fs');

var params = {
Bucket: 'VALID_BUCKET',
Key: 'BIG_FILE.mp4'
};
var s3 = new AWS.S3();
var file = fs.createWriteStream('test.mp4');
file.on('close', function(){
console.log('done'); //prints, file created
});
s3.getObject(params).createReadStream().on('error', function(err){
console.log(err);
}).pipe(file);

Let me know if the above example doesn't work for you.

—Reply to this email directly or view it on GitHub.

[ghost]

Thanks a lot, @chrisradek .

My error was my file path in fs.createWriteStream. I was writing the path as it exists in the file structure I'm working with during development (pre-build) as opposed to the path once the full Meteor package is built.

[soumya-github]

Hi @chrisradek ,
I tried to use your solution. After the "s3.getObject(params).createReadStream().on('error', function(err){
console.log(err);
}).pipe(file);" code snippet what I have to write to get the file in my local storage. I am not able to get it in my local path.

[dembeleu]

@chrisradek Thank you verry much, works perfectly

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.