Meteor-files: Example for AWS S3 Integration

[tcastelli]

I have seen thi is in the Roadmap, and I just want to give it a :+1: since it would be a very common use case (even more than dropbox I think).
Good work on this package :)

[santiq]

Based on this guide https://github.com/VeliovGroup/Meteor-Files/wiki/Third-party-storage, I would like to integrate this package with Amazon S3. Can you guide me on how could I get this done? Should I use some NPM package, like this one https://www.npmjs.com/package/s3 ?

[dr-dimitru]

Hi @santiq ,

Can't help you right now. But you have a two options:

1. Wait until integration example will be published
2. Try it yourself, and share your experience, I may add your example to official docs.

There is actually two ways to of integration:

1. Direct upload to S3 (_Difficult way_)
2. Upload via server (_more easier, and allows to work with file on server-side, like create thumbnails_). This will be almost the same as for DropBox

[egorgrushin]

Install lepozepo:s3 first;
Setup lepozepo:s3 step by step on https://github.com/Lepozepo/S3.

And this code will help you

var bound = {};

if (Meteor.isServer) {
    bound = Meteor.bindEnvironment(function(callback) {
        return callback();
    });
}

Images = new FilesCollection({
    debug: false, // Change to `true` for debugging
    throttle: false,
    storagePath: 'assets/app/uploads/images',
    collectionName: 'Images',
    allowClientCode: false,
    onBeforeUpload: function (file) {
        return true;
    },
    onAfterUpload: function (fileRef) {
        //Don't use AWS S3 on localost
        if(Meteor.absoluteUrl() == "http://localhost:3000/"){
            return;
        }
        // In onAfterUpload callback we will move file to AWS S3
        var self;
        self = this;
        var fileName = fileRef._id + '.' + fileRef.extension;
        var relativePath = 'images/' + fileName;
        Logger.log({message: 'start putting a file'});
        S3.knox.putFile(fileRef.path, relativePath, function (err, res) {
            bound(function (){
                if(err){
                    Logger.log({message: err.message});
                } else {
                    var imageUrl = S3.knox.http(relativePath);
                    var imageSecureUrl = S3.knox.https(relativePath);
                    self.collection.update({
                        _id: fileRef._id
                    }, {
                        $set: {
                            'meta.url': imageUrl,
                            'meta.secure_url': imageSecureUrl,
                            'meta.relative_url': relativePath
                        }
                    }, function (error) {
                        bound(function (){
                            if(error){
                                Logger.log({message: error.message});
                            }else{
                                Logger.log({message: 'end image update'});
                                Logger.log({message: 'start unlink file'});
                                // Remove file from FS
                                self.unlink(fileRef);
                                Logger.log({message: 'end unlink file'});
                            }
                        });
                    });
                }
                res.resume();
            });
        });


    },
    interceptDownload: function (http, fileRef, version) {
        //Don't use AWS S3 on localost
        if(Meteor.absoluteUrl() == "http://localhost:3000/"){
            return false;
        }
        var path, ref;
        path = fileRef != null ? (ref = fileRef.meta) != null ? ref.secure_url : void 0 : void 0;
        if (path) {
            // If file is moved to  AWS S3
            // We will redirect browser to  AWS S3
            http.response.writeHead(302, {
                'Location': path
            });
            // Alternatively you can pipe request to DropBox
            // like: `request.get(path).pipe(http.response)`
            // But you need to handle all headers yourself
            // This example just more simple
            http.response.end();
            return true;
        } else {
            // While file is not uploaded to AWS S3
            // We will serve file from FS
            return false;
        }
    }
});

// Intercept File's collection remove method
// to remove file from DropBox
var _origRemove = Images.remove;
Images.remove = function (search) {
    if(Meteor.absoluteUrl() == "http://localhost:3000/"){
        // Call original method
        _origRemove.call(this, search);
        return;
    }
    var cursor;
    cursor = this.collection.find(search);
    cursor.forEach(function (fileRef) {
        var ref;
        if (fileRef != null ? (ref = fileRef.meta) != null ? ref.relative_url : void 0 : void 0) {
            Logger.log({message: 'start deleting file from aws s3: ' + fileRef.meta.relative_url});
            S3.knox.deleteFile(fileRef.meta.relative_url, function (err) {
                bound(function () {
                    if(err){
                        Logger.log({message: err.message});
                    }
                });

            });
            Logger.log({message: 'end deleting file from aws s3'});
        }
    });
    // Call original method
    _origRemove.call(this, search);
};

I hope I help you.

[dr-dimitru]

Hi @GrushinEgor ,

I've tried your suggestion, but new versions of lepozepo:s3 is client only, which version your example is refers to?

[egorgrushin]

@dr-dimitru My solution has been built on latest version of lepozero:s3. It provide S3.knox object on serverside. See my code above more attentively! :) Documentation for knox: https://github.com/Automattic/knox

[dr-dimitru]

@GrushinEgor ,

On my end S3.knox is always empty object.
Anyways, I've used knox directly. As lepozero:s3 has a lot of security issues.

Thank you for pointing me to right direction

[dr-dimitru]

Hi everyone!
I've added tutorial for AWS S3 integration to Wiki

Big thanks to @GrushinEgor for pointing into right direction
Please note: _given example is for_ v1.6.0 _which will be released soon. Meanwhile fell free to test it from_ dev _branch._

[dr-dimitru]

v1.6.0 is published, see in Wiki: https://github.com/VeliovGroup/Meteor-Files/wiki/AWS-S3-Integration