Filepond: Can it upload to AWS S3 Bucket?

[inglesuniversal]

I would like to know if you have some sample tests for AWS uploads

Thanks

Hugo

[rikschennink]

Hi!

Yes, you can integrate with S3 by providing FilePond with a custom process method.

Something like this works:

<script src="https://sdk.amazonaws.com/js/aws-sdk-2.250.1.min.js"></script>
<script>

// set up s3 connection
var s3 = new AWS.S3({
    accessKeyId: 'access-key',
    secretAccessKey: 'secret-access-key',
    region: 'eu-central-1'
});

// set custom FilePond file processing method
FilePond.setOptions({
    server: {
        process: function(fieldName, file, metadata, load, error, progress, abort) {

            s3.upload({
                Bucket: 'your-bucket-name',
                Key: Date.now() + '_' + file.name,
                Body: file,
                ContentType: file.type,
                ACL: 'public-read'
            }, function(err, data) {

                if (err) {
                    error('Something went wrong');
                    return;
                }

                // pass file unique id back to filepond
                load(data.Key);

            });

        }
    }
});

</script>

[simioluwatomi]

Please could you provide a sample implementation of signing the uploads on the server before file(s) are actually uploaded to s3?

I simply don't trust my client enough to provide s3 configuration variables to it.

[rikschennink]

@simioluwatomi I wouldn't also.

I don't know how to sign files on the server, I'm not very experienced with S3 or AWS, to be honest ( the above example is my first stab at working with it ).

You could supply your API endpoint to the server property.
https://pqina.nl/filepond/docs/patterns/api/server/

Then upload to your server and pass the file to S3 from the server and return the S3 key back to the client. That way you don't have to allow everyone upload access on your S3 bucket.

[simioluwatomi]

I'll walk you through the process of signing file uploads with dropzone which I'm currently using in a Laravel project. The code might be long but I think you can read it and understand what it does. I don't have enough time on my hands right now but I'll look into the filepond documentation and the events that fire when files are added to the filepond instance

var dropzone = new Dropzone("#myDropzone", {
        // I'm leaving out dropzone config options for brevity
        init: function () {
            // this event fires when a file is added to the dropzone instance.
            this.on("addedfile", function (file) {
                fetch("/audio/signed", {
                    headers: {
                        "Content-Type": "application/json",
                        "Accept": "application/json",
                        "X-Requested-With": "XMLHttpRequest",
                        "X-CSRF-Token": $('input[name="_token"]').val()
                    },
                    method: "post",
                    credentials: "same-origin",
                    body: JSON.stringify({
                        key: file.name
                    })
                })
                    .then(function (response) {
                        return response.json();
                    })
                    .then(function (json) {
                        dropzone.options.url = json.attributes.action;
                        file.additionalData = json.additionalData;
                        dropzone.processFile(file);
                    });
            });
            this.on("sending", function (file, xhr, formData) {
                xhr.timeout = 99999999;
                // Add the additional form data from the AWS SDK to the HTTP request.
                for (var field in file.additionalData) {
                    formData.append(field, file.additionalData[field]);
                }
            });            
        }
    });

The Audio controller that handles signing the file uploads

<?php

namespace App\Http\Controllers\Web;

use Aws\S3\S3Client;
use Aws\S3\PostObjectV4;
use Illuminate\Http\Request;
use App\Http\Controllers\Controller;
use Illuminate\Support\Facades\Storage;

class AudioController extends Controller
{
    protected $client;

    public function __construct(S3Client $client)
    {
        $this->client = $client;
    }

    /**
     * Generate a presigned upload request.
     * This is the method that responds to the audio/signed route
     *
     * @return \Illuminate\Http\JsonResponse
     */

    public function signed(Request $request)
    {
        $user = auth()->user();

        $options = [
            ['bucket' => config('services.s3.bucket')],            
            ['starts-with', '$key', env('AUDIO_PATH')]
        ];
        $file = strtolower(preg_replace('/[^A-Za-z0-9-.\/]/', '', $request->key));
        $filename = bin2hex(openssl_random_pseudo_bytes(12)) . $file;
        $postObject = new PostObjectV4(
            $this->client,
            config('services.s3.bucket'),
            ['key' => env('AUDIO_PATH') . $filename,],
            $options,
            '+1 minute'
        );
        $this->formInputs = $postObject->getFormInputs();
        return response()->json([
            'attributes' => $postObject->getFormAttributes(),
            'additionalData' => $postObject->getFormInputs()
        ]);
    }
}

You can visit here and here for more explanation. So in summary, one can listen to the relevant events in filepond and use them to sign uploads on the server before uploads are performed to s3. I hope I helped.

P.S: If you get this working, I'll swap out dropzone for filepond in my project; it's interface is so much better than dropzone's.

I've been reading the documentation and would like to be clear on the difference between the following events onaddfilestart vs onaddfile and onprocessfilestart vs onprocessfileprogress. To be specific, what do you mean by "file load" and "Processing a file"?

[rikschennink]

@simioluwatomi When a file is dropped it needs to be loaded to FilePond ( as you can also drop remote or local URLs ).

The start events tell you when loading or processing is started, the onaddfile one tells you when it's done. progress gives information on the total amount of bytes processed.

Therefor there's a file load (file has been loaded to FilePond) and file process event ( File has been sent to the server ).

About the signing. I don't see why you can't do the signing in a custom process method before uploading?

FilePond.setOptions({
    server: {
        process: function(fieldName, file, metadata, load, error, progress, abort) {

            // set busy state

            // do signing

            // wait for signing to complete

            // create FormData object with file and fieldName

            // append signing information to formData

            // start actual upload

        }
    }
});

[simioluwatomi]

@rikschennink I worked on this a little before sleeping and this is what I have done so far

FilePond.registerPlugin(
        FilePondPluginFileValidateSize,
        FilePondPluginFileValidateType
    );

const pond = new FilePond.create(document.querySelector("#filepond"), {
        maxFiles: 1,
        allowPaste: false,
        maxFileSize: "3MB",
        acceptedFileTypes: ['image/png', 'image/jpeg'],
        server: {
            timeout: 99999999
        }
    });
    pond.on("addfile", function(error, file) {
        if (error) {
            console.log("File error");
            return;
        }
        fetch("/audio/signed", {
            headers: {
                "Content-Type": "application/json",
                "Accept": "application/json",
                "X-Requested-With": "XMLHttpRequest",
                "X-CSRF-Token": $('input[name="_token"]').val()
            },
            method: "post",
            credentials: "same-origin",
            body: JSON.stringify({
                key: file.fileExtension
            })
        })
            .then(function (response) {
                return response.json();
            })
            .then(function (json) {
                pond.server = json.attributes.action;
                file.additionalData = json.additionalData;
                pond.processFile(file);
            });
    });

I was thinking of hooking into the onprocessfilestart event to create FormData object and append signing information to it.

But in your comment above, how would I start the actual upload? I know we would have a working example in the next few hours.

[rikschennink]

@simioluwatomi You'd have to set up an XMLHttpRequest, there's a very basic example at the top of this page:
https://developer.mozilla.org/en-US/docs/Web/API/FormData/Using_FormData_Objects

[rikschennink]

An example implementation of S3 can be found here:
https://github.com/pqina/filepond/issues/58