Functions-samples: function sample that uses firebase storage ref

[davvit]

We need a function sample that simply creates a file in firebase storage. Current samples make a file on the server using mkdirp and use temp file path for uploading that file to storage.

I would assume doing something like this would be valid and straightforward but it does not work since admin.storage().bucket().ref(..) is not valid along with functions.storage.bucket().ref(...)

var fileRef = admin.storage().bucket().ref('myFolder/myfile.txt');
// Uint8Array
var bytes = new Uint8Array([0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x2c, 0x20, 0x77, 0x6f, 0x72, 0x6c, 0x64, 0x21]);
fileRef .put(bytes).then(function(snapshot) {
  console.log('Uploaded an array!');
});

Here is a scenario and starting function :

export const newAccountCreated = functions.auth.user().onCreate(event => {
    const user = event.data;

    return admin.database().ref('users/' + user.uid).set({
        uid: user.uid,
        email: user.email
    }).then(() => {
       //Create a folder with user id and write data to info.txt file
        //this does not work
        var fileRef = admin.storage().bucket().ref(user.uid + '/info.txt');
        var bytes = new Uint8Array([0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x2c, 0x20, 0x77, 0x6f, 0x72, 0x6c, 0x64, 0x21]);
        fileRef.put(bytes).then(() => {
           console.log('file created');
        })
    });
});

[davvit]

I should add that i am using typescript and i import google cloud as follows and tried to use it...but it did not work because i cant get instance of a bucket...i only have access to the Bucket Object for initialization.

import * as gcs from '@google-cloud/storage';
import { Bucket, Storage, File } from '@google-cloud/storage';

private bucketName = functions.config().firebase.storageBucket;
public storageBucket = gcs.bucket(this.bucketName);     <-not able to do this

[inlined]

If you use the @google-cloud/storage library directly, you'll need to initialize it with:

import * as storageAPI from `@google-cloud/storage`
const storage = storageAPI(); // Auto initialize in Google environments

or you can use the firebase-admin API to get an initialized copy of the API with:

const bucket = admin.storage().bucket();

The Admin SDK uses the @google-cloud/storage directly, so you'll use "files" instead. E.g.

const stream = bucket.file('myFolder/myfile.txt').createWriteStream();
stream.end(bytes)
// The stream finish event will let you know the upload has completed

[berkcoker]

How can we access file metadata using the admin API? Shouldn't this exists ideally? @inlined

[inlined]

getMetadata and setMetadata

[00hello]

I really feel that these types of things should be added to the documentation. It was only after going through the functions documentation and the storage documentation and looking at the different things that can be done with each did I ascertain that the storage function's event.data could not have the same properties as the storage's storage.ref().

It seems to me that this should be clearer in the function documentation. Because it's easy to assume they should have the same properties. And it's not easy to find out how to actually do those things with event.data that a storage.ref() allows you to do.

[nickjuntilla]

Is there a full example of this anywhere?

[Antoinebr]

+1 for @nickjuntilla request

[nickjuntilla]

@Antoinebr So it turns out this was a major journey and is not 'officially' documented anywhere. For starters firebase functions have a bug (maybe an intentional bug) that does not allow you to use the standard multer library to parse out files:

https://stackoverflow.com/questions/47242340/how-to-perform-an-http-file-upload-using-express-on-cloud-functions-for-firebase

As a result you need to use busboy.

Then you need to get a special firebase link as defined here:
https://stackoverflow.com/questions/42956250/get-download-url-from-file-uploaded-with-cloud-functions-for-firebase

Sorry I can't post an entire example but those are the basic steps. It's a shame none of this is mentioned because it's the only way to have full control over upload permissions when you don't want to nest photos under user IDs. It seems like it would be a very common circumstance.

[davvit]

Let me update everyone with soln that I have been using for another purpose. Its not tested as I had to peace it together here now. The idea is to create a file in temp path on the server -> write our data to it -> AND THEN upload it to the final path. I have not seen a way to directly write to a storage ref path.

import * as functions from 'firebase-functions';
import * as admin from 'firebase-admin';

import { tmpdir } from 'os';
import { join, dirname } from 'path';

import * as fs from 'fs-extra';

export const newAccountCreated = functions.auth.user().onCreate(event => {
    const user = event.data;

    return admin.database().ref('users/' + user.uid).set({
        uid: user.uid,
        email: user.email
    }).then(() => {        
        // Get a reference to the storage service, which is used to create references in your storage bucket 
        // You can leave bucket name empty or use the other option/buket names
        const bucketRef = admin.storage().bucket([YOUR - BUCKET - NAME]);
        // const bucketRef= admin.storage().bucket().ref();

        const workingdir = join(tmpdir(), 'tempfiles');
        const tmpFilePath = join(workingdir, 'info.txt');

        //make sure OS gave us a tempdir
        await fs.ensureDir(workingDir);

        var someData = 'this is my data for info.txt file';

        //write the date to temp file path.
        await fs.outputFile(tmpFilePath, someData, function (err) {
            console.log(err);
        });

        var finalPath = join('userfiles', uid, 'info.txt'); 
        var finalDestination = bucketRef.child(finalPath);

        await bucketRef.upload(tmpFilePath, { destination: finalDestination });
    });
});

```

[nicolasgarnier]

FYI this sample shows how you can use streams to modify a file and re-upload it directly to Cloud storage. The sample is using sharp to create a thumbnail:

https://github.com/firebase/functions-samples/blob/Node-8/image-sharp/functions/index.js

Note that the code is using an old version of the cloud storage SDK (0.4.0) if you use something newer there are some changes to do (especially initialising the library).

[nickjuntilla]

@nicolasgarnier I was also able use bucket.file().createWriteStream to write directly to the bucket from busboy file.pipe, but the problem I ran into is that I couldn't get a firebase bucket link after passing the meta data the same way I did using .upload so--

metadata: {
          contentType: mimetype,
         metadata: {
            // Enable long-lived HTTP caching headers
            // Use only if the contents of the file will never change
            // (If the contents will change, use cacheControl: 'no-cache')
            cacheControl: 'public, max-age=31536000',
            functions-for-firebase
            metadata: {
              firebaseStorageDownloadTokens: uuid
            }
          },
        }

--yielded no firebase download token link from createWriteStream finish event.

So I was forced to use bucket.ref().upload() as in @davvit 's example. It would seem more ideal to not have to save to a temp directory on the server first, but then how does one get the firebase storage download token.