Meteor-files: Looking for better documentation for how to convert from CollectionFS

[schlaegerz]

I currently have a production app that is running CollectionFS, but it is now causing more and more headache and I am looking to switch as soon as possible.

I already have all my files in S3 so I shouldn't need to transfer any files over, but I am trying to find if there is any good way to just migrate all the Mongo Collection records over to this package to make sure it continues to work.

Is there any good guide in how to just add existing S3 files to a Meteor-Files collection, without downloading and re-uploading them, I'm trying to make sure we do this with as little risk as possible, as I do have a few thousand files that will need to be migrated.

Also, I can't quite find anything about this in the documentation, but is there support for multiple copies/stores of one file? In my current app we store multiple copies of each image so that we can quickly load a thumbnail instead of loading the entire image.

[dr-dimitru]

@schlaegerz what you've described is requires a lot of work to fit your specific case. But we have some links to help you to start with:

• CFS to MeteorFiles migration guide
• How to use file's subversions

If you want to hire someone to help you with this case, we have team members who are available on an hourly basis and ready to help you.

[schlaegerz]

Thanks for the links I am actually a decent way through figuring out how to convert all my data. The one thing that would make things a lot easier is if I can fully choose what url the file is loaded from.

Right now cfs uses the url '/cfs/files/images/{id}/{filename}.png?store={store}

And if I try to use the same collection which is call 'cfs.images.filerecord' I can end up with
'/cfs/files/cfs.images.filerecord/{id}/{store}/{filename}.png

So my question is there any way to change the collectionName part of the path separate from the actual name of the collection so I can get it from '/cfs/files/images/...' ? Or are they always interlinked

Is there a way to accept a wider variety of the id/filename for the last part, ideally can I make it work without specifying the store in the url at all? And just have the original file returned from '/cfs/files/images/{id}/{filename}.png'

I can probably get this handled with redirects from my server since it is just a certain case that I may have some old urls trying to load the images, but I just want to check if there is a supported way within the package to do it.

[dr-dimitru]

Redirect will work for sure. You can build any url you like, even skipping collectionName part, just do not forget to put it into request to FilesCollection.

And /cfs/files part of URL can be tuned via downloadRoute.

wdyt?

[schlaegerz]

For anyone else looking into this I was able to figure out all the issues I was having. I was able to get the majority of it by looking at the demo

And then the other key is to just use the old cfs collections

Images = new FilesCollection({
   collection: new Mongo.Collection('cfs.Images.filerecord',
    downloadRoute: '/cfs/files',
   ....

And then you can convert all the entries like this:

tranformItem = function(collection, item, isImage, copyToFolder, defaultType)
{
  try{
    if(!item)
    {
      console.warn("Item not passed in")
      return;
    }

    if(!item.original)
    {
      console.warn("Updating already updated item")
      return //Already updated
    }

    let extension = item.original && item.original.name && item.original.name.replace(/^.*\./, '') // ".stl";

    item.extension = extension
    let versions = {}

    let fileKey = item.original && item.original.key
    if(!fileKey)
    {
      if(item.copies)
      {
        let copyKeyList = Object.keys(item.copies)
        if(copyKeyList)
        {
          let copyKey = copyKeyList[0]
          let copy = copyKey && item.copies[copyKey]
          fileKey = copy && copy.key
          if(!fileKey)
          {
            return
          }
        }
      }
    }

    let set =  {
      isImage: isImage,

      isAudio:false,
      isJSON:false,
      isPDF:false,
      isText:false,
      isVideo:false,
      ext: extension,
      extension: extension,
      extensionWithDot: "." + extension,
      'mime-type': item.original.type || defaultType,
      _storagePath: filesLocation,
      _downloadRoute: collection.downloadRoute,
      _collectionName: collection.collectionName,
      meta: item.meta || item.metadata,
      userId: item.userId || item.metadata && item.metadata.owner,
    }

    set["size"] = item.original.size,
    set["name"] = item.original.name,
    set["mime"] = item.original.type  || defaultType,
    set["type"] = item.original.type  || defaultType,
    set["path"] = filesLocation + "/" + fileKey

    const setPipe =  USES3
    if(item.copies)
    {
      Object.keys(item.copies).forEach((key)=>{
        let copy = item.copies[key]
        let itemKey = copy.key || fileKey
        versions[key] = {
          path: filesLocation + "/" + itemKey,
          size: copy.size,
          name: copy.name,
          extension: extension,
          mime: copy.type,
          type: copy.type,
        }

        if(setPipe)
        {
          versions[key]['meta'] = {
            pipePath:  copyToFolder[key] +"/" + item.copies[key].key
          }
        }
      })
      set.versions = versions
    }
    collection.update({_id:item._id}, {$set:set})
  }
  catch(error)
  {
    console.error("Failed to update item", error, item)
  }
}

And then for redirecting if you have cases like me we you have old clients that cannot be updated easily. This probably doesn't get all the same features as the other files, but it works for me.

WebApp.connectHandlers.use('/cfs/files/images/', function(request, response, next) {
    let id = request.url.split('/')[1]
    if(id)
    {
      const params = {
        query: {},
        file: id + '.png', // You will have to get the right extension, my case is always the same
        _id: id,
        version: "thumbnails", // You will have to get the right version, my case is always the same
        name: id + '.png',
      };
      const http = {request: request, response: response, params}
      if(Images._checkAccess(http))
      {
        Images.download(http, "thumbnails", Imagess.findOne({_id:id}))
      }
      else
      {
        next()
      }
    }
    else
    {
      next()
    }
  })