Keystone-classic: Remove Local, S3 and Azure field types

[JedWatson]

To finalise the move to the new API (#3209) we need to remove the LocalFile, LocalFiles, S3File and AzureFile fields which aren't up to spec and have been deprecated in favour of the new generic File field (see #3052)

Right now, we've got File working with the FS adapter, but the Azure and S3 adapters are still in progress. We're also missing a Files field which would replace LocalFiles.

Despite this I really need to push on with integrating the new API, so I'm proposing a transitional window where the master branch will lose the old field types without having replacement functionality ready. Since master isn't _really_ stable at the moment, that's sort of OK, but I'm aware that some people may be testing it with apps that use these old field types and I don't want to cause issues.

My proposal is to create a 0.4-alpha-legacy-fields tag on master soon, which will basically represent the last working version before the fields are removed so that there's still a version of Keystone available for those users during the transition.

If anybody's actually in this position (using master and those field types) please shout out here - unfortunately I don't know if I'm boxing at shadows or not since we have know way of knowing what's being used in the wild. It would be great to get some help ensuring the S3 and Azure adapters (and the new File field) get completed and meet requirements.

[mxstbr]

unfortunately I don't know if I'm boxing at shadows or not since we have know way of knowing what's being used in the wild.

This is exactly why we need analytics. Remind me to link to this issue when we have this discussion again.

[terenzeyuen]

Hm we have customised s3file to support large file uploads which works really well. So the legacy support is really important.

[pr1ntr]

We are using S3 file in the wild right now. On 2 projects. One we still using 3.x for the ohter we are on 0.4.x. I think there should be a solution for this before a release.

[josephg]

@terenzeyuen: Has your customized version of s3file been merged into keystone's s3file implementation?

[josephg]

I've written up a replacement implementation for S3 files here. I'd love everyone using S3 files could take a look while the code is still in flux. We'll merge it into keystonejs/keystone-s3 soon.

[dudv]

I'm using the version from master with both LocalFile and LocalFiles fields which are the key features for my client.

[christroutner]

I'm using the LocalFile quite extensively for both my clients as in the ConnextCMS extension. All File and Image uploads use it. I'm worried these changes are going to represent a major break for me and my users. I should be OK as long as I can keep abreast of the changes and how to retrofit my code.

A gist or other example on how to switch from legacy code to the new changes would be greatly appreciated!

Also, a tag for knowing when the legacy is officially not supported would be good. I can use that version of Keystone until I'm ready to refactor my code.

[iyobo]

Much like in wordpress, local files are essential to any Web app.
Personally, I think FS (Both singular and multiple) should be first priority.
Using something like Digital Ocean or Vultr that gives you tens of gigabytes of SSD HD space (and now block storage which grants even more!), there is little point in a web app that is just starting out being forced to use S3. (An external service, you must pay for, to store files when you already have space)

Besides, if S3 is a must, there are certain file-system adapters that handle S3 bucket syncing without getting KeystoneJS involved. The reverse is not the case.

[josephg]

@iyobo we aren't taking out support for local files. In fact, that was the first FS adapter we've ported across to the new interface: https://github.com/keystonejs/keystone/blob/master/lib/storage/adapters/fs/index.js

@christroutner @dudv (and anyone else): I'd like to put some documentation online next week describing how to use the new file field. Do you mind trying it out? I'd like to find & resolve any issues in the new code before we take out the old code.

[christroutner]

@josephg, I'm anxious to try it out. Let me know when you have something to test and I'll get right on it. I've already got an initial test script going for the new Admin API in this repo. I can add test cases for file and image uploads to it.

[emsimo]

@JedWatson - when using the new FileType it doesn't look like the underlying file in the filesystem is deleted if I delete the parent item using the main grid or the delete link in the item view. Am I missing something? Thanks.

[JedWatson]

@emsimo that is a missing feature. It's not trivial to add or we would have supported it in the initial release, looking for a way to elegantly support it. We should add it to productpains.

[mepawan]

Hi,

I have tried to use Types.File. But when I tried to start keystone I'm getting this error:

/home/ubuntu/kslms/lms/node_modules/keystone/lib/storage/adapters/fs/index.js:36
        throw e;
              ^
TypeError: Object #<Object> has no method 'accessSync'
    at ensurePath (/home/ubuntu/kslms/lms/node_modules/keystone/lib/storage/adapters/fs/index.js:24:6)

Here is my code:

var MediaStorage = new keystone.Storage({
  adapter: keystone.Storage.Adapters.FS,
  fs: {
    path: keystone.expandPath('./uploads'),
    publicPath: '/public/uploads',
  },
});
SectionLession.add({
    title: { type: String, required: true },
    section: { type: Types.Relationship, initial: true, ref: 'CourseSection', index: true },
    banner: { type: Types.CloudinaryImage },
    content:{ type: Types.Html, wysiwyg: true, height: 400 },
    files: { type: Types.File, storage: MediaStorage },
    author: { type: Types.Relationship, initial: true, ref: 'User', index: true },
    state: { type: Types.Select, options: ['published', 'draft', 'archived'], default: 'published', index: true },
});

nodejs version is v0.10.46

can you please help me to fix this issue?

thanks,

[wmertens]

@mepawan minimum node version for 4.0 is v0.12…

[josephg]

Yes, nodejs 0.10 is extremely old now and no longer supported. If you want a rock solid version use the 4.X LTS.

[mepawan]

Hi,

How I can use field Types.File for multiple files upload feature?

[wmertens]

You will have to wait until we merge the new ListType, soon. We'll also
make something to replace LocalFiles.

[Kiina]

With the new ListType is it possible to provide extra info to file uploads? In 0.3 I always wanted to add a String for extra information to my File Uploads (for tags etc), but i couldn't

[Noviny]

As of current master, these fields return an error if you attempt to include them, preventing keystone from starting.

Since these are effectively deprecated (if not yet removed), I am closing this issue.

[VinayaSathyanarayana]

Where can one get the latest documentation on the "File" Field Type?
What is the status of AWS S3?
Any plans for Google Storage?

[mx781]

@VinayaSathyanarayana this was most helpful to me, and I believe is the bulk of documentation available. There's some more in the actual source folder, but it seems less complete.

[bbqsrc]

@wmertens was said ListType ever merged? I can't find it in master and the lack of being able to handle arrays of objects is becoming a major pain point.

[iyobo]

@bbqsrc I created an amorphous ListType component a while back.
It worked for the most part, but there were issues with it in some fields because of the way certain things are designed in the new admin-ui.

I have since then created my own NodeJS framework (JollofJS) with a backend Admin UI as well with the following pros:

• Expansive Field System:
  All sorts of fields, including location fields and file fields. You can make Arrays of any field type or objects of any combination of field types. You'll never really quite find any admin UI out there that's this expressive.
• Fully ES7 Async/Await Ready (So naturally, requires Node 8)
• Convention over config, but highly configurable (even per environment)
• Based on Koa 2.
• Complexity-Friendly. I have already used this to deliver complex projects for some clients.
  This is in-part to the Jollof Data Model system which does not force you to use MongoDB. The data layer is abstracted from any datasource and has it's own query language (JollofQL). It connects to datasources only by adapters. Comes with the MongoDB Adapter by default.

Cons:

• Sucky Documentation: Not ashamed to admit it. Been too busy polishing it that I haven't put quality time into documentation yet. I'll have to set aside a weekend to really give it a good doc'n :D

[jacargentina]

I want to be the first to tell.... HAPPY BIRTHDAY 🎂 to this issue!!!! 3 days remaining but.... you know, I´m being sarcastic.

Cant use the 0.4 whistles because of this "pretty and nice" issue (sarcastic again...)

[wmertens]

This is pretty much in the hands of the folks at ThinkMill, Jed said a
while back that they are working on it a lot behind the scenes.

On Wed, Jul 26, 2017, 12:15 AM Javier Castro [e-mail]
wrote:

I want to be the first to tell.... HAPPY BIRTHDAY 🎂 to this issue!!!! 3
days remaining but.... you know, I´m being sarcastic.

Cant use the 0.4 whistles because of this "pretty and nice" issue
(sarcastic again...)

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/keystonejs/keystone/issues/3228#issuecomment-317889233,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADWlhN3ug10LDWgJEO-WtLvxcFECJ77ks5sRmkZgaJpZM4JXPNq
.

[jacargentina]

A day later to the party... HAPPY 2 BIRTHDAY to the issue!!! 🎂🎂

[zalbastaki]

Any updates on allowing multiple files being uploaded?