Keystone: Virtual Fields (aka; Calculated Fields)

[jesstelford]

Virtual field allow adding a field to the GraphQL end point, as well as the Admin UI, which are not backed by a direct database field, but instead are calculated at request time based on either the result of the other requested fields, related fields, or external data.

Examples of virtual fields:

• _label; a human readable short summary of an item
• approvedPosts; A shorthand for:
  graphql query { User { approvedPosts: _postsMeta(where: { status: 'Approved' }) { count } } }

Could be implemented as a new Field Type:

import { Virtual, Integer } from '@keystone-alpha/fields';

keystone.createList('User', {
  fields: {
    approvedPosts: {
      // Use Integer's views, outputType etc.
      // But override all the functionality with the `Virtual` type.
      type: { ...Integer, Virtual },
      // Specify any data required to calculate this field.
      // Will be mixed into the graphQL query for getting this item's data.
      queryFragment: `
        _postsMeta(where: { status: 'Approved' }) {
          count
        }
      `,
      resolver: item => item._postsMeta.count,
    }
  }
});

Or, could be more explicit in setting views/outputType so it becomes the Virtual field's responsibility to take those options and mix them into the type correctly.

import { Virtual, Integer } from '@keystone-alpha/fields';

keystone.createList('User', {
  fields: {
    approvedPosts: {
      type: Virtual,
      views: Integer.views,
      outputType: Integer.outputType,
      // Specify any data required to calculate this field.
      // Will be mixed into the graphQL query for getting this item's data.
      queryFragment: `
        _postsMeta(where: { status: 'Approved' }) {
          count
        }
      `,
      resolver: item => item._postsMeta.count,
    }
  }
});

To describe a nested object type, may need to also allow overwriting getGqlAuxTypes().

Notes

• To ensure the correct typing information for the GraphQL API, and also for the display aspect, we need to allow setting the GraphQL return type.
• The field would be displayed in the AdminUI's list & details views
  
  
  
  • This display info is specified like any other field type; via _views_ on the type (but set by options).
  
  
• The field would _not_ be displayed in the AdminUI's edit views
• The field would show up in the GraphQL API's queries
  
  
  
  • The output type would be specified as an option.
  
  
• The field would _not_ be available as input to the GraphQL API's mutations

Todo

• [ ] Implement #220: _Optimize database queries to only pull out fields actually requested_
• [ ] Implement query dependencies (#1116 _Enable _label field to specify its GraphQL dependencies_)
• [x] Fix #487: _encodeSortBy not working with non-virtual fields_

[molomby]

Another approach for this would be to use read-only lists backed by DB views. This doesn't help at all for the _label use case but might be a much better solution in some cases.

A fairly trivial example:

keystone.createList('User', {
  fields: {
    name: { type: Text },
    dob: { type: Date },
    virtuals: { type: Relationship, ref: 'UserVirtual', many: false }
  }
});

// This is backed by a view, created by the app developer, that contains the required fields
keystone.createList('UserVirtual', {
  options: { isReadOnly: true },
  fields: {
    daysTillBirthday: { type: Integer },
    postCount: { type: Integer },
    quoteOfTheDay: { type: String }
  }
});

Callers can then query the calculated values by joining through to them in graphQL:

query {
  User {
    name
    virtuals: { daysTillBirthday }
}

And the underlying view can contain whatever kind of logic the DB supports including things like materialised views, when available.

Pros:

• Should work on most (almost all) DBs (inc. Mongo which has views since v3.4)
• Doesn't resolve/calculate the virtual fields unless they (any of them) are accessed
• The "virtual" values are available at a DB level so can be reused in related systems (eg. BI dashboards, etc.)

Cons:

• Requires DB support (although support is very common)
• Requires the dev to create the underlying views manually (and keep the KS list definition in sync)
• Requires "read-only" lists which we don't currently have (but should be pretty trivial to implement?)
• As is, if _any_ field on the related list is accessed it will cause all fields to be calculated (until we address #220)

This is kind of solution is complementary to other approaches. You could use a related, read-only list and the Virtual field type from Jess' example on the same list.

[jesstelford]

No reason these can't both be types 🎉

[molomby]

Yeah, at the extremes they solve very different problems -- for things like _label (#1116) you really want a Virtual type. On the other hand, if you have a set of reporting views that power you BI dashboard or something, being able to access that data via it's relationship with the operational data can be very useful.

There are also a lot of use cases in the middle ground where either approach could be used; a set of "calculated" fields that can pull in data from elsewhere in the system and perform sophisticated transformation on it.

I've separate this "read-only list" solution (#1133) so as not to side-track this issue.

[VinayaSathyanarayana]

Any update on this?

[VinayaSathyanarayana]

This will enable lot more solutions to be built on Keystone. Any progress on this