Loopback: EPIC: KeyValue models

Created on 20 Jul 2016  路  12Comments  路  Source: strongloop/loopback

Problem

I should be able to easily read and write values in cache-friendly data stores. I should have control over exactly when each value expires. I should be able to choose my store.

Example

var wait = require('timer-as-promised');
var Color = app.createModel({
  base: 'KeyValueModel',
  keyValue: {
    ttl: {
      default: 120
    },
    namespace: 'colors'
  }
});
var redis = app.createDataSource('redis', {
  host: 'localhost',
  port: 6379
});
Color.attachTo(redis);

var SECONDS_TO_LIVE = 60;
var key = Math.random().toString().split('.')[1];
var myValue = new Color({
  key: key,
  value: 'blue',
});

myValue
  .setWithTTL(SECONDS_TO_LIVE)
  .then(function() {
    return myValue.ttl();
  })
  .then(function(ttl) {
    console.log(ttl); // 60
    return myValue.expire(120);
  })
  .then(function() {
    return wait(120);
  })
  .then(function() {
    return myValue.get();
  })
  .then(function() {
    console.log(myValue.value); // => undefined
  });

Tasks

customer feature major
Source
picture ritch

All 12 comments

@ritch I found few confusing parts of your example, could you please edit or clarify?

1) Where is the constructor MyCacheItem defined?
2) What's the difference between myValue.setWithTTL() and myValue.expire()?

We already have a redis connector (link). I am worried about possible confusion - when to use loopback-connector-redis and loopback-connector-kv-redis.

I made a quick research, it seams to me that SQL servers don't provide key expiration, and on the other hand, Redis does not support advanced data-access methods like "updateOrCreate", so perhaps there should be only one connector for Redis, and it should implement the key/value API only (not DataAccessObject)?

Thoughts?

bajtos picture bajtos on 22 Jul 2016

Do you prefer static methods or prototype methods for KV operations? For example:

MyKVModel.set(key, val, ttl, cb) vs MyKVModel.prototype.set(ttl, cb)

Please note that we can use the model name as namespace for keys.

raymondfeng picture raymondfeng on 22 Jul 2016

Connect to #2545

superkhau picture superkhau on 25 Jul 2016

Please note that we can use the model name as namespace for keys.

馃憤

Do you prefer static methods or prototype methods 

// I prefer this as the implementation
var kv = new KeyValue({key: 'my key', value: 'my value'});
kv.set();

// and this as the syntax sugar
KeyValue.set('my key', 'my value');

That being said we should just support both flavors.

Where is the constructor MyCacheItem defined?

Typo, edited.

What's the difference between myValue.setWithTTL() and myValue.expire()?

Atomic version of set() + expire() (expire() only sets the TTL)

I made a quick research, it seams to me that SQL servers don't provide key expiration, and on the other hand, Redis does not support advanced data-access methods like "updateOrCreate", so perhaps there should be only one connector for Redis, and it should implement the key/value API only (not DataAccessObject)?

This is exactly the reason I am suggesting we create a key-value based set of connectors. The redis connector would probably be deprecated, since it doesn't really provide enough of the DAO connector interface to be interoperable anyway.

ritch picture ritch on 26 Jul 2016
👍1

IMO, we should provide an in-memory implementation of the KV store, similarly how we have in-memory implementation of regular DataSource. This in-memory implementation should be created as part of our work on key-value models.

cc @superkhau

bajtos picture bajtos on 27 Jul 2016
👍1

This in-memory implementation should be created as part of our work on key-value models.

I agree with this because we need an in-mem connector to write tests against.

superkhau picture superkhau on 27 Jul 2016

Example RI recommended by @raymondfeng: https://github.com/isaacs/node-lru-cache

superkhau picture superkhau on 28 Jul 2016
  • Use it as the underlying module but expose our apis

    • Create our own in-mem connector that uses lru cache as the underlying module

superkhau picture superkhau on 28 Jul 2016

@bajtos I added a new bullet list item blog post for the feature + example. Also created corresponding issue at https://github.com/strongloop-internal/scrum-loopback/issues/1015.

superkhau picture superkhau on 22 Aug 2016
👍1

@bajtos @superkhau - should we deprecate the redis connector in favor of this new redis kv connector?

ritch picture ritch on 19 Sep 2016

@ritch I'm not sure about this yet, there was one person using it recently in the PersistedModel fashion. I spoke with @raymondfeng about this before and we need a way to inject usage styles/templates to the connector at boot/runtimes to allow different usage styles for the same connector.

superkhau picture superkhau on 19 Sep 2016

Closing this issue in favour of leftover individual stories:

superkhau picture superkhau on 19 Oct 2016
Was this page helpful?
0 / 5 - 0 ratings

Related issues

bajtos picture bajtos  路  3Comments
JoeShi picture JoeShi  路  4Comments
bajtos picture bajtos  路  4Comments
imasif picture imasif  路  3Comments
nmklong picture nmklong  路  3Comments