Ioredis: using the results of "KEYS" when a `keyPrefix` option is being used

[iamjochem]

hi there,

when a keyPrefix option is used is becomes a little tricky to work with the KEYS command:

1. you have to manually apply the prefix in the argument to "KEYS" in order to reasonably find keys
   that match
2. you have to manually remove the prefix from values in the returned array of in order to use them

the following example shows searching for keys and then attempting to deleting them (nothing is deleted because redis.del() will actually attempt to delete "foo:foo:bar:baz")

    const Promise = require('bluebird');
    const Redis = require('ioredis');
    const redis = new Redis({keyPrefix: 'foo:'});

    redis.set('bar:baz', 1)
            .then(() => redis.keys('foo:*'))
            .then(keys => Promise.map(keys, k => redis.del(k)))
            ; 

I think it would be nice if redis.keys() supported a second argument of options so that you could do something like this (I'm hoping that the suggested keysOptions are self explanatory):

    const Promise = require('bluebird');
    const Redis = require('ioredis');
    const redis = new Redis({keyPrefix: 'foo:'});

    const keysOptions = {
        applyKeyPrefix : true,
        stripKeyPrefixFromResults : true,
    };

    redis.set('bar:baz', 1)
            .then(() => redis.keys('*', keysOptions)) // performs KEYS with "foo:*"
            .then(keys => Promise.map(keys, k => redis.del(k))) // returns ["bar:baz"]
            ; 

Granted you may not agree with this idea, regardless I'm interested to know if you have any suggestions/advice for using the results of "KEYS" when a keyPrefix option is being used.

Currently I'm using something akin to the following for stripping the keyPrefix value from the returned keys (bare in mind that the keyPrefix may contain a regex meta-character that needs to be escaped):

// "k" is a item from the arrya returned by "redis.keys()"
k.replace(RegExp('^' + redis.options.keyPrefix.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')), '')

IMO this is a bit inelegant (and error prone) to have to always do this in consuming code.

rgds,
Jochem

PS - enjoying using your lib, especially the defineCommand functionality! nice work!

[luin]

There're already several related issues posted: #254 #359 #325. A warning is added to the "Transparent Key Prefixing" section in README for this matter.

Currently I'm not planning to add support for prefixing patterns and replies given it's not a good practice to use key prefix as namespace to allow multiple applications share a same redis database.

PS - thank you for using ioredis 😆

[iamjochem]

hi,

I was aware of the related issues and the relevant part of the README. I am confused about your second comment ... are you saying it is NOT good practice to use a keyPrefix? (I thought that the whole point of the keyPrefix was to promote consistent name-spacing).

So is your advice to not use a keyPrefix ?

[marshallford]

Like @iamjochem, I too am confused. From reading and watching talks about Redis I was under the impression that name spacing was recommended.

[luin]

Name spacing is recommended only when it's used in the same application. For example, it's useful to store users and posts details in the same database by prefixing them with users: & posts::

SET users:178.name "Bob"
SET posts:201.name "Welcome to the WordPress"

keyPrefix is designed for this use case: you may create two Redis instances with different prefixes, one for users and the other for posts:

userRedis.set(`${id}.name`, 'bob')
postRedis.set(`${id}.name`, 'Welcome to the WordPress')

In the above case, keys (and scan) is rarely needed to list all keys that have the same prefixes since, in the production, you may have a SET for indexing them.

However, it's not recommended to have multiple application sharing the same database:

SET APP1:data.1 good
SET App2:data.1 bad

You should setup a single Redis database for each application.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 7 days if no further activity occurs, but feel free to re-open a closed issue if needed.