Ioredis: Connection Pooling

[ScheerMT]

I noticed you had that on the roadmap. I don't see any public work going on with it currently.. Have you put any thought in how to accomplish this yet or began work? I can at least help with spitballing ideas.

[luin]

The most useful feature I want to implement is Read Write Splitting. For instance, if I have a master and N slaves, I would like to splitting reads and writes so that reads go to the slave servers and writes go to the master server:

// here we initialized a typical Redis instance connected to the master server
var master = new Redis();

// and then use it to initialize a Pool instannce.
// the pool will connect to all slaves of the master automatically.
var pool = new Redis.Pool(master);

// writes will be sent to the master
pool.set('foo', 'bar');

// and reads will be sent to one of the slaves.
// however, the result may not be 'bar' because of the replication latency.
pool.get('foo');

[ScheerMT]

Wouldn't this kind of splitting already be handled if you have a redis cluster running?
Relevant Redis documentation

For pooling, I was imagining a situation where no matter how many times you require('ioredis'), it would always use the same connection to talk to the redis server. This would create a problem for pub sub type applications though now that I think about it...

[AVVS]

Pooling is having multiple open connections to the same db and use them in parallel. Technically it can improve speed if the latency is a problem - ie bottleneck is network and not cpu of redis server

On 14 Sep 2015, at 18:09, Michael Taylor Scheer [email protected] wrote:

Wouldn't this kind of splitting already be handled if you have a redis cluster running?
Relevant Redis documentation

For pooling, I was imagining a situation where no matter how many times you require('ioredis'), it would always use the same connection to talk to the redis server. This would create a problem for pub sub type applications though now that I think about it...

—
Reply to this email directly or view it on GitHub.

[ScheerMT]

Ah, yeah. After having read over another library (seriate) which does leverage pooling in that context I should have instead thought of that.

Thanks for clearing that up :)

Would pooling in the context of AVVS be something that could have a benefit for redis performance?

[fisherwj]

我用redis作nodejs的session，但是redis的server端会针对闲置的连接设置了超时断开。 虽然ioredis有重试机制，但是在重试过程中遇到有用户请求时，是不是会读不到session呢？我看ioredis没有连接池，如果遇到这种情况一般是怎么解决的呢？

[luin]

这个和连接池没关系。重连过程很快，期间的命令都会被记录下来，并在连接成功后重新发送。另外新版 ioredis 默认支持 tcp keepalive

[vieks]

There is absolutely any interest having connections pooling per process (application) since redis server is single threaded and is processing sequentially all the client queries.

[majintao]

也就是说，ioRedis 的 Cluster 性能，单Client 连接就行了？ 不建议用其他的，比如：
https://github.com/coopernurse/node-pool

[majintao]

确实我也试过用这个https://github.com/coopernurse/node-pool， 连接数一直下不来！

[luin]

@majintao 没有必要用 pool，单 client 连接足够了。

[doublesharp]

@luin What do you think about scaleReads for Sentinel groups? It could possibly use a connection pool or just two connections - master for writes and a slave (by preference/priority) for reads?

[luin]

@doublesharp scaleReads for Sentinel groups should not be hard to be implemented on the client side (Unlike the one for cluster, which have to deal with slots and redirections), so I suggest to not include this feature in ioredis, and instead, write a library for it.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 7 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[guyisra]

So no connection pooling?

[doublesharp]

You can use generic-pool to do it if you want. I actually implemented connection pooling with a min/max of 1 to help handle lock contentions for an app I built.

const Redis = require('ioredis');
const Pool = require('generic-pool');

const pool = Pool.createPool({
  create: function() {
    // create a new client
    return Promise.resolve(new Redis(...));
  },
  destroy: function(client) {
    // do something when destroyed?
    return Promise.resolve(client);
  },
},
{
  min: 1,
  max: 1,
  autostart: true,
});

// acquire a client
pool.acquire()
.then((client) => {
  // do something with the the client
});

[doublesharp]

You could use a similar method to the above with logic in the create to point each pool resource to a different read-only slave for example. You can only write to a single threaded instance so there isn't much benefit to pooling except in some unique cases.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 7 days if no further activity occurs, but feel free to re-open a closed issue if needed.