Node-redis: Roadmap

[BridgeAR]

EDIT: @Salakar: See comment below; https://github.com/NodeRedis/node_redis/issues/1040#issuecomment-581418899

---

This roadmap is not sorted and has no fixed dates but is more a general mashup of things.

Main features to be implemented

• [ ] Cluster
• [ ] Sentinel
• [ ] Input transformers
• [ ] Output transformers
• [ ] Native promise support
• [ ] Offline queue limiter
• [ ] Better script support / add individual commands

• [ ] Better keep alive function / improve detecting dead connections

  Other things that should be tackled

• [ ] Document all code (JSDoc)

• [ ] Make non documented API privat
• [ ] Update README documentation to become usefuller
• [ ] Fix windows spawn on appveyor
• [x] Better stacktraces while not in production mode
• [ ] Smoke tests

If you feel like there's something missing, please feel free to make further suggestions / open a feature request for it.

[darksigma]

1085 Do we have support for NX/XX flag support for commands like ZADD

[MattJustMatt]

It's been a few months, is there any updated timeline on this front?

[superlbr]

+1

[englercj]

Any chance cluster/sharding support will come soon? AWS has redis cluster support on ElastiCache and I would like to use it, but this lib needs to catch up to that feature set for that to really be viable :(

[naseemkullah]

This issue should be pinned IMHO

Also https://github.com/gosquared/redis-clustr might be a good enough solution for Cluster support?

An equivalent wrapper for sentinel would be great. Maybe https://www.npmjs.com/package/redis-sentinel but seems kind of dead (4 years since last publish).

[Salakar]

Perhaps we should discuss the future of this repository here; the last publish to NPM of this package was over 2 years ago, there's fixes on master that haven't been published to NPM for almost 2 years, e.g. https://github.com/NodeRedis/node_redis/issues/1331;

Note that this isn't me complaining about @BridgeAR, he's actually doing a great job over in @nodejs, so understandably his time for this repo is limited.

I'm willing to pickup some of the maintenance burden but I'd like some thoughts on what course of action we could take for this given that NPM publish access is not available for the existing repository (have requested it several times over the years).

Right now it seems like the only option is to fork and start again, unless we gain access.

[stockholmux]

@Salakar I'm also willing to help move this package forward. I'm not quite sure why we need to abandon the NPM package name ('redis' is powerful). Doesn't @BridgeAR control the NPM package and needs to transfer this to someone? There has been little action in many months, and I don't quite understand the logic of just sitting on it, to be honest.

I don't think we should deprecate this package - it's a good codebase that can be brought up to to date and huge number of other packages depend upon it.

The other thing I'd like to surface is the upcoming RESP3 / Redis 6 changes that will require significant alterations. I've looked into the ACL feature in Redis 6 which should be easy to support but we'll need some serious refactoring for node_redis. My job (at Redis Labs) would support my work on this module, but if we can't do a NPM release, it doesn't make sense investing the time.

[Salakar]

Doesn't @BridgeAR control the NPM package and needs to transfer this to someone? There has been little action in many months, and I don't quite understand the logic of just sitting on it, to be honest.

Correct, but I've been requesting NPM publish access since Feb 2018, again in Feb 2019, and the most recent request in Sep 2019. I've had responses but not on the topic of requesting NPM publish access 🤷‍♂. https://github.com/NodeRedis/node_redis/issues/1402#issuecomment-490273744 maybe gives some indication of intentions?

If the transfer isn't going to happen then I think it needs to be made clear so we can move on.

ioredis for example is real nice, and there was talks of consolidating the libraries back in the day (I worked on some of the underlying tooling for that such as the new parser, denque lib, cluster key slot calc etc): https://github.com/NodeRedis/node_redis#consolidation-its-time-for-celebration - which I still think should be the long term goal?

[Salakar]

Off topic but I did start experimenting on building a new client a while back; https://twitter.com/mikediarmid/status/1074240036936318976 - but I stopped on it in the hopes of getting this one started again or helping consolidate;

Perhaps thats also a route 🤷‍♂

[stockholmux]

@Salakar have you talked to any of the other NPM owners (Matt, Ben, or Bryce)? If Ruben is MIA, then if we want the project to move forward I don't think a single person should be a roadblock. This sort of problem is why it was setup this way, I assume. I find the comment on 1402 troubling for an open source project, especially one that isn't tied (organizationally) to a single person.

I agree, ioredis is good but I don't think it's a one-size fits all solution. As far as consolidation, I thought the unified parser was the major goal that was already accomplished. I never considered that there would be fully a single module, just given the syntax differences.

[Salakar]

@stockholmux: My job (at Redis Labs) would support my work on this module, but if we can't do a NPM release, it doesn't make sense investing the time.

Ditto also, we're willing to dedicate resource on this too @invertase, but if we can't publish it then it doesn't make sense for us either.

---

@stockholmux: @Salakar have you talked to any of the other NPM owners (Matt, Ben, or Bryce)?

This is a good point, I have not, I'll reach out to them shortly.

---

@stockholmux: I agree, ioredis is good but I don't think it's a one-size fits all solution. As far as consolidation, I thought the unified parser was the major goal that was already accomplished. I never considered that there would be fully a single module, just given the syntax differences.

Out of interest what is your requirements that the redis lib meets but ioredis does not? My requirements involved clustering & sentinel which redis currently does not support except though some third-party packages, some of which have also been abandoned.

Perhaps the underlying connection protocols between the two libraries could be shared also, then its purely just how you interface with each that is different?

[stockholmux]

@Salakar I like the modular approach to sentinel and clustering over the monolithic approach of ioredis (again, we need to deal with abandonware). Some people need this, others don't. The overall Redis ecosystem is getting bigger and modular is the way of supporting more without complexity, IMHO. Ioredis is a lot larger of a codebase (18,897 vs 7,038 lines of code), which may be more features, but I'd rather not have that much extra stuff I'm not using.

The place where I think node_redis has a big edge is Redis module support, which is easy with node_redis and rather a pain with ioredis.

[Salakar]

I've reached out to @mranney via Twitter DMs and asked if he could grant both myself and @stockholmux owner access to the GitHub org and the NPM packages, so we'll see what comes from that.

I think myself & @stockholmux agree that maintaining and publishing everything from where they are currently is the best way forward - should we be able to. If not I guess we can look at alternatives, though I'd rather not.

[szagi3891]

A little suggestion. Maybe you should consider migrating source code to TypeScript.

[Salakar]

Hey everyone, I've taken over as lead maintainer and have all the required access now 🎉 huge thanks to @BridgeAR for all the work he's done (and is doing) on this library and for letting me take over.

I've spent the last few days getting master ready for a release, and as of a few minutes ago I've just published v3.0.0 to NPM; https://github.com/NodeRedis/node-redis/releases/tag/v3.0.0 - which includes this change.

Expect regular releases - my first priority at the moment is making this project more contributor friendly to ensure that the project lives and continues to grow and is not blocked by any singular person's time. To do this I'd like to grow a larger set of shallow contributors. With this I hope to mitigate the previous problem of the project needing updates but there being no-one who has the power to do so. I'm working through the following on this;

• [x] Contributing docs & Code of Conduct
• [x] Setup an Open Collective and a contributors expenses policy
  
  
  
  • You'll notice the shiny new Sponsor button at the top of GitHub, I've also gone ahead and sponsored it myself and through my company to help kick start it for any future contributors
  
  
• WIP: Release automation & semantic versioning (publishing to NPM, generating changelogs, etc)
• [x] Improve CI, e.g. Windows CI is super slow & flakey right now

After that I'll be switching my attention to modernising (e.g. promises, typescript) & clearing technical debt in the Node Redis codebase. @BridgeAR has already done a boat load of stuff for this, if you're curious check out the WIP v4 branch and its changelog.

[GCSBOSS]

@Salakar Congrats! I wanted to pick up the promises job (would love to ditch async-redis) but I assume it's mostly done by now. You guys have any idea of a timeframe? Are you accepting contributions on that front (eg.: do we have some sort of checklist)?

[Salakar]

Hey @GCSBOSS, check out the 'v4' branch - its the in progress refactor, which has promise support, no eta / timeframe though yet sorry