Ioredis: Chaining custom commands in cluster mode

[manast]

According to the documentation "Chaining custom commands in the pipeline is not supported in Cluster mode.".
I am wondering, is this a limitation of ioredis or of redis itself? and do you have any information or link on why this limitation exists?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 7 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[manast]

@luin please, could you be so kind to share any insight you have in this issue?

[luin]

Sorry for the late response.

That's not a limitation of Redis. It's just not easy to support chaining custom commands in the cluster setup: ioredis always try evalsha first when sending custom commands, and when the command fails, eval will be sent instead. However, when it comes to pipelines, to make sure all commands in a pipeline will be received by Redis in order, ioredis sends script exists command to check if all lua scripts have loaded in the target redis node and will load the scripts which haven't loaded before sending the pipeline.

In the cluster setup, the nodes may change (e.g. failover happens) during the process of loading scripts, so it's hard to make sure that the scripts are loaded to the correct redis node. That's why this feature haven't been implemented.

[manast]

thanks for the throughout explanation. When calling defineCommand, why can't we just call SCRIPT LOAD and then after that just assume that the command is available?

[luin]

@manast Script cache may be cleared by the SCRIPT FLUSH command. Despite that, calling SCRIPT LOAD when defining commands indeed solves the problem (we also need to resend all SCRIPT LOAD again when a new connection has been made). Good idea! 👍

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 7 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[manast]

Bump! :)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 7 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[btd]

Bump

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 7 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[manast]

Bump!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 7 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[btd]

Bump

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 7 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[manast]

Bump because it is an important issue!

[Rush]

Bump - interested in it as well.

[Gappa88]

interested

[firedfox]

Bump because we are using more Redis clusters in recent projects. This problem is getting more serious.

[ajwootto]

Also running into this.

[btzsoft]

we have the same issue.

[botzill]

Any updates on this issue? @luin

Thx.

[jl9404]

@manast Script cache may be cleared by the SCRIPT FLUSH command. Despite that, calling SCRIPT LOAD when defining commands indeed solves the problem (we also need to resend all SCRIPT LOAD again when a new connection has been made). Good idea!

@luin wondering if we could always send SCRIPT LOAD to the pipeline so we can EVALSHA safely? Thx.

[RPallas92]

Same issue here

[andreimc]

same issue here

[kaikash]

same issue here

[arttolstykh]

Any updates on this issue? @luin

[NivLipetz]

Hi @luin , are there any plans for this issue to be fixed or is it a lost cause?
Thanks!

[shyimo]

Hi @luin , any plans about adding this ?
we are now fully use Redis cluster and it's getting more serious now

[ioredis-robot]

:tada: This issue has been resolved in version 4.19.0 :tada:

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot :package::rocket:

[manast]

FYI, I just upgraded to 4.19.0 and some of Bull tests are failing. I need to investigate more, but for now I will need to lock ioredis to 4.18.0 in order to not break bull users environments.