kue (reds?) creates always a Redis connection to localhost when requiring

[esamattis]

Don't know if this is kue or reds issue, but it certainly affects kue. Tested on kue master.

Consider following:

var redis = require("redis");                                                                                                                
var client = redis.createClient(1234, "example.com");                           

setTimeout(function() {                                                         
    require("kue");                                                   
}, 1000);                                                                       

client.on("connect", function() {                                               
  console.log("Connected!");                                                    
});                                                                             

It dies immediately after kue is required in the timeout:

Connected!

node.js:134
        throw e; // process.nextTick error, or 'error' event on first tick
        ^
Error: Redis connection to 127.0.0.1:6379 failed - ECONNREFUSED, Connection refused
    at Socket.<anonymous> (/home/epeli/tmp/kue/node_modules/reds/node_modules/redis/index.js:123:28)
    at Socket.emit (events.js:64:17)
    at Array.<anonymous> (net.js:830:27)
    at EventEmitter._tickCallback (node.js:126:26)

shell returned 1

Creates a second connection to localhost?

I think the reason is here:

A search is created on require
https://github.com/LearnBoost/kue/blob/b2cd5d3393d12e4bdd1c2e654a3ef5eb45e5bf11/lib/queue/job.js#L27

which means new Search on Reds
https://github.com/visionmedia/reds/blob/156c046803df6b96fa40e90be26b933f7d324d01/lib/reds.js#L68

which then again means a new Redis client which connects to localhost
https://github.com/visionmedia/reds/blob/156c046803df6b96fa40e90be26b933f7d324d01/lib/reds.js#L236

This happens also if you override the createClient function in kue.

[davidwood]

I'm confused - are you using Kue or reds? I haven't used reds, but in looking at the package.json, it doesn't look like it uses Kue.

If we just focus on Kue and look at your example, you're requiring the node.js Redis client, but not actually overriding the Kue function that creates the client. Try the following to create a Kue instance (named jobs) with a connection to Redis on example.com:1234:

var redis = require('redis'),
    kue = require('kue');

kue.redis.createClient = function() {
    var client = redis.createClient(1234, "example.com");
    return client;
}

var jobs = kue.createQueue();

[esamattis]

I'm using Kue and Kue uses Reds under the hood.

As I stated, it doesn't help if you override the creator function in kue. Your code fails with the same error.

[davidwood]

What version of Kue are you using? I just ran the above code using Kue 0.2.0 without issue. Your code (and subsequent stack trace) wasn't overriding the Kue creator function - can you provide code that overrides the creator function and the stack trace?

[esamattis]

I'm using Kue master from Github.

Is there a Redis instance running on your localhost when you tested the code?

[esamattis]

I'll get this if I use the code you posted (just changed the server and port):

node.js:134
        throw e; // process.nextTick error, or 'error' event on first tick
        ^
Error: Redis connection to 127.0.0.1:6379 failed - ECONNREFUSED, Connection refused
    at Socket.<anonymous> (/home/epeli/tmp/kue/node_modules/reds/node_modules/redis/index.js:123:28)
    at Socket.emit (events.js:64:17)
    at Array.<anonymous> (net.js:830:27)
    at EventEmitter._tickCallback (node.js:126:26)

[esamattis]

Also if add an another Redis instance to my localhost: Both Redis instances, remote and the local, will get connected when I run your code and I won't get the error. Obviously.

[davidwood]

Ok, I see what's happening now. I was looking at the 0.2.0 release, not master. I have a workaround - it's not pretty, but it works :)

var redis = require('redis'),
    reds = require('reds');

reds.createClient = function() {
  var client = redis.createClient(1234, "example.com");
  return client;
}

var kue = require('kue');
kue.redis.createClient = reds.createClient;
var jobs = kue.createQueue();    

[davidwood]

In order for this to work, you'll need to make sure reds is installed for your application (with npm install reds) and not installed under Kue (rm -R node_modules/kue/node_modules/reds)

[tj]

haha the joys of trying to share config like this _blah_. Yeah, we'll definitely need a reasonable solution for this in the next release. We dont have much choice other than to do kue.reds.createClient = kue.redis.createClient = fn etc, ugly but other than duplicating a ton of other logic in redis etc there's no easy way around it

[davidwood]

The thing that makes this tricky is that lib/queue/job.js and lib/http/routes/json.js create the reds search when required, so currently you need to override the reds Redis client creation function before Kue is required. This seems less than ideal, so I've created a branch (named lazy-create-reds) that makes this a bit easier by lazily creating the reds search with an overridden reds.createClient upon first access.

This change updates lib/queue/job.js and lib/http/routes/json.js.

[tj]

it's ideal as far as simplicity goes but yeah im fine with changing it for this

[tj]

@davidwood merged

I wish we could do it with less of a hack, but I can't think of anything off hand

[davidwood]

I'm with you - it works, but doesn't feel quite right

[larsemann]

hi,

the problem still exists, with version 0.3.2 :/

is there any fix now? i'm trying to use kue on different machines with one redis server :)

Update: switched to 0.3.1. - everything working fine...
thank you anyway! keep up the good work!

[ntresch]

I don't understand the assertion that Kue uses Reds under the hood when Reds was written to emulate Kue?

[tj]

@ntresch huh? reds is full-text search, nothing more

[ntresch]

@visionmedia "I'm using Kue and Kue uses Reds under the hood.", was said by @epeli. I don't believe that this is true, and I don;t understand the assertion, hence the question.

[tj]

it is true, Kue uses Reds for the search

[ntresch]

That makes more sense, with the qualification "for the search", and I see it now in the source. Thanks!

[loudin]

This is still an issue for 0.4.0:

Error: Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED

[judyhe]

Yep same with 0.5.0

[georgecoltart]

Pretty sure this is still happening, I can't stop the connection to localhost

[schonfeld]

Still an issue with 0.7.5. Any chance of seeing a fix for this soon?

[behrad]

@schonfeld would you provide the code for reproducing this?

[behrad]

@schonfeld !!!!????

[iamkale]

I wandered over from Google, but this is happening to me in a standard node/redis/express app (bare bones..). Running on locahost connecting to a remote redis server (Elasticache on AWS) works fine, but from my EC2 server that doesn't have redis installed I can't seem to connect, since it tries to connect to localhost first even though I have config specific for the remote server running.

Sorry for off-topic, but this was one of few places I found this issue.

[behrad]

would u please provide the stack trace of connection error?

[judyhe]

We forked awhile ago (0.5.0) and updated locally: https://github.com/floored/kue/commit/ec021e07b6ceea9f77bb9556ac2fb8035a880620. Never did a pull request because we assumed it would get fixed "for real".

[behrad]

should be fixed in 0.8

[rvbsanjose]

I'm trying to connect with Redis that is deployed on Heroku and I used the example @davidwood provided but I am still receiving the Error: Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED. Any other ideas on how to fix this?

[behrad]

@rvbsanjose make sure if you are calling createQueue(...) before accessing kue.app express application, and also try with disableSearch: true.
Finally provide the code so that I can fix it, however I don't believe it is a bug.

[rvbsanjose]

@behrad I feel like a dummy. I was following along with the tutorials and never bothered to read the full doc on connecting. Once I read further down the doc page and followed the instructions, it worked just fine. Sorry for the inconvenience.

[petermekhaeil]

What was the solution? I am getting this problem after a successful connect to a non-localhost url.

[eaglevision20]

I've been beating my head against this wall for a full day now. I'm running kue version 0.11 and node 4.5 but still cannot connect to a remote host. here is what my connection looks like:

const config = require('./config');
var Kue = require('kue');
var queue = Kue.createQueue({
redis: {
port: 6379, //process.env.REDIS_PORT,
host: 'DOMAIN_HERE.com' //process.env.REDIS_HOST,
//auth: process.env.REDIS_PASS
},
jobEvents: false,
disableSearch: true,
});

No matter what I try I get the same error back:

events.js:141
throw er; // Unhandled 'error' event
^

Error: Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED 127.0.0.1:6379
at Object.exports._errnoException (util.js:907:11)
at exports._exceptionWithHostPort (util.js:930:20)
at TCPConnectWrap.afterConnect as oncomplete

I have tried every example I can find and still nothing.. :(

Any help is greatly appreciated.

[59023g]

@eaglevision20 closest thing I could find: https://github.com/Automattic/kue/issues/875

If I have time I'll fork and look..

Maybe check the Pull Requests

[ganeshcse2991]

Still facing the same issue. Tried all the examples and pull requests too. Please help. Its still creating queue with local redis.

Error: Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED 127.0.0.1:6379
at Object.exports._errnoException (util.js:907:11)
at exports._exceptionWithHostPort (util.js:930:20)
at TCPConnectWrap.afterConnect as oncomplete

[fabulator]

@ganeshcse2991 as was said in comment earlier https://github.com/Automattic/kue/issues/54#issuecomment-57683955 Problem is requiring app from kue before creating a queue. Something like this: import kue, { app } from 'kue'; At least it was my issue and it is working fine for me now.

[shierro]

@behrad thanks for the input about creating the queue before calling kue.app you are awesome!

queueService.initQueue(REDIS_HOST, REDIS_PORT);
app.use('/kue-api/', basicAuth, require('kue').app);

[sinisavukovic]

@behrad @shierro. Thanks!

For those who are still looking for a quick answer:

import kue from "kue";
import express from "express";

kue.createQueue({
  prefix: 'q',
  redis: {
    port: <REDIS_PORT>,
    host: "<REDIS_HOST>"
  }
});

const app = express();
app.use("/kue-api/", kue.app);