Node-redis: ECONNRESET after 2 hours

[strumwolf]

Currently running an application in Docker, in an Ubuntu 12.04 container. Running docker 1.5 and node-redis 0.12.1. The redis connection receives the following error after 2 hours, every time.
```Error: Redis connection to web-redis01.twec.com:6379 failed - read ECONNRESET

``````
at RedisClient.on_error (/home/node/tjs0006/node_modules/redis/index.js:196:24)

at Socket. (/home/node/tjs0006/node_modules/redis/index.js:106:14)

at Socket.emit (events.js:95:17)

at net.js:441:14

at process._tickCallback (node.js:448:13)```
``````

Logs are coming...

[strumwolf]

Stream connected web-redis01.twec.com:6379 id 1
Using default parser module: javascript
checking server ready state...
send web-redis01.twec.com:6379 id 1: *1
$4
info

send_command buffered_writes: 0  should_buffer: false
net read web-redis01.twec.com:6379 id 1: $1993
# Server
redis_version:2.8.19
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:a789ed9377dc12b9
redis_mode:standalone
os:Linux 3.13.0-45-generic x86_64
arch_bits:64
multiplexing_api:epoll
gcc_version:4.6.3
process_id:1478
run_id:6edf95e3c651843f8c26ea0e1030717d82419fcd
tcp_port:6379
uptime_in_seconds:9604215
uptime_in_days:111
hz:10
lru_clock:7985573
config_file:/etc/redis/redis.conf

# Clients
connected_clients:521
client_longest_output_list:0
client_biggest_input_buf:817
blocked_clients:108

# Memory
used_memory:668850272
used_memory_human:637.87M
used_memory_rss:715665408
used_memory_peak:4418948960
used_memory_peak_human:4.12G
used_memory_lua:35840
mem_fragmentation_ratio:1.07
mem_allocator:jemalloc-3.6.0

# Persistence
loading:0
rdb_changes_since_last_save:232
rdb_bgsave_in_progress:0
rdb_last_save_time:1434048755
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:8
rdb_current_bgsave_time_sec:-1
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok

# Stats
total_connections_received:14731602
total_commands_processed:1397335226
instantaneous_ops_per_sec:60
total_net_input_bytes:111442619264
total_net_output_bytes:166924266821
instantaneous_input_kbps:4.35
instantaneous_output_kbps:0.61
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:1320224
evicted_keys:0
keyspace_hits:325125706
keyspace_misses:358675060
pubsub_channels:1
pubsub_patterns:0
latest_fork_usec:61950

# Replication
role:master
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

# CPU
used_cpu_sys:45186.85
used_cpu_user:22707.47
used_cpu_sys_children:25963.62
used_cpu_user_children:203823.30

# Keyspace
db0:keys=88916,expires=2405,avg_ttl=29636490


Redis server ready.
send web-redis01.twec.com:6379 id 1: *3
$4
sadd
$25
tjs0006-dev:todaysBatches
$18
batch_514C6E711866

send_command buffered_writes: 0  should_buffer: false
send web-redis01.twec.com:6379 id 1: *3
$3
set
$19
tjs0006-dev:lastRun
$24
2015-06-11T19:02:11.190Z

send_command buffered_writes: 0  should_buffer: false
net read web-redis01.twec.com:6379 id 1: :1

net read web-redis01.twec.com:6379 id 1: +OK

send web-redis01.twec.com:6379 id 1: *3
$5
lpush
$36
tjs0006-dev:batch_514C6E711866:error
$180
{"message":"Non-successful status code (500) returned","location":"RENEW QUEUE:Get Customer","batchId":"batch_514C6E711866","businessDate":"","file":"","cardNumber":"466708132564"}

send_command buffered_writes: 0  should_buffer: false
net read web-redis01.twec.com:6379 id 1: :1

send web-redis01.twec.com:6379 id 1: *4
$6
lrange
$36
tjs0006-dev:batch_514C6E711866:error
$1
0
$2
-1

send_command buffered_writes: 0  should_buffer: false
net read web-redis01.twec.com:6379 id 1: *1
$180
{"message":"Non-successful status code (500) returned","location":"RENEW QUEUE:Get Customer","batchId":"batch_514C6E711866","businessDate":"","file":"","cardNumber":"466708132564"}

send web-redis01.twec.com:6379 id 1: *4
$6
lrange
$38
tjs0006-dev:batch_514C6E711866:warning
$1
0
$2
-1

send_command buffered_writes: 0  should_buffer: false
net read web-redis01.twec.com:6379 id 1: *0

Redis connection to web-redis01.twec.com:6379 failed - read ECONNRESET
Error: Redis connection to web-redis01.twec.com:6379 failed - read ECONNRESET
    at RedisClient.on_error (/home/node/tjs0006/node_modules/redis/index.js:196:24)
    at Socket.<anonymous> (/home/node/tjs0006/node_modules/redis/index.js:106:14)
    at Socket.emit (events.js:95:17)
    at net.js:441:14
    at process._tickCallback (node.js:448:13)
Error: Redis connection to web-redis01.twec.com:6379 failed - read ECONNRESET
    at RedisClient.on_error (/home/node/tjs0006/node_modules/redis/index.js:196:24)
    at Socket.<anonymous> (/home/node/tjs0006/node_modules/redis/index.js:106:14)
    at Socket.emit (events.js:95:17)
    at net.js:441:14
    at process._tickCallback (node.js:448:13)
error: Forever detected script exited with code: 1
error: Script restart attempt #1

[strumwolf]

I have tested other libraries and received the same issue. It does not seem like it is specific to node-redis.

[cabrinoob]

@strumwolf : I have exactly the same problem. Did you solve it ?

I'am using it in docker too (and rancher), and this is the log of my container :

02/11/2016 17:15:11Redis connection OK
02/11/2016 17:15:11Client subscribes to "test"
02/11/2016 19:15:32REDIS error { [Error: Redis connection to xxx.euw1.cache.amazonaws.com:6379 failed - read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
02/11/2016 19:15:32Redis reconnection
02/11/2016 19:15:32Redis connection OK
02/11/2016 19:15:32Client subscribes to "test"
02/11/2016 21:15:41REDIS error { [Error: Redis connection to xxx.euw1.cache.amazonaws.com:6379 failed - read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
02/11/2016 21:15:41Redis reconnection
02/11/2016 21:15:41Redis connection OK
02/11/2016 21:15:41Client subscribes to "test"
02/11/2016 23:15:50REDIS error { [Error: Redis connection to xxx.euw1.cache.amazonaws.com:6379 failed - read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
02/11/2016 23:15:50Redis reconnection
02/11/2016 23:15:50Redis connection OK

Every 2 hours as you said, I have an ECONNRESET. The reconnection seems to work, and the subscription too. If I PUSH a new message right after this reconnection, the subscriber recieve it.

But if I wait a certain time (approaching the next reconnection), messages I push are no more treated by the subscriber. It seems to be another timeout somewhere ...

Thank your for your help.

[strumwolf]

@cabrinoob, I solved the issue but it was so long ago I don't remember the solution. Plus I am no longer working for the company where I had he issue and therefore cannot go back and look it up. Sorry.