Lumen-framework: Redis queue : "Error while reading line from the server"

[trompx]

Hello,

First I am not sure this is a lumen specific error or a Laravel error too.
I have an app running on laravel (5.1) which dispatch jobs, handled by a lumen (5.1) worker.

I have multiple jobs running totally fine, however, if a job runs more than 60 seconds (I made some tests 59.75727891922s -> no error, 60.091041898727s -> error), I get the following :

`[2016-02-05 02:27:07] lumen.ERROR: exception 'PredisConnectionConnectionException' with message 'Error while reading line from the server. [tcp://127.0.0.1:6379]' in /var/www/app/vendor/predis/predis/src/Connection/AbstractConnection.php:168
Stack trace:

0 /var/www/app/vendor/predis/predis/src/Connection/StreamConnection.php(210): PredisConnectionAbstractConnection->onConnectionError('Error while rea...')

1 /var/www/app/vendor/predis/predis/src/Connection/AbstractConnection.php(133): PredisConnectionStreamConnection->read()

2 /var/www/app/vendor/predis/predis/src/Connection/AbstractConnection.php(125): PredisConnectionAbstractConnection->readResponse(Object(PredisCommandZSetRemove))

3 /var/www/app/vendor/predis/predis/src/Client.php(326): PredisConnectionAbstractConnection->executeCommand(Object(PredisCommandZSetRemove))

4 /var/www/app/vendor/predis/predis/src/Client.php(310): PredisClient->executeCommand(Object(PredisCommandZSetRemove))

5 /var/www/app/vendor/illuminate/queue/RedisQueue.php(154): PredisClient->__call('zrem', Array)

6 /var/www/app/vendor/illuminate/queue/RedisQueue.php(154): PredisClient->zrem('queues:processi...', '{"job":"Illumin...')

7 /var/www/app/vendor/illuminate/queue/Jobs/RedisJob.php(73): IlluminateQueueRedisQueue->deleteReserved('processing', '{"job":"Illumin...')

8 /var/www/app/vendor/illuminate/queue/CallQueuedHandler.php(46): IlluminateQueueJobsRedisJob->delete()

9 /var/www/app/vendor/illuminate/queue/Jobs/Job.php(129): IlluminateQueueCallQueuedHandler->call(Object(IlluminateQueueJobsRedisJob), Array)

10 /var/www/app/vendor/illuminate/queue/Jobs/RedisJob.php(51): IlluminateQueueJobsJob->resolveAndFire(Array)

11 /var/www/app/vendor/illuminate/queue/Worker.php(208): IlluminateQueueJobsRedisJob->fire()

12 /var/www/app/vendor/illuminate/queue/Worker.php(159): IlluminateQueueWorker->process('redis', Object(IlluminateQueueJobsRedisJob), '3', 0)

13 /var/www/app/vendor/illuminate/queue/Worker.php(111): IlluminateQueueWorker->pop(NULL, 'processing', 0, '3', '3')

14 /var/www/app/vendor/illuminate/queue/Worker.php(87): IlluminateQueueWorker->runNextJobForDaemon(NULL, 'processing', 0, '3', '3')

15 /var/www/app/vendor/illuminate/queue/Console/WorkCommand.php(103): IlluminateQueueWorker->daemon(NULL, 'processing', 0, 128, '3', '3')

16 /var/www/app/vendor/illuminate/queue/Console/WorkCommand.php(71): IlluminateQueueConsoleWorkCommand->runWorker(NULL, 'processing', 0, 128, true)

17 [internal function]: IlluminateQueueConsoleWorkCommand->fire()

18 /var/www/app/vendor/illuminate/container/Container.php(502): call_user_func_array(Array, Array)

19 /var/www/app/vendor/illuminate/console/Command.php(150): IlluminateContainerContainer->call(Array)

20 /var/www/app/vendor/symfony/console/Command/Command.php(259): IlluminateConsoleCommand->execute(Object(SymfonyComponentConsoleInputArgvInput), Object(SymfonyComponentConsoleOutputConsoleOutput))

21 /var/www/app/vendor/illuminate/console/Command.php(136): SymfonyComponentConsoleCommandCommand->run(Object(SymfonyComponentConsoleInputArgvInput), Object(SymfonyComponentConsoleOutputConsoleOutput))

22 /var/www/app/vendor/symfony/console/Application.php(878): IlluminateConsoleCommand->run(Object(SymfonyComponentConsoleInputArgvInput), Object(SymfonyComponentConsoleOutputConsoleOutput))

23 /var/www/app/vendor/symfony/console/Application.php(195): SymfonyComponentConsoleApplication->doRunCommand(Object(IlluminateQueueConsoleWorkCommand), Object(SymfonyComponentConsoleInputArgvInput), Object(SymfonyComponentConsoleOutputConsoleOutput))

24 /var/www/app/vendor/symfony/console/Application.php(126): SymfonyComponentConsoleApplication->doRun(Object(SymfonyComponentConsoleInputArgvInput), Object(SymfonyComponentConsoleOutputConsoleOutput))

25 /var/www/app/vendor/laravel/lumen-framework/src/Console/Kernel.php(78): SymfonyComponentConsoleApplication->run(Object(SymfonyComponentConsoleInputArgvInput), Object(SymfonyComponentConsoleOutputConsoleOutput))

26 /var/www/app/artisan(35): LaravelLumenConsoleKernel->handle(Object(SymfonyComponentConsoleInputArgvInput), Object(SymfonyComponentConsoleOutputConsoleOutput))

27 {main}

`

I stumble upon this issue https://github.com/nrk/predis/issues/121 but that wasn't that.

I have set an expire time of 120 seconds (https://github.com/laravel/framework/issues/8577), tried to add the 'read_write_timeout' => -1 parameter (http://stackoverflow.com/questions/11776029/predis-is-giving-error-while-reading-line-from-server) in database.php but nothing works.

For the record, some of the jobs use PhpRedis (no persistent connection) so I don't know if that may conflict with Predis (which seems not as all jobs running under 60s don't trigger the error).

My worker config in supervisor is the following :
[program:queue_processing] command=php artisan queue:work --daemon --sleep=3 --queue=processing --tries=3 process_name=%(program_name)s_%(process_num)02d numprocs=4 autostart=true autorestart=true

Let me know if I can provide more infos.

[GrahamCampbell]

It doesn't look like an issue with laravel, rather either with your server, or the packages we're using to communicate with redis.

[HashmatWaziri]

try this
$redis = $app->make('redis'); // I am using this in route
$redis->set('stats', 'd');

[rbdoer]

same problem. any updates?

[shmuelgutman]

I encounter this problem also, for me it's happening while I'm trying the example from https://laravel.com/docs/5.5/redis#pubsub
after a few minutes of listening I get the error Error while reading line from the server. [tcp://127.0.0.1:6379] and the process terminated.

[ivanpeter]

I had the same problem. Solved it by installing redis-server on ubuntu, and then starting it using redis-cli ping

1. sudo apt-get install redis-tools
2. sudo apt-get install redis-server
3. redis-cli ping
   Hope this will be helpful

[freeman3s]

@shmuelgutman I had the same problem with Laravel 5.4. Have you solve it?

[freeman3s]

@shmuelgutman I have fixed it! I've added
'read_write_timeout' => 0 in config/database.php

[maokeyang]

@freeman3s i have added,but nothing works. something not to do?

[freeman3s]

@maokeyang strange, it helped me
'read_write_timeout' => 0

[robinvdvleuten]

You could also try to set the value to -1 to completely disable the timeout as mentioned by the author of Predis https://github.com/nrk/predis/issues/33#issuecomment-1395652. This was the case for me as I was running Laravel Horizon as a daemon script on the server.

[jezmck]

You may also need to ensure you have the scheme set to tls if appropriate.

[craigy-waigy]

@shmuelgutman I have fixed it! I've added
'read_write_timeout' => 0 in config/database.php

'redis' => [

    'client' => env('REDIS_CLIENT', 'predis'),

    'options' => [
        'cluster' => env('REDIS_CLUSTER', 'redis'),
        'prefix' => env('REDIS_PREFIX', Str::slug(env('APP_NAME', 'laravel'), '_').'_database_'),
    ],

    'default' => [
        'url' => env('REDIS_URL'),
        'host' => env('REDIS_HOST', '127.0.0.1'),
        'password' => env('REDIS_PASSWORD', null),
        'port' => env('REDIS_PORT', '6379'),
        'database' => env('REDIS_DB', '0'),
        'read_write_timeout' => 0,
    ],