Freescout: artisan queue:work freezes

[davrot]

Hi,

I have a problem with sending out the response email (email test works fine). If I force queue:work to run then it sends out the emails. However artisan queue:work doesn't stop on its own. It hangs and goes on vacation. Cleaning the cache doesn't impress queue:work. No errors are shown:

/usr/local/bin/php /home/webserver/www/htdocs/neurotec/freescout/artisan queue:work --queue='emails,default' --sleep=5 --tries=1

[2021-04-19 01:31:46] Processing: App\Jobs\SendReplyToCustomer
[2021-04-19 01:31:47] Processed: App\Jobs\SendReplyToCustomer
[2021-04-19 01:31:47] Processing: App\Jobs\SendNotificationToUsers
[2021-04-19 01:31:47] Processed: App\Jobs\SendNotificationToUsers
[2021-04-19 01:31:47] Processing: App\Jobs\SendReplyToCustomer
[2021-04-19 01:31:47] Processed: App\Jobs\SendReplyToCustomer
[2021-04-19 01:31:47] Processing: App\Jobs\SendNotificationToUsers
[2021-04-19 01:31:47] Processed: App\Jobs\SendNotificationToUsers
[2021-04-19 01:31:47] Processing: App\Jobs\SendReplyToCustomer
[2021-04-19 01:31:47] Processed: App\Jobs\SendReplyToCustomer
[2021-04-19 01:31:47] Processing: App\Jobs\SendNotificationToUsers
[2021-04-19 01:31:47] Processed: App\Jobs\SendNotificationToUsers
[2021-04-19 01:31:47] Processing: App\Jobs\SendReplyToCustomer
[2021-04-19 01:31:47] Processed: App\Jobs\SendReplyToCustomer
[2021-04-19 01:31:47] Processing: App\Jobs\TriggerAction
[2021-04-19 01:31:47] Processed: App\Jobs\TriggerAction
[2021-04-19 01:31:47] Processing: App\Notifications\WebsiteNotification
[2021-04-19 01:31:47] Processed: App\Notifications\WebsiteNotification
[2021-04-19 01:31:47] Processing: App\Notifications\BroadcastNotification
[2021-04-19 01:31:47] Processing: App\Events\RealtimeBroadcastNotificationCreated
[2021-04-19 01:31:47] Processed: App\Events\RealtimeBroadcastNotificationCreated
[2021-04-19 01:31:47] Processed: App\Notifications\BroadcastNotification
[2021-04-19 01:31:47] Processing: App\Jobs\TriggerAction
[2021-04-19 01:31:47] Processed: App\Jobs\TriggerAction
[2021-04-19 01:31:47] Processing: App\Notifications\WebsiteNotification
[2021-04-19 01:31:47] Processed: App\Notifications\WebsiteNotification
[2021-04-19 01:31:47] Processing: App\Notifications\BroadcastNotification
[2021-04-19 01:31:47] Processing: App\Events\RealtimeBroadcastNotificationCreated
[2021-04-19 01:31:47] Processed: App\Events\RealtimeBroadcastNotificationCreated
[2021-04-19 01:31:47] Processed: App\Notifications\BroadcastNotification
[2021-04-19 01:31:47] Processing: App\Jobs\TriggerAction
[2021-04-19 01:31:47] Processed: App\Jobs\TriggerAction
[2021-04-19 01:31:47] Processing: App\Notifications\WebsiteNotification
[2021-04-19 01:31:47] Processed: App\Notifications\WebsiteNotification
[2021-04-19 01:31:47] Processing: App\Notifications\BroadcastNotification
[2021-04-19 01:31:47] Processing: App\Events\RealtimeBroadcastNotificationCreated
[2021-04-19 01:31:47] Processed: App\Events\RealtimeBroadcastNotificationCreated
[2021-04-19 01:31:47] Processed: App\Notifications\BroadcastNotification
[2021-04-19 01:31:47] Processing: App\Jobs\TriggerAction
[2021-04-19 01:31:47] Processed: App\Jobs\TriggerAction
=> And here it hangs...

And now the same happens with artisan schedule:run :

/usr/local/bin/php /home/webserver/www/htdocs/neurotec/freescout/artisan schedule:run

Running scheduled command: '/usr/local/bin/php-7.3' 'artisan' freescout:fetch-monitor > '/dev/null' 2>&1
Running scheduled command: '/usr/local/bin/php-7.3' 'artisan' freescout:check-conv-viewers > '/dev/null' 2>&1
Running scheduled command: '/usr/local/bin/php-7.3' 'artisan' freescout:fetch-emails > '/home/webserver/www/htdocs/neurotec/freescout/storage/logs/fetch-emails.log' 2>&1
Running scheduled command: '/usr/local/bin/php-7.3' 'artisan' queue:work --queue='emails,default,b0045ee07a77c41aa18f7c3e8802cb24' --sleep=5 --tries=1 > '/home/webserver/www/htdocs/neurotec/freescout/storage/logs/queue-jobs.log' 2>&1
=> And here it hangs...

I am not sure if this something I didn't understood or the OpenBSD. Any idea how to debug this without error messages?

Thanks!

best wishes
David

[davrot]

Okay, I fell for the Installation guide
https://github.com/freescout-helpdesk/freescout/wiki/Installation-Guide#-permissions-issue

It was written that his might help (under queue:work is not running):
* * * * * php /var/www/html/artisan queue:work --queue='emails,default' --sleep=5 --tries=1 >> /dev/null 2>&1
However, for me
* * * * * php /var/www/html/artisan queue:work --queue='emails,default' --once --tries=1 >> /dev/null 2>&1
would have been correct.

[fvdm]

I have the same problem. The scheduler gets stuck causing nothing to happen anymore. Including dispatching user notifications. Only after about 12-24 hours it recovers itself and continues running the queue. This is a bug.

On our organization FreeScout we bypass the scheduler loop by changing the cronjobs to:

*/5 * * * * php artisan cache:clear
1,6,11,16,21,26,31,36,41,46,51,56 * * * * php artisan queue:work --queue='emails,default' --once --tries=1
3,8,13,18,23,28,33,38,43,48,53,58 * * * * php artisan schedule:run

This way everything runs every 5 minutes. The clear:cache causes a stuck scheduler to reset, according to its code comment. In practice this sometimes doesn't help. Therefor the queue:work --once to force a queue run before the scheduler becomes active.

I think the whole scheduler needs to be redone. It is unclear to me why the --sleep and those shell_exec lines in kernel.php exist. Seems to me like a dirty hack for something else that is not running well. A web app should not need direct shell access, let alone messing with processes.