Horizon: MANUALLY retrying jobs (in Horizon) with time based limits will always fail with `Illuminate\Queue\MaxAttemptsExceededException`

[satyago]

• Horizon Version: 3.4.0
• Laravel Version: 5.8.35

Description:

A retry on jobs that use time based limits will always fail, when the time limit has passed.
The (assumed) reason being that the attempts will get reset during a retry, but the timeoutAt is still serialized on the queued job and taken into account at time of execution.

[driesvints]

A retry on jobs that use time based limits will always fail, when the time limit has passed.

Isn't this the expected behavior?

[satyago]

@driesvints thank you for reviewing this and sorry for the hassle.

Unfortunately I don't think this is intended behavior, or it shouldn't be.
The problem to me is that there seems no option available to retry a failed job that has time limit constraints, while there is an option to retry jobs with an "attempt" count based limit.

With that, some jobs you'll just never be able to retry even though the option is given. And unless you're aware that the limitations of time based constraints carry over to retries, you're lost.

It's an opinionated report of course, I see that.
But if the retry option is given in Horizon, IMHO it should work no matter the implementation of the limit constraint?

I think the job should be treated as if it has just been added to the database and re-evaluate the retryUntil(). That seems to be the thought behind resetting attempt as seen in my reference above, to me at least.

[driesvints]

I now see what you mean. That indeed sounds like a bug. Thanks for reporting.

[Fossil01]

This sounds the same as my issue: https://github.com/laravel/framework/issues/30264

[Fossil01]

So yeah, I'm now seeing that not only scheduled jobs fail but also jobs dispatched from a controller.

The job actually succeeds (and I am getting my email from that Job), Horizon (and queue:work too) show them as OK, then after a minute that same job is now marked as failed.

Since the same behavior exists when using artisan queue:work I suspect the issue might not be in Horizon itself?

[satyago]

@Fossil01 I've read through the issue you referenced.
It's the same error we're getting, but to me the two are different in cause and shouldn't be connected. I'll comment on that issue with more.

This bug here is about Horizon not allowing the manual retry of a failed record, as it doesn't reset previous state. That's not what you're trying to solve.

[Fossil01]

@satyago I see now. Still might be related though, but I'll keep my findings in the other issue ;-)

[themsaid]

@satyago I don't believe this is a bug, when you say retryUntil you set a specific timestamp, it's not like saying retry for 15 seconds.

Closing since it's not actually a bug.

[satyago]

@themsaid I'm afraid you misunderstood.
Horizon allows you to retry jobs manually and it does so by resetting the jobs retry count.
If you want to retry a job with a time constraint, you can't, because Horizon does not reset the "specific timestamp" and thus always fails.

[themsaid]

@satyago I understood correct but I believed since you set a specific retry until date, retries after this date should fail because you said don't retry after that date.

But after checking with Taylor, we decided to allow this so I opened this PR: https://github.com/laravel/horizon/pull/736

[satyago]

@themsaid okay, I understand your reasoning now, but I'm glad you and @taylorotwell seem to agree that a manual reset should in fact result in a positive outcome.
Thank you to you both, belated Merry Christmas and a happy new year to you and your families !