Sidekiq: Long running jobs

[kapso]

Is there a timeout for Sidekiq job?

We have some long running jobs (pull data from an external source via API) -- we loop through & batch until we run into rate limit and then restart that job later based on delay time in the header.

Also, is it advisable to use Sidekiq for such long-running jobs?

[mikegee]

Is there a timeout for Sidekiq job?

No

then restart that job later based on delay time in the header

Does "restart" mean something like self.class.perform_in(10.minutes, args)? Because rescheduling the job to run later should work well.

is it advisable to use Sidekiq for such long-running jobs?

Sidekiq is designed to run quick jobs. Many people use it for long-running jobs with some success, but troubleshooting problems with long-running jobs is hard.

I suggest breaking down your work into multiple Sidekiq worker classes. Enqueue "children" jobs for each item in an API response, for instance.

[kapso]

@mikegee Thanks for your quick response 👌