Hangfire: Is there a way to cancel all pending jobs in a named queue?

[JustMaier]

Is there a way to cancel all pending jobs in a named queue?

[JustMaier]

So I don't know if this is the best way to do this, but I found two ways of doing this.

The first one is pretty slow:

``` c#
var monitoringApi = JobStorage.Current.GetMonitoringApi();
var queues = monitoringApi.Queues();
var toDelete = new List();
for (var i = 0; i < Math.Ceiling((double)(queue.Length/1000)) ; i++) {
monitoringApi.EnqueuedJobs(queue.Name, 1000 * i, 1000).ForEach(x => toDelete.Add(x.Key));
}

foreach (var jobId in toDelete) {
BackgroundJob.Delete(jobId);
}

This one is much faster, but feels kind of hacky and doesn't leave a job as deleted.

``` c#
db.Database.ExecuteSqlCommand(string.Format(@"DELETE j FROM HangFire.Job AS j
    LEFT JOIN HangFire.JobQueue AS jq ON jq.JobId=j.Id
    WHERE jq.[Queue]='{0}' AND (StateName='Enqueued');
    DELETE FROM HangFire.JobQueue WHERE [Queue]='{0}';", queue.Name));

[danodonovan]

Great tip, thanks. To save future googlers some time,

var monitor = JobStorage.Current.GetMonitoringApi();
var queues = monitor.Queues();

[JustMaier]

@danodonovan Updated my little example to include that. Sorry if I made you search for that...

[JamieG]

Throwing it all together

    public static class HangfireExtensions
    {
        public static void PurgeJobs(this IMonitoringApi monitor)
        {
            var toDelete = new List<string>();

            foreach (QueueWithTopEnqueuedJobsDto queue in monitor.Queues())
            {
                for (var i = 0; i < Math.Ceiling(queue.Length / 1000d); i++)
                {
                    monitor.EnqueuedJobs(queue.Name, 1000 * i, 1000)
                        .ForEach(x => toDelete.Add(x.Key));
                }
            }

            foreach (var jobId in toDelete)
            {
                BackgroundJob.Delete(jobId);
            }
        }
    }

Called with:

JobStorage.Current?.GetMonitoringApi()?.PurgeJobs();

[markalanevans]

@JustMaier Is this still the best way to do this?

[JustMaier]

@markalanevans as far as I know. I never got a different response or saw anything different...

[markalanevans]

@JustMaier So this looks like it would delete all jobs in all queues. Ideally we could specify a queue name right?

[JustMaier]

You can modify @JamieG's code to purge a queue by name. Just FYI - from my experience, the SQL query is actually way more performant if you have more than 100 or so items in the queue.

[markalanevans]

Right. Below is what i ended up with. It is very slow, and while it does work, you do have to have a worker running and monitoring each queue otherwise the jobs state changes to deleted but they don't actually get moved into the "Deleted" bucket in the Dashboard.

I wanted to use the methods because otherwise when we switch to redis it won't work. @odinserj is there a faster way to do this while still using the methods provided?

public class JobService : IJobService
    {
        public void PurgeAllQueues()
        {
            var monitor = JobStorage.Current.GetMonitoringApi();
            foreach (QueueWithTopEnqueuedJobsDto queue in monitor.Queues())
            {
                PurgeQueue(queue.Name);
            }
        }

        public void PurgeQueue(string queueName)
        {
            var toDelete = new List<string>();
            var monitor = JobStorage.Current.GetMonitoringApi();

            var queue = monitor.Queues().First(x => x.Name == queueName);
            for (var i = 0; i < Math.Ceiling(queue.Length / 1000d); i++)
            {
                monitor.EnqueuedJobs(queue.Name, 1000 * i, 1000)
                    .ForEach(x => toDelete.Add(x.Key));
            }
            foreach (var jobId in toDelete)
            {
                BackgroundJob.Delete(jobId);
            }
        }
    }

[AhmedMozaly]

I thought the proposed code above will reset the recurring jobs too, but it didn't
So, I wrote my own method to reset recurring jobs at app Start, this is

public void ResetRecurringJobs(List<string> newJobIds)
{
    using (var connection = JobStorage.Current.GetConnection())
    {
        var setKey = "recurring-jobs";
        var savedJobIds = connection.GetAllItemsFromSet(setKey);
        var missingJobsIds = savedJobIds.Except(newJobIds).ToList();

        foreach (var jobId in missingJobsIds)
        {
            RecurringJob.RemoveIfExists(jobId);
        }
    }            
}

[r4hulp]

is JobStorage.Current.GetMonitoringApi(); designed for designing custom job monitoring portals? I am thinking of having my own portal for the same. Is this the right implementation to go with or there is any other alternate method that serves me data in better way?

[alexanderdevoid]

Can someone clarify the 'slowness' part?

I would imagine the cancellation and thus deletion has to go through the cancellation tokens. So if you are using cancellation tokens on all your hangfire calls:

BackgroundJob.Enqueue(() => ProcessRequestWorker(JobCancellationToken.Null, brokerRequest));

Is it faster or it doesn't make any difference?

And another thing, I suppose the polling intervals that the job servers use play a part on this process as well. If you use longer intervals for polls then it would obviously take longer to delete the jobs.

[markalanevans]

@odinserj Thoughts on this? It would be really nice to have a button next to any queue with jobs in it and the ability to delete all jobs.

[zerokewl88]

Just using a direct SQL would be fastest wouldn't it ?

DELETE FROM [DatabaseName].[HangFire].[JobQueue]

[burningice2866]

@zerokewl88 if you're using sql-storage, yes. Not everyone is using sql storage :)

[burningice2866]

@rahulrulez if what you want to build exposes the same data as the existing dashboard, then you can use the MonitoringApi for sure. The existing dashboard solely relies on this.

[elmer0927]

DELETE FROM [DatabaseName].[HangFire].[JobQueue] WHERE QUEUE = 'your queue name'
DELETE FROM [DatabaseName].[HangFire].Job
where statename <> 'Enqueued'

[markalanevans]

@odinserj Any feedback here? What do you suggest for clearing a queue.
BackgroundJob.EmptyQueue("queue-name") and make it fast?

[Jon-Murray]

So i figured i'd contribute here as this was still somewhat of a pain point for me. In my situation we have a dev box where we don't care about the state of the redis database, we just want to wipe queued and recurring jobs so we don't get spammed a million times. We are using hangfire.aspnetcore. Simply doing:

        await db.KeyDeleteAsync("myPrefix:Hangfireschedule");
        await db.KeyDeleteAsync("myPrefix:Hangfirerecurring-jobs");

Did the trick for us. When a new job gets added, or recurring job scheduled, the zset is recreated properly and there are no issues. Hope this helps someone. (do not use this in prod).

[quantum-fulltilt]

This works for my needs

var mon = JobStorage.Current.GetMonitoringApi();
mon.EnqueuedJobs("printer",0,99999999).ForEach(x => {
    BackgroundJob.Delete(x.Key);
});

searched for a minute but no solution worked the way i needed it. That there killed all fire and forget jobs which is what I needed.

[elgerm]

Addition on Redis, if you are just developing and getting errors because methods don't exist anymore deleting all hangfire entries in with redis-cli works too:
redis-cli KEYS "{hangfire*" | xargs redis-cli DEL

[simeyla]

One way to magically watch your queued jobs disappear is to add a [LatencyTimeout] attribute to the job. In this case jobs queued longer than 2 minutes will get deleted and neatly marked with Background job has exceeded latency timeout of 120 second(s)

    [Hangfire.LatencyTimeout(60 * 2)]

In my case I had a named queue that wasn't active, so a bunch of jobs accumulated - for days! Adding this attribute made them all just disappear before my eyes in the console.

For 'background jobs' that aren't transactional in nature it looks like it's important to add this LatencyTimeout to make sure the job doesn't run more than it should.

Yes you can add this to an existing job.

[Recodify]

DELETE FROM [DatabaseName].[HangFire].[JobQueue] WHERE QUEUE = 'your queue name'
DELETE FROM [DatabaseName].[HangFire].Job
where statename <> 'Enqueued'

where statename = 'Enqueued'?

[UnwashedAnd]

So I think you need to delete the Jobs, and then the JobQueues - I can't find this in the documentation but it works for me. First find what your queue is called by exploring the Hangfire.Job table, then run this pair of queries after adding your queue name where indicated:

DECLARE @Queue varchar(40) = 'j_queue_medium' -- CHANGE THIS QUEUE NAME AS REQUIRED

DELETE HangFire.Job
FROM HangFire.Job INNER JOIN
HangFire.JobQueue ON HangFire.Job.Id = HangFire.JobQueue.JobId
WHERE (HangFire.JobQueue.Queue = @Queue)

DELETE FROM [HangFire].[JobQueue] WHERE QUEUE = @Queue