Kombu: Azure service bus queue not working

[ronnie-webb]

Im setting up celery to use Azure service bus as a broker and azure storage as a results backend. I was using Rabbitmq and redis the typical installation when my application ran into some issues. I needed a more robust backend and broker so i chose azure services. I have an issue with a particular queue which doesn't receive messages which intern the results backend does not does not have the message. Can I get some guidance on maybe the azure setup.. should i be using peek and lock?

my celery config

  "task_acks_late": True,
    "task_queue_ha_policy": "all",
    "broker_url": CELERY_BROKER_URL,
    "result_backend": CELERY_RESULT_BACKEND,
    "enable_utc": True,
    "soft_time_limit": 5 * 60,
    "task_track_started": True,
    "task_reject_on_worker_lost": True,
    "worker_prefetch_multiplier": 1,
    "worker_max_tasks_per_child": tryGetEnvVar("CELERY_MAX_TASKS", 100),
    "result_backend_always_retry": True,
    "result_backend_max_retries": 10,

@celery.task(
    bind=True,
    name="auto_label",
    base=BaseTask,
    soft_time_limit=30 * 60,
    acks_late=False,
    acks_on_failure_or_timeout=False,
    autoretry_for=(),
)

/usr/local/bin/python /usr/local/bin/celery --app=tasks worker --loglevel=info --pool prefork --concurrency 4 --queues

My azure queue config

<Queue><LockDuration>00:00:05</LockDuration><MaxSizeInMegabytes>2048</MaxSizeInMegabytes><RequiresDuplicateDetection>False</RequiresDuplicateDetection><RequiresSession>False</RequiresSession><DefaultMessageTimeToLive>14.02:48:05.4800000</DefaultMessageTimeToLive><AutoDeleteOnIdle>10675199.02:48:05.4775807</AutoDeleteOnIdle><EnableDeadLetteringOnMessageExpiration>True</EnableDeadLetteringOnMessageExpiration><DuplicateDetectionHistoryTimeWindow>00:10:00</DuplicateDetectionHistoryTimeWindow><Path>autolabel_queue</Path><MaxDeliveryCount>2000</MaxDeliveryCount><EnableBatchedOperations>True</EnableBatchedOperations><IsAnonymousAccessible>False</IsAnonymousAccessible><SupportOrdering>True</SupportOrdering><Status>Active</Status><ForwardTo/><ForwardDeadLetteredMessagesTo/><EnablePartitioning>False</EnablePartitioning><UserMetadata/><EnableExpress>False</EnableExpress><IsReadOnly>False</IsReadOnly></Queue>

[matusvalo]

Can you verify this issue against master? #1284 could fix your issue.

[thedrow]

@ronnie-webb Ping?

[matusvalo]

@thedrow if you agree, I would close this Issue due to inactivity of reporter and potential fix of Azure Broker. If it is still valid, it can be reopened anytime (or new one spawned).

[thedrow]

Yeah, let's do that.