Alertmanager: repeat_interval set 99999h, but in fact it send alert every 120h

[FirstEncounter]

What did you do?

I don't want it send alert repeatly , so I set "repeat_interval: 99999h", but it will send alert again after 120h.

What did you expect to see?

I want to know whether there is a maximum value of repeat_interval or not.

What did you see instead? Under which circumstances?

Environment

• System information:

  insert output of uname -srm here

• Alertmanager version:

  v0.15.2

• Prometheus version:

  v2.4.3

• Alertmanager configuration file:

global:
  resolve_timeout: 5m

route:
  group_wait: 30s
  group_interval: 5m
  repeat_interval: '8h'
  receiver: 'wechat_platform' 
  group_by: [alertname, instance]
  routes:
  - match:
      source: platform
    receiver: wechat_platform
    repeat_interval: 99999h
  - match:
      source: business
    receiver: wechat_business
    repeat_interval: 99999h
    continue: true
  - match:
      telephone: true
    receiver: webhook
    repeat_interval: 99999h

templates:
  - '/etc/alertmanager/template/*.tmpl'

receivers:
  - name: 'wechat_platform'
    wechat_configs:
    - send_resolved: false
      corp_id: xx
      to_party: xx
      agent_id: xx
      api_secret: xx
      message: '{{ template "platform.message" . }}'

  - name: 'wechat_business'
    wechat_configs:
    - send_resolved: false
      corp_id: xx
      to_party: xx
      agent_id: xx
      api_secret: xx
      message: '{{ template "business.message" . }}'

  - name: 'webhook'
    webhook_configs:
    - send_resolved: false
      url: 'xxx'

• Prometheus configuration file:

insert configuration here (if relevant to the issue)

• Logs:

insert Prometheus and Alertmanager logs relevant to the issue here

[simonpasquier]

I suspect that you need to increase --data.retention as its default value is 120h. That being said, I don't think it is documented as it should be.

[yeya24]

Can we add an option or config for never repeating alerts?

[FirstEncounter]

I suspect that you need to increase --data.retention as its default value is 120h. That being said, I don't think it is documented as it should be.

Thank you very much! It is indeed caused by this problem

[simonpasquier]

Can we add an option or config for never repeating alerts?

I wouldn't add more options as the configuration is already complex. Also it wouldn't solve the problem as the notification logs would be deleted eventually and AlertManager would forget what had been notified. Maybe we could log a warning whenever we detect that a repeat_interval is greater than the data retention period.