Telegraf: Postfix plugin does not correctly count deferred messages when hash_queue_depth is greater than 1

[couloum]

Relevant telegraf.conf:

System info:

Telegraf v1.6.2 from official influxdata repo.
Ubuntu 16.04

Steps to reproduce:

1. Install postfix
2. Add parameter hash_queue_depth = 3 in file /etc/postfix/main.cf
3. Restart postfix
4. Send 100 emails to that will be deferred to generate activity in deferred queue.
5. Drop all emails in deferred queue with postsuper -d ALL deferred
6. Install telegraf
7. Create config file /etc/telegraf/telegraf.d/input-postfix.conf with following content:

[[inputs.postfix]]
  ## Postfix queue directory. If not provided, telegraf will try to use
  ## 'postconf -h queue_directory' to determine it.
  queue_directory = "/var/spool/postfix"

1. Execute command telegraf --config /etc/telegraf/telegraf.d/input-postfix.conf --input-filter postfix --test

Expected behavior:

For deferred queue, length should be 0:

> postfix_queue,host=vceu1mtao02s.internal.vadesecure.com,queue=deferred length=0i,size=0i,age=0i 1525987766000000000

Actual behavior:

Deferred queue length is not 0. It is equal to the number of folders inside each sub-folder of /var/spool/postfix/deferred/.

Additional info:

Example on my server:

$ postqueue -p
-Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient-------
03500408FC52     6230 Wed May  9 18:16:30  *****@*****.com
                 (connect to ****[*****]:25: Connection refused)
                                         ****@****

-- 7 Kbytes in 1 Request.

$ find  /var/spool/postfix/deferred -type f | wc -l
1

$ tree --prune /var/spool/postfix/deferred
/var/spool/postfix/deferred
└── 0
    └── 3
        └── 5
            └── 03500408FC52

3 directories, 1 file

$ find /var/spool/postfix/deferred -maxdepth 0 -type d | wc -l
1

$ find /var/spool/postfix/deferred -maxdepth 1 -type d | wc -l
17

$ find /var/spool/postfix/deferred -maxdepth 2 -type d | wc -l
157

# 157 = 1 (/var/spool/postfix/deferred) + 16 (0-9, A-F) + 140

$ find /var/spool/postfix/deferred -maxdepth 3 -type d | wc -l
378

$ /usr/bin/telegraf --config /etc/telegraf/telegraf.d/input-postfix.conf --input-filter postfix --test
* Plugin: inputs.postfix, Collection 1
> postfix_queue,host=vceu1mtao02s.internal.vadesecure.com,queue=active length=0i,size=0i,age=0i 1525988325000000000
> postfix_queue,host=vceu1mtao02s.internal.vadesecure.com,queue=hold length=0i,size=0i,age=0i 1525988325000000000
> postfix_queue,host=vceu1mtao02s.internal.vadesecure.com,queue=incoming length=0i,size=0i,age=0i 1525988325000000000
> postfix_queue,host=vceu1mtao02s.internal.vadesecure.com,queue=maildrop length=0i,size=0i,age=0i 1525988325000000000
> postfix_queue,host=vceu1mtao02s.internal.vadesecure.com,queue=deferred length=140i,size=2829i,age=3601i 1525988325000000000

# We see that deferred queue contains 140 items while it should contains 1 item.

[danielnelson]

@phemmer Seems like the fix is to walk this directory recursively counting only files?

[phemmer]

I think that would work fine yes.
We should probably adjust the plugin to do the same for the other directories as well. It's possible for for any queue to use the nested hash scheme. Just not the default, and very rare to do so (though I expected we'd see that case before we saw this, so that goes to show what I know :-P).

[couloum]

Hello,

Do you have any news about this issue?

Thanks

[danielnelson]

@couloum No news yet, sorry. We will link a pull request when we have a fix in progress.

[phemmer]

pull request link: #4333 :-)