Rspamd: [BUG] after update to rspamd2 - no update of user settings and multimaps from remote server (http/s)

Created on 23 Oct 2019  路  8Comments  路  Source: rspamd/rspamd

Describe the bug
multimap and user settings are just read once from remote webserver on rspamd startup after update to rspamd2.

Steps to Reproduce

  1. /etc/rspamd/rspamd.conf.override

settings = "https://portal.mydomain.com/wl-bl.txt

/etc/rspamd/local.d/multimap.conf

RCPT_WHITELIST { 
  type = "rcpt"; 
  map = "https://portal.mydomain.com/wl-rcpt.txt"; 
  prefilter = true;
  action = "accept";
  map_watch_interval = 120;
}

Reload rspamd.

2019-10-19 19:05:48 #12331(main) <przmwy>; map; rspamd_map_add: added map https://portal.mydomain.com/wl-rcpt.txt
2019-10-19 19:05:48 #12331(main) <6porf9>; map; rspamd_map_add: added map https://portal.mydomain.com/wl-bl.txt
2019-10-19 19:05:49 #12331(main) <6porf9>; map; rspamd_map_read_http_cached_file: read cached data for https://portal.mydomain.com/wl-bl.txt from /var/lib/rspamd/be41523e2fbce26cfd3a5bc53460c4b6c5436f6f.map, 1052 bytes
2019-10-19 19:05:49 #12331(main) <przmwy>; map; rspamd_map_read_http_cached_file: read cached data for https://portal.mydomain.com/wl-rcpt.txt from /var/lib/rspamd/8ddc45c1bc737b1319e046ecfa26ba83a2c88a49.map, 26 bytes
2019-10-19 19:30:17 #15652(main) <przmwy>; map; rspamd_map_add: added map https://portal.mydomain.com/wl-rcpt.txt
2019-10-19 19:30:17 #15652(main) <6porf9>; map; rspamd_map_add: added map https://portal.mydomain.com/wl-bl.txt
2019-10-19 19:30:17 #15652(main) <6porf9>; map; rspamd_map_read_http_cached_file: read cached data for https://portal.mydomain.com/wl-bl.txt from /var/lib/rspamd/be41523e2fbce26cfd3a5bc53460c4b6c5436f6f.map, 1052 bytes
2019-10-19 19:30:17 #15652(main) <przmwy>; map; rspamd_map_read_http_cached_file: read cached data for https://portal.mydomain.com/wl-rcpt.txt from /var/lib/rspamd/8ddc45c1bc737b1319e046ecfa26ba83a2c88a49.map, 26 bytes
2019-10-19 19:30:17 #15675(controller) <przmwy>; map; http_map_finish: need to reread map from https://portal.mydomain.com/wl-rcpt.txt

Expected behavior
on given intervall or at least at default interval there should be a HEAD against remote webserver to check for new version / change of file.

Remote server does not even log a connection from rspamd

Versions

Rspamd daemon version 2.0
Debian 10 buster, latest updates
Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz and several others (issues is on all systems)

Additional Information

Works until update to rspamd 2.

bug

Most helpful comment

2.1-1~buster fixed the issue for me.

All 8 comments

map_watch_interval = 120; is meaningless there. Also, Rspamd uses Expires header. Experimental packages are more verbose about it.

No Expires header are in place. No expires-modules are enabled nor have ever been.

apache access.log from portal, that shows, how the update stops the requests, step by step.
No new logs until now, all systems got restarted many times. No config changes since weeks.

192.168.0.1 - - [11/Oct/2019:16:00:49 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:16:01:23 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:16:03:08 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:16:04:26 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:16:05:17 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:16:07:37 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:16:08:46 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:16:09:42 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:16:10:08 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:16:11:30 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:16:13:00 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:16:13:50 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:16:15:15 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:16:16:52 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:16:18:22 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:16:18:59 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:16:20:33 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:16:21:23 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:16:23:33 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:16:24:56 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:16:25:19 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:16:27:19 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:16:29:15 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:16:29:55 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:16:32:12 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:16:32:48 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:16:34:01 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:16:34:31 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:16:35:11 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:16:35:14 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:16:38:59 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:16:39:05 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:16:41:57 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:16:42:01 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:16:42:13 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:16:43:52 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:16:45:22 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:16:47:16 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:16:47:22 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:16:48:20 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:16:49:52 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:16:51:47 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:16:52:34 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:16:53:36 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:16:55:06 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:16:55:18 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:16:57:57 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:16:58:35 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:16:59:01 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:16:59:20 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:17:00:20 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:17:00:23 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:17:03:17 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:17:04:39 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:17:04:51 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:17:05:29 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:17:07:11 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:17:08:00 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:17:09:38 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:17:11:28 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:17:13:17 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:17:13:17 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:17:15:29 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:17:15:53 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:17:15:58 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:17:19:40 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:17:20:12 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:17:20:57 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:17:21:52 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:17:24:56 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:17:25:52 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:17:27:10 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:17:27:53 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:17:28:12 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:17:29:17 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:17:31:38 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:17:33:31 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:17:34:10 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:17:34:19 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:17:34:29 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:17:36:16 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:17:40:50 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:17:40:59 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:17:42:58 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:17:43:58 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:17:44:03 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:17:44:36 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:17:46:12 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:17:48:39 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:17:50:53 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:17:52:28 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:17:53:08 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:17:53:29 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:17:55:51 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:17:55:57 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:17:56:49 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:18:01:38 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:18:01:42 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:18:02:18 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:18:02:19 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:18:03:41 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:18:05:44 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:18:07:01 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:18:10:26 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:18:11:13 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:18:11:23 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:18:11:36 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:18:12:26 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:18:13:00 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:18:18:11 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:18:18:48 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:18:20:18 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:18:20:58 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:18:21:12 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:18:22:53 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:18:25:29 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:18:26:10 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:18:26:51 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:18:28:37 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:18:28:57 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:18:30:32 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:18:31:00 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:18:34:26 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:18:34:46 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:18:35:54 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:18:36:00 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:18:37:12 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:18:40:03 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:18:40:28 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:18:41:00 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:18:42:18 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:18:42:37 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:18:45:22 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:18:46:57 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:18:48:08 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:18:50:00 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:18:51:32 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:18:51:46 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:18:53:06 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:18:54:26 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:18:55:15 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:18:57:58 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:18:59:33 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:18:59:48 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:19:00:00 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:19:03:57 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:19:04:03 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:19:05:09 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:19:05:29 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:19:06:18 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:19:07:39 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:19:12:27 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:19:12:40 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:19:12:47 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:19:13:01 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:19:13:23 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:19:15:16 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:19:19:36 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:19:20:04 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:19:20:15 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:19:20:39 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.1 - - [11/Oct/2019:19:22:11 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.2 - - [11/Oct/2019:19:23:21 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:19:26:06 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-1.9.4"
192.168.0.3 - - [11/Oct/2019:19:26:11 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-2.0"
192.168.0.1 - - [11/Oct/2019:19:26:42 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-2.0"
192.168.0.1 - - [11/Oct/2019:19:26:42 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-2.0"
192.168.0.2 - - [11/Oct/2019:19:27:24 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-2.0"
192.168.0.1 - - [11/Oct/2019:19:32:07 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-2.0"
192.168.0.1 - - [11/Oct/2019:19:34:48 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-2.0"
192.168.0.1 - - [11/Oct/2019:19:39:13 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-2.0"
192.168.0.1 - - [11/Oct/2019:19:41:12 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-2.0"
192.168.0.1 - - [11/Oct/2019:19:48:27 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-2.0"
192.168.0.1 - - [11/Oct/2019:19:48:34 +0200] "HEAD /wl-rcpt.txt HTTP/1.1" 304 4396 "-" "rspamd-2.0"
192.168.0.1 - - [11/Oct/2019:19:54:17 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-2.0"
192.168.0.1 - - [11/Oct/2019:20:01:56 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-2.0"
192.168.0.1 - - [11/Oct/2019:20:09:19 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-2.0"
192.168.0.1 - - [11/Oct/2019:20:16:03 +0200] "HEAD /wl-bl.txt HTTP/1.1" 304 4397 "-" "rspamd-2.0"

I'am seeing the same behaviour as @nafetsreuab using "2.0-1~bionic". I have four maps configured at the moment - two of them got queried once on restart. No queries after that, although there was a "next check" scheduled according to rspamd.log.

Same issue here with 2.0-1~buster.

http multimaps are checked (HEAD) and sometimes loaded (GET) on restart, but it is not checked regularly. No other entries in remote Apache logfiles.

I'am seeing the same behaviour as @nafetsreuab using "2.0-1~bionic". I have four maps configured at the moment - two of them got queried once on restart. No queries after that, although there was a "next check" scheduled according to rspamd.log.

I have to correct myself a little...
It's actually just three maps - and the behaviour seems kinda random to me.

I cleared out /var/lib/rspamd/ and flushed redis as well (redis-cli FLUSHDB;redis-cli FLUSHALL) on each rspamd restart.
No configuration changes have been made inbetween restarts. map_watch_interval is set to 10s (for debugging purposes only).
The queried server sends no expires header, only an md5 hashed Etag.

2.0.1

Example 1

All three maps got queried on restart. Two got requeried twice and one got requeried once. No further requests after the last one.

rspamd.log (logdate: Mon Oct 28 13:57:01 CET 2019)

2019-10-28 13:49:55 #22542(main) <8pumrj>; map; rspamd_map_add: added map https://domain.tld/?scope=mail&option=accept_bl_files
2019-10-28 13:49:55 #22542(main) <4m4m99>; map; rspamd_map_add: added map https://domain.tld/?scope=mail&option=reject_bl_files
2019-10-28 13:49:55 #22542(main) <6susux>; map; rspamd_map_add: added map https://domain.tld/?scope=mail&option=disabledeliver
2019-10-28 13:49:55 #22542(main) <umpkkh>; lua; lua_maps.lua:121: reuse url for https://domain.tld/?scope=mail&option=reject_bl_files(hash)
2019-10-28 13:50:00 #22556(controller) <4m4m99>; map; http_map_finish: need to reread map from https://domain.tld/?scope=mail&option=reject_bl_files
2019-10-28 13:50:01 #22556(controller) <6susux>; map; http_map_finish: need to reread map from https://domain.tld/?scope=mail&option=disabledeliver
2019-10-28 13:50:01 #22556(controller) <8pumrj>; map; http_map_finish: need to reread map from https://domain.tld/?scope=mail&option=accept_bl_files
2019-10-28 13:50:01 #22556(controller) <4m4m99>; map; http_map_finish: https://domain.tld/?scope=mail&option=reject_bl_files(127.0.0.1:443): read map data 25 bytes, next check at Mon, 28 Oct 2019 12:50:11 GMT
2019-10-28 13:50:01 #22556(controller) <4m4m99>; map; rspamd_map_save_http_cached_file: saved data from https://domain.tld/?scope=mail&option=reject_bl_files in /var/lib/rspamd/7ae9f5ff8697f2896c4ecbed42799aaf69dede1a.map, 25 bytes
2019-10-28 13:50:01 #22559(normal) <4m4m99>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=reject_bl_files: read map data cached 25 bytes
2019-10-28 13:50:01 #22555(rspamd_proxy) <4m4m99>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=reject_bl_files: read map data cached 25 bytes
2019-10-28 13:50:01 #22560(normal) <4m4m99>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=reject_bl_files: read map data cached 25 bytes
2019-10-28 13:50:01 #22558(normal) <4m4m99>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=reject_bl_files: read map data cached 25 bytes
2019-10-28 13:50:01 #22556(controller) <6susux>; map; http_map_finish: https://domain.tld/?scope=mail&option=disabledeliver(127.0.0.1:443): read map data 1 bytes, next check at Mon, 28 Oct 2019 12:50:11 GMT
2019-10-28 13:50:01 #22556(controller) <6susux>; map; rspamd_map_save_http_cached_file: saved data from https://domain.tld/?scope=mail&option=disabledeliver in /var/lib/rspamd/de4e3b5f4b09073e9c28da48244b447fc577493b.map, 1 bytes
2019-10-28 13:50:01 #22556(controller) <8pumrj>; map; http_map_finish: https://domain.tld/?scope=mail&option=accept_bl_files(127.0.0.1:443): read map data 30 bytes, next check at Mon, 28 Oct 2019 12:50:11 GMT
2019-10-28 13:50:01 #22556(controller) <8pumrj>; map; rspamd_map_save_http_cached_file: saved data from https://domain.tld/?scope=mail&option=accept_bl_files in /var/lib/rspamd/a7cd459273cca0990234c33251c3a20ea991e95b.map, 30 bytes
2019-10-28 13:50:01 #22557(normal) <4m4m99>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=reject_bl_files: read map data cached 25 bytes
2019-10-28 13:50:01 #22555(rspamd_proxy) <8pumrj>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=accept_bl_files: read map data cached 30 bytes
2019-10-28 13:50:01 #22559(normal) <8pumrj>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=accept_bl_files: read map data cached 30 bytes
2019-10-28 13:50:01 #22558(normal) <8pumrj>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=accept_bl_files: read map data cached 30 bytes
2019-10-28 13:50:01 #22557(normal) <8pumrj>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=accept_bl_files: read map data cached 30 bytes
2019-10-28 13:50:01 #22555(rspamd_proxy) <6susux>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=disabledeliver: read map data cached 1 bytes
2019-10-28 13:50:01 #22560(normal) <8pumrj>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=accept_bl_files: read map data cached 30 bytes
2019-10-28 13:50:01 #22557(normal) <6susux>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=disabledeliver: read map data cached 1 bytes
2019-10-28 13:50:01 #22558(normal) <6susux>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=disabledeliver: read map data cached 1 bytes
2019-10-28 13:50:01 #22560(normal) <6susux>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=disabledeliver: read map data cached 1 bytes
2019-10-28 13:50:01 #22559(normal) <6susux>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=disabledeliver: read map data cached 1 bytes
2019-10-28 13:50:12 #22556(controller) <8pumrj>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 12:50:22 GMT
2019-10-28 13:50:18 #22556(controller) <6susux>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 12:50:28 GMT
2019-10-28 13:50:18 #22556(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 12:50:28 GMT
2019-10-28 13:50:30 #22556(controller) <6susux>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 12:50:40 GMT
2019-10-28 13:50:36 #22556(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 12:50:46 GMT
2019-10-28 13:50:50 #22556(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 12:51:00 GMT

apiserver.log (logdate: Mon Oct 28 13:56:41 CET 2019)

127.0.0.1 - - [28/Oct/2019:13:50:00 +0200] "HEAD /scope=mail&option=reject_bl_files" 200 25 - "rspamd-2.0" 0
127.0.0.1 - - [28/Oct/2019:13:50:00 +0200] "HEAD /scope=mail&option=disabledeliver" 200 1 - "rspamd-2.0" 0
127.0.0.1 - - [28/Oct/2019:13:50:01 +0200] "HEAD /scope=mail&option=accept_bl_files" 200 30 - "rspamd-2.0" 0
127.0.0.1 - - [28/Oct/2019:13:50:01 +0200] "GET /scope=mail&option=reject_bl_files" 200 25 - "rspamd-2.0" 0
127.0.0.1 - - [28/Oct/2019:13:50:01 +0200] "GET /scope=mail&option=disabledeliver" 200 1 - "rspamd-2.0" 0
127.0.0.1 - - [28/Oct/2019:13:50:01 +0200] "GET /scope=mail&option=accept_bl_files" 200 30 - "rspamd-2.0" 0
127.0.0.1 - - [28/Oct/2019:13:50:12 +0200] "HEAD /scope=mail&option=accept_bl_files" 304 0 - "rspamd-2.0" 29e0b386aae908a2b94932927b6398fe
127.0.0.1 - - [28/Oct/2019:13:50:18 +0200] "HEAD /scope=mail&option=disabledeliver" 304 0 - "rspamd-2.0" 68b329da9893e34099c7d8ad5cb9c940
127.0.0.1 - - [28/Oct/2019:13:50:18 +0200] "HEAD /scope=mail&option=reject_bl_files" 304 0 - "rspamd-2.0" 8d3577ecb432bfeda4e263bfeda7fda8
127.0.0.1 - - [28/Oct/2019:13:50:30 +0200] "HEAD /scope=mail&option=disabledeliver" 304 0 - "rspamd-2.0" 68b329da9893e34099c7d8ad5cb9c940
127.0.0.1 - - [28/Oct/2019:13:50:36 +0200] "HEAD /scope=mail&option=reject_bl_files" 304 0 - "rspamd-2.0" 8d3577ecb432bfeda4e263bfeda7fda8
127.0.0.1 - - [28/Oct/2019:13:50:50 +0200] "HEAD /scope=mail&option=reject_bl_files" 304 0 - "rspamd-2.0" 8d3577ecb432bfeda4e263bfeda7fda8

Example 2

Two maps got queried in restart. One gets requeried three times. No further requests after the last one.

rspamd.log (logdate: Mon Oct 28 14:04:41 CET 2019)

2019-10-28 13:58:38 #23384(main) <8pumrj>; map; rspamd_map_add: added map https://domain.tld/?scope=mail&option=accept_bl_files
2019-10-28 13:58:38 #23384(main) <4m4m99>; map; rspamd_map_add: added map https://domain.tld/?scope=mail&option=reject_bl_files
2019-10-28 13:58:38 #23384(main) <6susux>; map; rspamd_map_add: added map https://domain.tld/?scope=mail&option=disabledeliver
2019-10-28 13:58:38 #23384(main) <umpkkh>; lua; lua_maps.lua:121: reuse url for https://domain.tld/?scope=mail&option=reject_bl_files(hash)
2019-10-28 13:58:44 #23386(controller) <6susux>; map; http_map_finish: need to reread map from https://domain.tld/?scope=mail&option=disabledeliver
2019-10-28 13:58:44 #23386(controller) <4m4m99>; map; http_map_finish: need to reread map from https://domain.tld/?scope=mail&option=reject_bl_files
2019-10-28 13:58:44 #23386(controller) <6susux>; map; http_map_finish: https://domain.tld/?scope=mail&option=disabledeliver(127.0.0.1:443): read map data 1 bytes, next check at Mon, 28 Oct 2019 12:58:54 GMT
2019-10-28 13:58:44 #23386(controller) <6susux>; map; rspamd_map_save_http_cached_file: saved data from https://domain.tld/?scope=mail&option=disabledeliver in /var/lib/rspamd/de4e3b5f4b09073e9c28da48244b447fc577493b.map, 1 bytes
2019-10-28 13:58:44 #23386(controller) <4m4m99>; map; http_map_finish: https://domain.tld/?scope=mail&option=reject_bl_files(127.0.0.1:443): read map data 25 bytes, next check at Mon, 28 Oct 2019 12:58:54 GMT
2019-10-28 13:58:44 #23386(controller) <4m4m99>; map; rspamd_map_save_http_cached_file: saved data from https://domain.tld/?scope=mail&option=reject_bl_files in /var/lib/rspamd/7ae9f5ff8697f2896c4ecbed42799aaf69dede1a.map, 25 bytes
2019-10-28 13:58:44 #23387(normal) <4m4m99>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=reject_bl_files: read map data cached 25 bytes
2019-10-28 13:58:44 #23388(normal) <6susux>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=disabledeliver: read map data cached 1 bytes
2019-10-28 13:58:44 #23387(normal) <6susux>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=disabledeliver: read map data cached 1 bytes
2019-10-28 13:58:44 #23390(normal) <6susux>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=disabledeliver: read map data cached 1 bytes
2019-10-28 13:58:44 #23390(normal) <4m4m99>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=reject_bl_files: read map data cached 25 bytes
2019-10-28 13:58:44 #23388(normal) <4m4m99>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=reject_bl_files: read map data cached 25 bytes
2019-10-28 13:58:45 #23385(rspamd_proxy) <6susux>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=disabledeliver: read map data cached 1 bytes
2019-10-28 13:58:45 #23389(normal) <6susux>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=disabledeliver: read map data cached 1 bytes
2019-10-28 13:58:45 #23389(normal) <4m4m99>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=reject_bl_files: read map data cached 25 bytes
2019-10-28 13:58:46 #23385(rspamd_proxy) <4m4m99>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=reject_bl_files: read map data cached 25 bytes
2019-10-28 13:58:57 #23386(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 12:59:07 GMT
2019-10-28 13:59:12 #23386(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 12:59:22 GMT
2019-10-28 13:59:25 #23386(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 12:59:35 GMT

apiserver.log (logdate: Mon Oct 28 14:04:39 CET 2019)

127.0.0.1 - - [28/Oct/2019:13:58:44 +0200] "HEAD /scope=mail&option=disabledeliver" 200 1 - "rspamd-2.0" 0
127.0.0.1 - - [28/Oct/2019:13:58:44 +0200] "HEAD /scope=mail&option=reject_bl_files" 200 25 - "rspamd-2.0" 0
127.0.0.1 - - [28/Oct/2019:13:58:44 +0200] "GET /scope=mail&option=disabledeliver" 200 1 - "rspamd-2.0" 0
127.0.0.1 - - [28/Oct/2019:13:58:44 +0200] "GET /scope=mail&option=reject_bl_files" 200 25 - "rspamd-2.0" 0
127.0.0.1 - - [28/Oct/2019:13:58:57 +0200] "HEAD /scope=mail&option=reject_bl_files" 304 0 - "rspamd-2.0" 8d3577ecb432bfeda4e263bfeda7fda8
127.0.0.1 - - [28/Oct/2019:13:59:12 +0200] "HEAD /scope=mail&option=reject_bl_files" 304 0 - "rspamd-2.0" 8d3577ecb432bfeda4e263bfeda7fda8
127.0.0.1 - - [28/Oct/2019:13:59:25 +0200] "HEAD /scope=mail&option=reject_bl_files" 304 0 - "rspamd-2.0" 8d3577ecb432bfeda4e263bfeda7fda8

1.9.4

Example

All maps got queried initially and are requeried continuously

rspamd.log (logdate: Mon Oct 28 14:09:35 CET 2019)

2019-10-28 14:06:20 #24758(main) <8pumrj>; map; rspamd_map_add: added map https://domain.tld/?scope=mail&option=accept_bl_files
2019-10-28 14:06:20 #24758(main) <4m4m99>; map; rspamd_map_add: added map https://domain.tld/?scope=mail&option=reject_bl_files
2019-10-28 14:06:20 #24758(main) <6susux>; map; rspamd_map_add: added map https://domain.tld/?scope=mail&option=disabledeliver
2019-10-28 14:06:20 #24758(main) <br1rn9>; lua; multimap.lua:1013: reuse url for DOMAIN_REJECTS_BLACKLISTED_FILES: "https://domain.tld/?scope=mail&option=reject_bl_files"
2019-10-28 14:06:25 #24763(controller) <4m4m99>; map; http_map_finish: https://domain.tld/?scope=mail&option=reject_bl_files(127.0.0.1:443): read map data 25 bytes, next check at Mon, 28 Oct 2019 13:06:35 GMT
2019-10-28 14:06:25 #24763(controller) <4m4m99>; map; rspamd_map_save_http_cached_file: saved data from https://domain.tld/?scope=mail&option=reject_bl_files in /var/lib/rspamd/7ae9f5ff8697f2896c4ecbed42799aaf69dede1a.map, 25 bytes
2019-10-28 14:06:25 #24763(controller) <8pumrj>; map; http_map_finish: https://domain.tld/?scope=mail&option=accept_bl_files(127.0.0.1:443): read map data 30 bytes, next check at Mon, 28 Oct 2019 13:06:35 GMT
2019-10-28 14:06:25 #24763(controller) <8pumrj>; map; rspamd_map_save_http_cached_file: saved data from https://domain.tld/?scope=mail&option=accept_bl_files in /var/lib/rspamd/a7cd459273cca0990234c33251c3a20ea991e95b.map, 30 bytes
2019-10-28 14:06:25 #24763(controller) <6susux>; map; http_map_finish: https://domain.tld/?scope=mail&option=disabledeliver(127.0.0.1:443): read map data 1 bytes, next check at Mon, 28 Oct 2019 13:06:35 GMT
2019-10-28 14:06:25 #24763(controller) <6susux>; map; rspamd_map_save_http_cached_file: saved data from https://domain.tld/?scope=mail&option=disabledeliver in /var/lib/rspamd/de4e3b5f4b09073e9c28da48244b447fc577493b.map, 1 bytes
2019-10-28 14:06:26 #24765(normal) <4m4m99>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=reject_bl_files: read map data cached 25 bytes
2019-10-28 14:06:26 #24765(normal) <6susux>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=disabledeliver: read map data cached 1 bytes
2019-10-28 14:06:26 #24762(rspamd_proxy) <8pumrj>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=accept_bl_files: read map data cached 30 bytes
2019-10-28 14:06:26 #24764(normal) <6susux>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=disabledeliver: read map data cached 1 bytes
2019-10-28 14:06:27 #24764(normal) <4m4m99>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=reject_bl_files: read map data cached 25 bytes
2019-10-28 14:06:27 #24767(normal) <4m4m99>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=reject_bl_files: read map data cached 25 bytes
2019-10-28 14:06:27 #24767(normal) <8pumrj>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=accept_bl_files: read map data cached 30 bytes
2019-10-28 14:06:27 #24762(rspamd_proxy) <6susux>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=disabledeliver: read map data cached 1 bytes
2019-10-28 14:06:27 #24765(normal) <8pumrj>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=accept_bl_files: read map data cached 30 bytes
2019-10-28 14:06:27 #24766(normal) <8pumrj>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=accept_bl_files: read map data cached 30 bytes
2019-10-28 14:06:27 #24766(normal) <4m4m99>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=reject_bl_files: read map data cached 25 bytes
2019-10-28 14:06:27 #24762(rspamd_proxy) <4m4m99>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=reject_bl_files: read map data cached 25 bytes
2019-10-28 14:06:27 #24767(normal) <6susux>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=disabledeliver: read map data cached 1 bytes
2019-10-28 14:06:27 #24764(normal) <8pumrj>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=accept_bl_files: read map data cached 30 bytes
2019-10-28 14:06:27 #24766(normal) <6susux>; map; rspamd_map_read_cached: https://domain.tld/?scope=mail&option=disabledeliver: read map data cached 1 bytes
2019-10-28 14:06:39 #24763(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:06:49 GMT
2019-10-28 14:06:44 #24763(controller) <8pumrj>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:06:54 GMT
2019-10-28 14:06:45 #24763(controller) <6susux>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:06:55 GMT
2019-10-28 14:06:50 #24763(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:07:00 GMT
2019-10-28 14:06:58 #24763(controller) <8pumrj>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:07:08 GMT
2019-10-28 14:07:03 #24763(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:07:13 GMT
2019-10-28 14:07:03 #24763(controller) <6susux>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:07:13 GMT
2019-10-28 14:07:14 #24763(controller) <8pumrj>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:07:24 GMT
2019-10-28 14:07:17 #24763(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:07:27 GMT
2019-10-28 14:07:21 #24763(controller) <6susux>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:07:31 GMT
2019-10-28 14:07:28 #24763(controller) <8pumrj>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:07:38 GMT
2019-10-28 14:07:34 #24763(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:07:44 GMT
2019-10-28 14:07:34 #24763(controller) <6susux>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:07:44 GMT
2019-10-28 14:07:46 #24763(controller) <8pumrj>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:07:56 GMT
2019-10-28 14:07:50 #24763(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:08:00 GMT
2019-10-28 14:07:51 #24763(controller) <6susux>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:08:01 GMT
2019-10-28 14:07:59 #24763(controller) <8pumrj>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:08:09 GMT
2019-10-28 14:08:00 #24763(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:08:10 GMT
2019-10-28 14:08:10 #24763(controller) <6susux>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:08:20 GMT
2019-10-28 14:08:14 #24763(controller) <8pumrj>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:08:24 GMT
2019-10-28 14:08:17 #24763(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:08:27 GMT
2019-10-28 14:08:24 #24763(controller) <8pumrj>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:08:34 GMT
2019-10-28 14:08:25 #24763(controller) <6susux>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:08:35 GMT
2019-10-28 14:08:33 #24763(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:08:43 GMT
2019-10-28 14:08:41 #24763(controller) <8pumrj>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:08:51 GMT
2019-10-28 14:08:42 #24763(controller) <6susux>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:08:52 GMT
2019-10-28 14:08:44 #24763(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:08:54 GMT
2019-10-28 14:08:56 #24763(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:09:06 GMT
2019-10-28 14:08:58 #24763(controller) <8pumrj>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:09:08 GMT
2019-10-28 14:09:00 #24763(controller) <6susux>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:09:10 GMT
2019-10-28 14:09:11 #24763(controller) <8pumrj>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:09:21 GMT
2019-10-28 14:09:15 #24763(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:09:25 GMT
2019-10-28 14:09:18 #24763(controller) <6susux>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:09:28 GMT
2019-10-28 14:09:23 #24763(controller) <8pumrj>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:09:32 GMT
2019-10-28 14:09:31 #24763(controller) <6susux>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:09:41 GMT
2019-10-28 14:09:33 #24763(controller) <4m4m99>; map; http_map_finish: data is not modified for server domain.tld, next check at Mon, 28 Oct 2019 13:09:43 GMT

apiserver.log (logdate: Mon Oct 28 14:09:25 CET 2019)

127.0.0.1 - - [28/Oct/2019:14:06:25 +0200] "HEAD /scope=mail&option=reject_bl_files" 200 25 - "rspamd-1.9.4" 0
127.0.0.1 - - [28/Oct/2019:14:06:25 +0200] "HEAD /scope=mail&option=disabledeliver" 200 1 - "rspamd-1.9.4" 0
127.0.0.1 - - [28/Oct/2019:14:06:25 +0200] "HEAD /scope=mail&option=accept_bl_files" 200 30 - "rspamd-1.9.4" 0
127.0.0.1 - - [28/Oct/2019:14:06:25 +0200] "GET /scope=mail&option=reject_bl_files" 200 25 - "rspamd-1.9.4" 0
127.0.0.1 - - [28/Oct/2019:14:06:25 +0200] "GET /scope=mail&option=accept_bl_files" 200 30 - "rspamd-1.9.4" 0
127.0.0.1 - - [28/Oct/2019:14:06:25 +0200] "GET /scope=mail&option=disabledeliver" 200 1 - "rspamd-1.9.4" 0
127.0.0.1 - - [28/Oct/2019:14:06:39 +0200] "HEAD /scope=mail&option=reject_bl_files" 304 0 - "rspamd-1.9.4" 8d3577ecb432bfeda4e263bfeda7fda8
127.0.0.1 - - [28/Oct/2019:14:06:44 +0200] "HEAD /scope=mail&option=accept_bl_files" 304 0 - "rspamd-1.9.4" 29e0b386aae908a2b94932927b6398fe
127.0.0.1 - - [28/Oct/2019:14:06:45 +0200] "HEAD /scope=mail&option=disabledeliver" 304 0 - "rspamd-1.9.4" 68b329da9893e34099c7d8ad5cb9c940
127.0.0.1 - - [28/Oct/2019:14:06:50 +0200] "HEAD /scope=mail&option=reject_bl_files" 304 0 - "rspamd-1.9.4" 8d3577ecb432bfeda4e263bfeda7fda8
127.0.0.1 - - [28/Oct/2019:14:06:58 +0200] "HEAD /scope=mail&option=accept_bl_files" 304 0 - "rspamd-1.9.4" 29e0b386aae908a2b94932927b6398fe
127.0.0.1 - - [28/Oct/2019:14:07:03 +0200] "HEAD /scope=mail&option=reject_bl_files" 304 0 - "rspamd-1.9.4" 8d3577ecb432bfeda4e263bfeda7fda8
127.0.0.1 - - [28/Oct/2019:14:07:03 +0200] "HEAD /scope=mail&option=disabledeliver" 304 0 - "rspamd-1.9.4" 68b329da9893e34099c7d8ad5cb9c940
127.0.0.1 - - [28/Oct/2019:14:07:14 +0200] "HEAD /scope=mail&option=accept_bl_files" 304 0 - "rspamd-1.9.4" 29e0b386aae908a2b94932927b6398fe
127.0.0.1 - - [28/Oct/2019:14:07:17 +0200] "HEAD /scope=mail&option=reject_bl_files" 304 0 - "rspamd-1.9.4" 8d3577ecb432bfeda4e263bfeda7fda8
127.0.0.1 - - [28/Oct/2019:14:07:21 +0200] "HEAD /scope=mail&option=disabledeliver" 304 0 - "rspamd-1.9.4" 68b329da9893e34099c7d8ad5cb9c940
127.0.0.1 - - [28/Oct/2019:14:07:28 +0200] "HEAD /scope=mail&option=accept_bl_files" 304 0 - "rspamd-1.9.4" 29e0b386aae908a2b94932927b6398fe
127.0.0.1 - - [28/Oct/2019:14:07:34 +0200] "HEAD /scope=mail&option=reject_bl_files" 304 0 - "rspamd-1.9.4" 8d3577ecb432bfeda4e263bfeda7fda8
127.0.0.1 - - [28/Oct/2019:14:07:34 +0200] "HEAD /scope=mail&option=disabledeliver" 304 0 - "rspamd-1.9.4" 68b329da9893e34099c7d8ad5cb9c940
127.0.0.1 - - [28/Oct/2019:14:07:46 +0200] "HEAD /scope=mail&option=accept_bl_files" 304 0 - "rspamd-1.9.4" 29e0b386aae908a2b94932927b6398fe
127.0.0.1 - - [28/Oct/2019:14:07:50 +0200] "HEAD /scope=mail&option=reject_bl_files" 304 0 - "rspamd-1.9.4" 8d3577ecb432bfeda4e263bfeda7fda8
127.0.0.1 - - [28/Oct/2019:14:07:51 +0200] "HEAD /scope=mail&option=disabledeliver" 304 0 - "rspamd-1.9.4" 68b329da9893e34099c7d8ad5cb9c940
127.0.0.1 - - [28/Oct/2019:14:07:59 +0200] "HEAD /scope=mail&option=accept_bl_files" 304 0 - "rspamd-1.9.4" 29e0b386aae908a2b94932927b6398fe
127.0.0.1 - - [28/Oct/2019:14:08:00 +0200] "HEAD /scope=mail&option=reject_bl_files" 304 0 - "rspamd-1.9.4" 8d3577ecb432bfeda4e263bfeda7fda8
127.0.0.1 - - [28/Oct/2019:14:08:10 +0200] "HEAD /scope=mail&option=disabledeliver" 304 0 - "rspamd-1.9.4" 68b329da9893e34099c7d8ad5cb9c940
127.0.0.1 - - [28/Oct/2019:14:08:14 +0200] "HEAD /scope=mail&option=accept_bl_files" 304 0 - "rspamd-1.9.4" 29e0b386aae908a2b94932927b6398fe
127.0.0.1 - - [28/Oct/2019:14:08:17 +0200] "HEAD /scope=mail&option=reject_bl_files" 304 0 - "rspamd-1.9.4" 8d3577ecb432bfeda4e263bfeda7fda8
127.0.0.1 - - [28/Oct/2019:14:08:24 +0200] "HEAD /scope=mail&option=accept_bl_files" 304 0 - "rspamd-1.9.4" 29e0b386aae908a2b94932927b6398fe
127.0.0.1 - - [28/Oct/2019:14:08:25 +0200] "HEAD /scope=mail&option=disabledeliver" 304 0 - "rspamd-1.9.4" 68b329da9893e34099c7d8ad5cb9c940
127.0.0.1 - - [28/Oct/2019:14:08:33 +0200] "HEAD /scope=mail&option=reject_bl_files" 304 0 - "rspamd-1.9.4" 8d3577ecb432bfeda4e263bfeda7fda8
127.0.0.1 - - [28/Oct/2019:14:08:41 +0200] "HEAD /scope=mail&option=accept_bl_files" 304 0 - "rspamd-1.9.4" 29e0b386aae908a2b94932927b6398fe
127.0.0.1 - - [28/Oct/2019:14:08:42 +0200] "HEAD /scope=mail&option=disabledeliver" 304 0 - "rspamd-1.9.4" 68b329da9893e34099c7d8ad5cb9c940
127.0.0.1 - - [28/Oct/2019:14:08:44 +0200] "HEAD /scope=mail&option=reject_bl_files" 304 0 - "rspamd-1.9.4" 8d3577ecb432bfeda4e263bfeda7fda8
127.0.0.1 - - [28/Oct/2019:14:08:56 +0200] "HEAD /scope=mail&option=reject_bl_files" 304 0 - "rspamd-1.9.4" 8d3577ecb432bfeda4e263bfeda7fda8
127.0.0.1 - - [28/Oct/2019:14:08:58 +0200] "HEAD /scope=mail&option=accept_bl_files" 304 0 - "rspamd-1.9.4" 29e0b386aae908a2b94932927b6398fe
127.0.0.1 - - [28/Oct/2019:14:09:00 +0200] "HEAD /scope=mail&option=disabledeliver" 304 0 - "rspamd-1.9.4" 68b329da9893e34099c7d8ad5cb9c940
127.0.0.1 - - [28/Oct/2019:14:09:11 +0200] "HEAD /scope=mail&option=accept_bl_files" 304 0 - "rspamd-1.9.4" 29e0b386aae908a2b94932927b6398fe
127.0.0.1 - - [28/Oct/2019:14:09:15 +0200] "HEAD /scope=mail&option=reject_bl_files" 304 0 - "rspamd-1.9.4" 8d3577ecb432bfeda4e263bfeda7fda8
127.0.0.1 - - [28/Oct/2019:14:09:18 +0200] "HEAD /scope=mail&option=disabledeliver" 304 0 - "rspamd-1.9.4" 68b329da9893e34099c7d8ad5cb9c940
127.0.0.1 - - [28/Oct/2019:14:09:22 +0200] "HEAD /scope=mail&option=accept_bl_files" 304 0 - "rspamd-1.9.4" 29e0b386aae908a2b94932927b6398fe

2.1-1~buster fixed the issue for me.

2.1-1~bionic for me as well. Probably good to close this one...

Was this page helpful?
0 / 5 - 0 ratings