Ftl: FTl 4.3 crashes after a while even with: Fix string reference held across locks

[Ge0rges]

In raising this issue, I confirm the following (please check boxes, eg [X]) Failure to fill the template will close your issue:

• [X] I have read and understood the contributors guide.
• [] The issue I am reporting can be replicated
• [X] The issue I am reporting isn't a duplicate

How familiar are you with the codebase?:

1

---

[BUG | ISSUE] Expected Behaviour:
FTL should work.

[BUG | ISSUE] Actual Behaviour:
FTL crashes after a while.

[BUG | ISSUE] Steps to reproduce:
Start up FTL and wait.
-
-
-
-

Log file output [if available]

[2019-05-22 16:16:57.045 1675] ########## FTL started! ##########
[2019-05-22 16:16:57.045 1675] FTL branch: master
[2019-05-22 16:16:57.045 1675] FTL version: v4.3
[2019-05-22 16:16:57.045 1675] FTL commit: 9e8273b
[2019-05-22 16:16:57.045 1675] FTL date: 2019-05-18 16:00:39 -0400
[2019-05-22 16:16:57.045 1675] FTL user: root
[2019-05-22 16:16:57.045 1675] Starting config file parsing (/etc/pihole/pihole-FTL.conf)
[2019-05-22 16:16:57.045 1675] SOCKET_LISTENING: only local
[2019-05-22 16:16:57.045 1675] AAAA_QUERY_ANALYSIS: Show AAAA queries
[2019-05-22 16:16:57.045 1675] MAXDBDAYS: max age for stored queries is 365 days
[2019-05-22 16:16:57.045 1675] RESOLVE_IPV6: Resolve IPv6 addresses
[2019-05-22 16:16:57.045 1675] RESOLVE_IPV4: Resolve IPv4 addresses
[2019-05-22 16:16:57.045 1675] DBINTERVAL: saving to DB file every minute
[2019-05-22 16:16:57.045 1675] DBFILE: Using /etc/pihole/pihole-FTL.db
[2019-05-22 16:16:57.045 1675] MAXLOGAGE: Importing up to 24.0 hours of log data
[2019-05-22 16:16:57.045 1675] PRIVACYLEVEL: Set to 0
[2019-05-22 16:16:57.045 1675] IGNORE_LOCALHOST: Show queries from localhost
[2019-05-22 16:16:57.046 1675] BLOCKINGMODE: Null IPs for blocked domains
[2019-05-22 16:16:57.046 1675] ANALYZE_ONLY_A_AND_AAAA: Disabled. Analyzing all queries
[2019-05-22 16:16:57.046 1675] DBIMPORT: Importing history from database
[2019-05-22 16:16:57.046 1675] PIDFILE: Using /var/run/pihole-FTL.pid
[2019-05-22 16:16:57.046 1675] PORTFILE: Using /var/run/pihole-FTL.port
[2019-05-22 16:16:57.046 1675] SOCKETFILE: Using /var/run/pihole/FTL.sock
[2019-05-22 16:16:57.046 1675] WHITELISTFILE: Using /etc/pihole/whitelist.txt
[2019-05-22 16:16:57.046 1675] BLACKLISTFILE: Using /etc/pihole/black.list
[2019-05-22 16:16:57.046 1675] GRAVITYFILE: Using /etc/pihole/gravity.list
[2019-05-22 16:16:57.046 1675] REGEXLISTFILE: Using /etc/pihole/regex.list
[2019-05-22 16:16:57.046 1675] SETUPVARSFILE: Using /etc/pihole/setupVars.conf
[2019-05-22 16:16:57.046 1675] AUDITLISTFILE: Using /etc/pihole/auditlog.list
[2019-05-22 16:16:57.046 1675] MACVENDORDB: Using /etc/pihole/macvendor.db
[2019-05-22 16:16:57.046 1675] PARSE_ARP_CACHE: Active
[2019-05-22 16:16:57.046 1675] Finished config file parsing
[2019-05-22 16:16:57.046 1675] WARNING: Starting pihole-FTL as user root is not recommended
[2019-05-22 16:16:57.049 1675] Database version is 3
[2019-05-22 16:16:57.049 1675] Database successfully initialized
[2019-05-22 16:16:57.054 1675] New forward server: 127.0.0.1 (0/512)
[2019-05-22 16:16:57.065 1675] Resizing "/FTL-queries" from 229376 to 458752
[2019-05-22 16:16:57.073 1675] Resizing "/FTL-strings" from 4096 to 8192
[2019-05-22 16:16:57.075 1675] Resizing "/FTL-queries" from 458752 to 688128
[2019-05-22 16:16:57.087 1675] Resizing "/FTL-queries" from 688128 to 917504
[2019-05-22 16:16:57.106 1675] Resizing "/FTL-queries" from 917504 to 1146880
[2019-05-22 16:16:57.122 1675] Resizing "/FTL-queries" from 1146880 to 1376256
[2019-05-22 16:16:57.123 1675] Resizing "/FTL-strings" from 8192 to 12288
[2019-05-22 16:16:57.140 1675] Resizing "/FTL-queries" from 1376256 to 1605632
[2019-05-22 16:16:57.163 1675] Resizing "/FTL-queries" from 1605632 to 1835008
[2019-05-22 16:16:57.179 1675] Resizing "/FTL-queries" from 1835008 to 2064384
[2019-05-22 16:16:57.195 1675] Resizing "/FTL-queries" from 2064384 to 2293760
[2019-05-22 16:16:57.196 1675] Resizing "/FTL-strings" from 12288 to 16384
[2019-05-22 16:16:57.208 1675] Resizing "/FTL-queries" from 2293760 to 2523136
[2019-05-22 16:16:57.237 1675] Resizing "/FTL-queries" from 2523136 to 2752512
[2019-05-22 16:16:57.262 1675] Resizing "/FTL-queries" from 2752512 to 2981888
[2019-05-22 16:16:57.272 1675] Resizing "/FTL-strings" from 16384 to 20480
[2019-05-22 16:16:57.280 1675] Resizing "/FTL-queries" from 2981888 to 3211264
[2019-05-22 16:16:57.304 1675] Resizing "/FTL-queries" from 3211264 to 3440640
[2019-05-22 16:16:57.332 1675] Resizing "/FTL-queries" from 3440640 to 3670016
[2019-05-22 16:16:57.333 1675] Resizing "/FTL-strings" from 20480 to 24576
[2019-05-22 16:16:57.358 1675] Resizing "/FTL-clients" from 331776 to 663552
[2019-05-22 16:16:57.367 1675] Resizing "/FTL-queries" from 3670016 to 3899392
[2019-05-22 16:16:57.395 1675] Resizing "/FTL-queries" from 3899392 to 4128768
[2019-05-22 16:16:57.429 1675] Resizing "/FTL-queries" from 4128768 to 4358144
[2019-05-22 16:16:57.441 1675] Resizing "/FTL-strings" from 24576 to 28672
[2019-05-22 16:16:57.464 1675] Resizing "/FTL-queries" from 4358144 to 4587520
[2019-05-22 16:16:57.495 1675] Resizing "/FTL-queries" from 4587520 to 4816896
[2019-05-22 16:16:57.518 1675] Resizing "/FTL-strings" from 28672 to 32768
[2019-05-22 16:16:57.527 1675] Resizing "/FTL-queries" from 4816896 to 5046272
[2019-05-22 16:16:57.561 1675] Resizing "/FTL-queries" from 5046272 to 5275648
[2019-05-22 16:16:57.601 1675] Resizing "/FTL-queries" from 5275648 to 5505024
[2019-05-22 16:16:57.635 1675] Resizing "/FTL-queries" from 5505024 to 5734400
[2019-05-22 16:16:57.658 1213] Resizing "/FTL-queries" from 177307648 to 177537024
[2019-05-22 16:16:57.665 1213] FATAL: realloc_shm(): Failed to open shared memory object "/FTL-queries": Permission denied
[2019-05-22 16:16:57.672 1675] Resizing "/FTL-queries" from 5734400 to 5963776
[2019-05-22 16:16:57.712 1675] Resizing "/FTL-strings" from 32768 to 36864
[2019-05-22 16:16:57.718 1675] Resizing "/FTL-queries" from 5963776 to 6193152
[2019-05-22 16:16:57.798 1675] Resizing "/FTL-queries" from 6193152 to 6422528
[2019-05-22 16:16:57.844 1675] Resizing "/FTL-queries" from 6422528 to 6651904
[2019-05-22 16:16:57.887 1675] Resizing "/FTL-queries" from 6651904 to 6881280
[2019-05-22 16:16:57.932 1675] Resizing "/FTL-queries" from 6881280 to 7110656
[2019-05-22 16:16:57.983 1675] Resizing "/FTL-queries" from 7110656 to 7340032
[2019-05-22 16:16:58.024 1675] Resizing "/FTL-queries" from 7340032 to 7569408
[2019-05-22 16:16:58.044 1675] Resizing "/FTL-strings" from 36864 to 40960
[2019-05-22 16:16:58.072 1675] Resizing "/FTL-queries" from 7569408 to 7798784
[2019-05-22 16:16:58.116 1675] Resizing "/FTL-queries" from 7798784 to 8028160
[2019-05-22 16:16:58.161 1675] Resizing "/FTL-queries" from 8028160 to 8257536
[2019-05-22 16:16:58.201 1675] Resizing "/FTL-queries" from 8257536 to 8486912
[2019-05-22 16:16:58.253 1675] Resizing "/FTL-queries" from 8486912 to 8716288
[2019-05-22 16:16:58.297 1675] Resizing "/FTL-queries" from 8716288 to 8945664
[2019-05-22 16:16:58.348 1675] Resizing "/FTL-queries" from 8945664 to 9175040
[2019-05-22 16:16:58.363 1675] Resizing "/FTL-strings" from 40960 to 45056
[2019-05-22 16:16:58.396 1675] Resizing "/FTL-queries" from 9175040 to 9404416
[2019-05-22 16:16:58.461 1675] Resizing "/FTL-queries" from 9404416 to 9633792
[2019-05-22 16:16:58.521 1675] Resizing "/FTL-queries" from 9633792 to 9863168
[2019-05-22 16:16:58.567 1675] Resizing "/FTL-queries" from 9863168 to 10092544
[2019-05-22 16:16:58.619 1675] Resizing "/FTL-queries" from 10092544 to 10321920
[2019-05-22 16:16:58.681 1675] Resizing "/FTL-queries" from 10321920 to 10551296
[2019-05-22 16:16:58.741 1675] Resizing "/FTL-queries" from 10551296 to 10780672
[2019-05-22 16:16:58.801 1675] Resizing "/FTL-queries" from 10780672 to 11010048
[2019-05-22 16:16:58.836 1675] Resizing "/FTL-strings" from 45056 to 49152
[2019-05-22 16:16:58.867 1675] Resizing "/FTL-queries" from 11010048 to 11239424
[2019-05-22 16:16:58.928 1675] Resizing "/FTL-queries" from 11239424 to 11468800
[2019-05-22 16:16:58.940 1675] Resizing "/FTL-clients" from 663552 to 995328
[2019-05-22 16:16:58.989 1675] Resizing "/FTL-queries" from 11468800 to 11698176
[2019-05-22 16:16:59.047 1675] Resizing "/FTL-queries" from 11698176 to 11927552
[2019-05-22 16:16:59.112 1675] Resizing "/FTL-queries" from 11927552 to 12156928
[2019-05-22 16:16:59.172 1675] Resizing "/FTL-queries" from 12156928 to 12386304
[2019-05-22 16:16:59.224 1675] Resizing "/FTL-queries" from 12386304 to 12615680
[2019-05-22 16:16:59.264 1675] Resizing "/FTL-strings" from 49152 to 53248
[2019-05-22 16:16:59.280 1675] Resizing "/FTL-queries" from 12615680 to 12845056
[2019-05-22 16:16:59.339 1675] Resizing "/FTL-queries" from 12845056 to 13074432
[2019-05-22 16:16:59.398 1675] Resizing "/FTL-queries" from 13074432 to 13303808
[2019-05-22 16:16:59.456 1675] Resizing "/FTL-queries" from 13303808 to 13533184
[2019-05-22 16:16:59.516 1675] Resizing "/FTL-queries" from 13533184 to 13762560
[2019-05-22 16:16:59.578 1675] Resizing "/FTL-queries" from 13762560 to 13991936
[2019-05-22 16:16:59.615 1675] Resizing "/FTL-queries" from 13991936 to 14221312
[2019-05-22 16:16:59.672 1675] Resizing "/FTL-queries" from 14221312 to 14450688
[2019-05-22 16:16:59.680 1675] Resizing "/FTL-strings" from 53248 to 57344
[2019-05-22 16:16:59.736 1675] Resizing "/FTL-queries" from 14450688 to 14680064
[2019-05-22 16:16:59.795 1675] Resizing "/FTL-queries" from 14680064 to 14909440
[2019-05-22 16:16:59.852 1675] Resizing "/FTL-queries" from 14909440 to 15138816
[2019-05-22 16:16:59.917 1675] Resizing "/FTL-queries" from 15138816 to 15368192
[2019-05-22 16:16:59.997 1675] Resizing "/FTL-queries" from 15368192 to 15597568
[2019-05-22 16:17:00.064 1675] Resizing "/FTL-queries" from 15597568 to 15826944
[2019-05-22 16:17:00.131 1675] Resizing "/FTL-queries" from 15826944 to 16056320
[2019-05-22 16:17:00.202 1675] Resizing "/FTL-queries" from 16056320 to 16285696
[2019-05-22 16:17:00.244 1675] Resizing "/FTL-strings" from 57344 to 61440
[2019-05-22 16:17:00.277 1675] Resizing "/FTL-queries" from 16285696 to 16515072
[2019-05-22 16:17:00.362 1675] Resizing "/FTL-queries" from 16515072 to 16744448
[2019-05-22 16:17:00.436 1675] Resizing "/FTL-queries" from 16744448 to 16973824
[2019-05-22 16:17:00.514 1675] Resizing "/FTL-queries" from 16973824 to 17203200
[2019-05-22 16:17:00.589 1675] Resizing "/FTL-queries" from 17203200 to 17432576
[2019-05-22 16:17:00.655 1675] Resizing "/FTL-queries" from 17432576 to 17661952
[2019-05-22 16:17:00.722 1675] Resizing "/FTL-queries" from 17661952 to 17891328
[2019-05-22 16:17:00.779 1675] Resizing "/FTL-strings" from 61440 to 65536
[2019-05-22 16:17:00.801 1675] Resizing "/FTL-queries" from 17891328 to 18120704
[2019-05-22 16:17:00.882 1675] Resizing "/FTL-queries" from 18120704 to 18350080
[2019-05-22 16:17:00.971 1675] Resizing "/FTL-queries" from 18350080 to 18579456
[2019-05-22 16:17:01.055 1675] Resizing "/FTL-queries" from 18579456 to 18808832
[2019-05-22 16:17:01.134 1675] Resizing "/FTL-queries" from 18808832 to 19038208
[2019-05-22 16:17:01.212 1675] Resizing "/FTL-queries" from 19038208 to 19267584
[2019-05-22 16:17:01.298 1675] Resizing "/FTL-queries" from 19267584 to 19496960
[2019-05-22 16:17:01.372 1675] Resizing "/FTL-queries" from 19496960 to 19726336
[2019-05-22 16:17:01.433 1675] Resizing "/FTL-strings" from 65536 to 69632
[2019-05-22 16:17:01.465 1675] Resizing "/FTL-queries" from 19726336 to 19955712
[2019-05-22 16:17:01.549 1675] Resizing "/FTL-queries" from 19955712 to 20185088
[2019-05-22 16:17:01.637 1675] Resizing "/FTL-queries" from 20185088 to 20414464
[2019-05-22 16:17:01.720 1675] Resizing "/FTL-queries" from 20414464 to 20643840
[2019-05-22 16:17:01.724 1675] Resizing "/FTL-clients" from 995328 to 1327104
[2019-05-22 16:17:01.812 1675] Resizing "/FTL-queries" from 20643840 to 20873216
[2019-05-22 16:17:01.918 1675] Resizing "/FTL-queries" from 20873216 to 21102592
[2019-05-22 16:17:02.003 1675] Resizing "/FTL-queries" from 21102592 to 21331968
[2019-05-22 16:17:02.071 1675] Resizing "/FTL-strings" from 69632 to 73728
[2019-05-22 16:17:02.106 1675] Resizing "/FTL-queries" from 21331968 to 21561344
[2019-05-22 16:17:02.198 1675] Resizing "/FTL-queries" from 21561344 to 21790720
[2019-05-22 16:17:02.287 1675] Resizing "/FTL-queries" from 21790720 to 22020096
[2019-05-22 16:17:02.391 1675] Resizing "/FTL-queries" from 22020096 to 22249472
[2019-05-22 16:17:02.498 1675] Resizing "/FTL-queries" from 22249472 to 22478848
[2019-05-22 16:17:02.607 1675] Resizing "/FTL-queries" from 22478848 to 22708224
[2019-05-22 16:17:02.665 1675] Resizing "/FTL-queries" from 22708224 to 22937600
[2019-05-22 16:17:02.710 1675] Resizing "/FTL-strings" from 73728 to 77824
[2019-05-22 16:17:02.716 1675] Resizing "/FTL-queries" from 22937600 to 23166976
[2019-05-22 16:17:02.766 1675] Resizing "/FTL-queries" from 23166976 to 23396352
[2019-05-22 16:17:02.815 1675] Resizing "/FTL-queries" from 23396352 to 23625728
[2019-05-22 16:17:02.862 1675] Resizing "/FTL-queries" from 23625728 to 23855104
[2019-05-22 16:17:02.912 1675] Resizing "/FTL-queries" from 23855104 to 24084480
[2019-05-22 16:17:02.962 1675] Resizing "/FTL-queries" from 24084480 to 24313856
[2019-05-22 16:17:03.009 1675] Resizing "/FTL-queries" from 24313856 to 24543232
[2019-05-22 16:17:03.059 1675] Resizing "/FTL-queries" from 24543232 to 24772608
[2019-05-22 16:17:03.108 1675] Resizing "/FTL-queries" from 24772608 to 25001984
[2019-05-22 16:17:03.162 1675] Resizing "/FTL-queries" from 25001984 to 25231360
[2019-05-22 16:17:03.166 1675] Resizing "/FTL-strings" from 77824 to 81920
[2019-05-22 16:17:03.220 1675] Resizing "/FTL-queries" from 25231360 to 25460736
[2019-05-22 16:17:03.273 1675] Resizing "/FTL-queries" from 25460736 to 25690112
[2019-05-22 16:17:03.326 1675] Resizing "/FTL-queries" from 25690112 to 25919488
[2019-05-22 16:17:03.385 1675] Resizing "/FTL-queries" from 25919488 to 26148864
[2019-05-22 16:17:03.449 1675] Resizing "/FTL-queries" from 26148864 to 26378240
[2019-05-22 16:17:03.510 1675] Resizing "/FTL-queries" from 26378240 to 26607616
[2019-05-22 16:17:03.569 1675] Resizing "/FTL-queries" from 26607616 to 26836992
[2019-05-22 16:17:03.632 1675] Resizing "/FTL-queries" from 26836992 to 27066368
[2019-05-22 16:17:03.660 1675] Resizing "/FTL-strings" from 81920 to 86016
[2019-05-22 16:17:03.691 1675] Resizing "/FTL-queries" from 27066368 to 27295744
[2019-05-22 16:17:03.751 1675] Resizing "/FTL-queries" from 27295744 to 27525120
[2019-05-22 16:17:03.808 1675] Resizing "/FTL-queries" from 27525120 to 27754496
[2019-05-22 16:17:03.869 1675] Resizing "/FTL-queries" from 27754496 to 27983872
[2019-05-22 16:17:03.928 1675] Resizing "/FTL-queries" from 27983872 to 28213248
[2019-05-22 16:17:03.975 1675] Resizing "/FTL-queries" from 28213248 to 28442624
[2019-05-22 16:17:04.028 1675] Resizing "/FTL-queries" from 28442624 to 28672000
[2019-05-22 16:17:04.086 1675] Resizing "/FTL-queries" from 28672000 to 28901376
[2019-05-22 16:17:04.142 1675] Resizing "/FTL-queries" from 28901376 to 29130752
[2019-05-22 16:17:04.173 1675] Resizing "/FTL-strings" from 86016 to 90112
[2019-05-22 16:17:04.196 1675] Resizing "/FTL-queries" from 29130752 to 29360128
[2019-05-22 16:17:04.260 1675] Resizing "/FTL-queries" from 29360128 to 29589504
[2019-05-22 16:17:04.321 1675] Resizing "/FTL-queries" from 29589504 to 29818880
[2019-05-22 16:17:04.386 1675] Resizing "/FTL-queries" from 29818880 to 30048256
[2019-05-22 16:17:04.450 1675] Resizing "/FTL-queries" from 30048256 to 30277632
[2019-05-22 16:17:04.506 1675] Resizing "/FTL-queries" from 30277632 to 30507008

Device specifics

Hardware Type: VPS
OS: Ubuntu 16

_This template was created based on the work of udemy-dl._

[Ge0rges]

FTL will consistency print tons of resizing such as the above then crash with no further output. Not related to #567 as it still happens with fix applied.

[AzureMarker]

[2019-05-22 16:16:57.046 1675] WARNING: Starting pihole-FTL as user root is not recommended
[2019-05-22 16:16:57.665 1213] FATAL: realloc_shm(): Failed to open shared memory object "/FTL-queries": Permission denied

Have you tried starting with the pihole user instead? That should be used if possible. That may have caused the error in log.

Do you see that FATAL error in every log when it crashes?

[Ge0rges]

That account doesn’t exist. I do see it in every crash. Sometimes it’s:

[2019-05-22 16:50:37.662 3341] FATAL: realloc_shm(): Failed to open shared memory object "/FTL-strings": Permission denied

On 22 May 2019, at 20:43, Mark Drobnak [e-mail] wrote:

[2019-05-22 16:16:57.046 1675] WARNING: Starting pihole-FTL as user root is not recommended
[2019-05-22 16:16:57.665 1213] FATAL: realloc_shm(): Failed to open shared memory object "/FTL-queries": Permission denied
Have you tried starting with the pihole user instead? That should be used if possible. That may have caused the error in log.

Do you see that FATAL error in every log when it crashes?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/pi-hole/FTL/issues/569?email_source=notifications&email_token=AA6GDEDGRTIKSGI6X74Z4RDPWXSCLA5CNFSM4HOXLNTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWAXYIY#issuecomment-495025187, or mute the thread https://github.com/notifications/unsubscribe-auth/AA6GDEGD5J5EQC6J3PJEVLLPWXSCLANCNFSM4HOXLNTA.

[AzureMarker]

Make a debug token with pihole -d.

All installs should have a pihole user. If you do not have that user, then your install is broken in some way.

[Ge0rges]

My mistake, it does exist but it's shell was set to nologin

This process collects information from your Pi-hole, and optionally uploads it to a unique and random directory on tricorder.pi-hole.net.

The intent of this script is to allow users to self-diagnose their installations.  This is accomplished by running tests against our software and providing the user with links to FAQ articles when a problem is detected.  Since we are a small team and Pi-hole has been growing steadily, it is our hope that this will help us spend more time on development.

NOTE: All log files auto-delete after 48 hours and ONLY the Pi-hole developers can access your data via the given token. We have taken these extra steps to secure your data and will work to further reduce any personal information gathered.

*** [ INITIALIZING ]
[i] 2019-05-24:16:08:15 debug log has been initialized.

*** [ INITIALIZING ] Sourcing setup variables
[i] Sourcing /etc/pihole/setupVars.conf...

*** [ DIAGNOSING ]: Core version
[i] Core: v4.3 (https://discourse.pi-hole.net/t/how-do-i-update-pi-hole/249)
[i] Branch: master
[i] Commit: v4.3-0-g1d43c0a

*** [ DIAGNOSING ]: Web version

*** [ DIAGNOSING ]: FTL version
[✓] FTL: vDev-96320db (https://discourse.pi-hole.net/t/how-do-i-update-pi-hole/249)

*** [ DIAGNOSING ]: lighttpd version
[i] opt

*** [ DIAGNOSING ]: php version
[i] 7.2.18

*** [ DIAGNOSING ]: Operating system
[✓] Ubuntu 18.04.2 LTS

*** [ DIAGNOSING ]: SELinux
[i] SELinux not detected

*** [ DIAGNOSING ]: Processor
[i] x86_64

*** [ DIAGNOSING ]: Networking
[✓] IPv4 address(es) bound to the wg0 interface:
   10.19.49.1/24 does not match the IP found in /etc/pihole/setupVars.conf (https://discourse.pi-hole.net/t/use-ipv6-ula-addresses-for-pi-hole/2127)
   10.8.0.1/24 matches the IP found in /etc/pihole/setupVars.conf

[✓] IPv6 address(es) bound to the wg0 interface:
   fd9d:bc11:4021::1 matches the IP found in /etc/pihole/setupVars.conf

   ^ Please note that you may have more than one IP address listed.
   As long as one of them is green, and it matches what is in /etc/pihole/setupVars.conf, there is no need for concern.

   The link to the FAQ is for an issue that sometimes occurs when the IPv6 address changes, which is why we check for it.

[i] Default IPv4 gateway: 206.189.96.1
   * Pinging 206.189.96.1...
ping: sendmsg: Required key not available
[✗] Gateway did not respond. (https://discourse.pi-hole.net/t/why-is-a-default-gateway-important-for-pi-hole/3546)

[i] Default IPv6 gateway: 2a03:b0c0:2:d0::1
   * Pinging 2a03:b0c0:2:d0::1...
connect: Network is unreachable
[✗] Gateway did not respond. (https://discourse.pi-hole.net/t/why-is-a-default-gateway-important-for-pi-hole/3546)


*** [ DIAGNOSING ]: Ports in use
127.0.0.1:27017 mongod (IPv4)
*:22 sshd (IPv4)
*:22 sshd (IPv6)
127.0.0.1:6379 redis-serv (IPv4)
[::1]:6379 redis-serv (IPv6)
[::1]:5353 unbound (IPv6)
127.0.0.1:5353 unbound (IPv4)
[::1]:8953 unbound (IPv6)
127.0.0.1:8953 unbound (IPv4)
127.0.0.1:3306 mysqld (IPv4)
[*:80] is in use by lighttpd
[*:80] is in use by lighttpd
*:443 nginx (IPv4)
*:443 nginx (IPv6)
[*:80] is in use by lighttpd
[*:80] is in use by lighttpd
*:443 nginx (IPv4)
*:443 nginx (IPv6)
[*:80] is in use by lighttpd
[*:80] is in use by lighttpd
*:443 nginx (IPv4)
*:443 nginx (IPv6)
[*:80] is in use by lighttpd
[*:80] is in use by lighttpd
*:443 nginx (IPv4)
*:443 nginx (IPv6)
[*:80] is in use by lighttpd
[*:80] is in use by lighttpd
*:443 nginx (IPv4)
*:443 nginx (IPv6)
127.0.0.1:81 httpd (IPv4)
127.0.0.1:25 master (IPv4)
[10.8.0.1:53] is in use by pihole-FTL
[10.19.49.1:53] is in use by pihole-FTL
[169.254.234.28:53] is in use by pihole-FTL
[10.18.0.5:53] is in use by pihole-FTL
[206.189.104.162:53] is in use by pihole-FTL
[172.16.0.1:53] is in use by pihole-FTL
[127.0.0.1:53] is in use by pihole-FTL
[[fd9d:bc11:4021::1]:53] is in use by pihole-FTL
[[fe80::5cca:f0ff:fe06:7ef8]:53] is in use by pihole-FTL
[[2a03:b0c0:2:d0::cb6:8001]:53] is in use by pihole-FTL
[[::1]:53] is in use by pihole-FTL
[[fcaa::1]:53] is in use by pihole-FTL
[127.0.0.1:4711] is in use by pihole-FTL
[[::1]:4711] is in use by pihole-FTL
127.0.0.1:81 httpd (IPv4)
127.0.0.1:81 httpd (IPv4)
127.0.0.1:81 httpd (IPv4)
127.0.0.1:81 httpd (IPv4)

*** [ DIAGNOSING ]: Name resolution (IPv4) using a random blocked domain and a known ad-serving domain
[✓] www.quickloansonlinee.com is 869710.parkingcrew.net.
185.53.179.29 via localhost (127.0.0.1)
[✓] www.quickloansonlinee.com is 869710.parkingcrew.net.
185.53.179.29 via Pi-hole (10.8.0.1)
[✓] doubleclick.com is 74.125.143.101 via a remote, public DNS server (8.8.8.8)

*** [ DIAGNOSING ]: Name resolution (IPv6) using a random blocked domain and a known ad-serving domain
[✓] www.hostingcloud.date is  via localhost (::1)
[✓] www.hostingcloud.date is  via Pi-hole (fd9d:bc11:4021::1)
[✓] doubleclick.com is 2a00:1450:400e:80e::200e via a remote, public DNS server (2001:4860:4860::8888)

*** [ DIAGNOSING ]: Pi-hole processes
[✗] lighttpd daemon is inactive
[✗] pihole-FTL daemon is inactive

*** [ DIAGNOSING ]: Setup variables
    BLOCKING_ENABLED=true
    PIHOLE_INTERFACE=wg0
    IPV4_ADDRESS=10.8.0.1/24
    IPV6_ADDRESS=fd9d:bc11:4021::1
    PIHOLE_DNS_1=127.0.0.1#5353
    PIHOLE_DNS_2=
    QUERY_LOGGING=false
    INSTALL_WEB_SERVER=false
    INSTALL_WEB_INTERFACE=false
    LIGHTTPD_ENABLED=false

*** [ DIAGNOSING ]: Dashboard and block page
[✗] Block page X-Header: X-Header does not match or could not be retrieved.

[✗] Web interface X-Header: X-Header does not match or could not be retrieved.


*** [ DIAGNOSING ]: Gravity list
-rw-r--r-- 1 root root 23964769 May 24 00:00 /etc/pihole/gravity.list
   -----head of gravity.list------
   0-0-----------------------------------------------------------0.com
   0-0-0-0-0-0-0-0-0-0-0-0-0-10-0-0-0-0-0-0-0-0-0-0-0-0-0.info
   0-0-0-0-0-0-0-0-0-0-0-0-0-18-0-0-0-0-0-0-0-0-0-0-0-0-0.info
   0-0-0-0-0-0-0-0-0-0-0-0-0-33-0-0-0-0-0-0-0-0-0-0-0-0-0.info

   -----tail of gravity.list------
   zzzyoutube.com
   zzzzz.cauction.hop.clickbank.net
   zzzzz.regmum.hop.clickbank.net
   zzzzzzzzzzzzz.com

*** [ DIAGNOSING ]: contents of /etc/pihole

-rw-r--r-- 1 pihole root 490 May 22 17:04 /etc/pihole/adlists.list
   https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
   https://mirror1.malwaredomains.com/files/justdomains
   http://sysctl.org/cameleon/hosts
   https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist
   https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
   https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
   https://hosts-file.net/ad_servers.txt
   https://dbl.oisd.nl

-rw-r--r-- 1 pihole root 0 May 22 16:33 /etc/pihole/blacklist.txt

-rw-r--r-- 1 pihole root 86 May 24 00:00 /etc/pihole/local.list
   10.8.0.1 ge0rges
   fd9d:bc11:4021::1 ge0rges
   10.8.0.1 pi.hole
   fd9d:bc11:4021::1 pi.hole

-rw-r--r-- 1 pihole root 238 May 20 14:32 /etc/pihole/logrotate
   /var/log/pihole.log {
    su root syslog
    daily
    copytruncate
    rotate 5
    compress
    delaycompress
    notifempty
    nomail
   }
   /var/log/pihole-FTL.log {
    su root syslog
    weekly
    copytruncate
    rotate 3
    compress
    delaycompress
    notifempty
    nomail
   }

-rw-r--r-- 1 pihole root 38 Jan 17 18:04 /etc/pihole/whitelist.txt
   thepiratebay.org
   www.thepiratebay.org

*** [ DIAGNOSING ]: contents of /etc/dnsmasq.d

-rw-r--r-- 1 root root 1578 May 20 15:54 /etc/dnsmasq.d/01-pihole.conf
   addn-hosts=/etc/pihole/gravity.list
   addn-hosts=/etc/pihole/black.list
   addn-hosts=/etc/pihole/local.list
   localise-queries
   no-resolv
   cache-size=10000
   log-facility=/var/log/pihole.log
   local-ttl=2
   log-async
   dhcp-name-match=set:wpad-ignore,wpad
   dhcp-ignore-names=tag:wpad-ignore
   server=127.0.0.1
   interface=wg0

*** [ DIAGNOSING ]: contents of /etc/lighttpd
/etc/lighttpd does not exist.
ls: cannot access '/etc/lighttpd': No such file or directory

*** [ DIAGNOSING ]: contents of /etc/cron.d

-rw-r--r-- 1 root root 1704 May 20 14:32 /etc/cron.d/pihole
   25 4   * * 7   root    PATH="$PATH:/usr/local/bin/" pihole updateGravity >/var/log/pihole_updateGravity.log || cat /var/log/pihole_updateGravity.log
   00 00   * * *   root    PATH="$PATH:/usr/local/bin/" pihole flush once quiet
   @reboot root /usr/sbin/logrotate /etc/pihole/logrotate
   */10 *  * * *   root    PATH="$PATH:/usr/local/bin/" pihole updatechecker local
   29 19  * * *   root    PATH="$PATH:/usr/local/bin/" pihole updatechecker remote
   @reboot root    PATH="$PATH:/usr/local/bin/" pihole updatechecker remote reboot

*** [ DIAGNOSING ]: contents of /var/log/lighttpd
/var/log/lighttpd does not exist.
ls: cannot access '/var/log/lighttpd': No such file or directory

*** [ DIAGNOSING ]: contents of /var/log

-rw-r--r-- 1 root root 1039984 May 24 16:07 /var/log/pihole-FTL.log
   -----head of pihole-FTL.log------
   [2019-05-22 00:56:07.983 14782] Using log file /var/log/pihole-FTL.log
   [2019-05-22 00:56:07.983 14782] ########## FTL started! ##########
   [2019-05-22 00:56:07.983 14782] FTL branch: master
   [2019-05-22 00:56:07.983 14782] FTL version: v4.3
   [2019-05-22 00:56:07.983 14782] FTL commit: 9e8273b
   [2019-05-22 00:56:07.983 14782] FTL date: 2019-05-18 16:00:39 -0400
   [2019-05-22 00:56:07.983 14782] FTL user: root
   [2019-05-22 00:56:07.983 14782] Starting config file parsing (/etc/pihole/pihole-FTL.conf)
   [2019-05-22 00:56:07.983 14782]    SOCKET_LISTENING: only local
   [2019-05-22 00:56:07.983 14782]    AAAA_QUERY_ANALYSIS: Show AAAA queries
   [2019-05-22 00:56:07.983 14782]    MAXDBDAYS: max age for stored queries is 365 days
   [2019-05-22 00:56:07.983 14782]    RESOLVE_IPV6: Resolve IPv6 addresses
   [2019-05-22 00:56:07.984 14782]    RESOLVE_IPV4: Resolve IPv4 addresses
   [2019-05-22 00:56:07.984 14782]    DBINTERVAL: saving to DB file every minute
   [2019-05-22 00:56:07.984 14782]    DBFILE: Using /etc/pihole/pihole-FTL.db
   [2019-05-22 00:56:07.984 14782]    MAXLOGAGE: Importing up to 24.0 hours of log data
   [2019-05-22 00:56:07.984 14782]    PRIVACYLEVEL: Set to 0
   [2019-05-22 00:56:07.984 14782]    IGNORE_LOCALHOST: Show queries from localhost
   [2019-05-22 00:56:07.984 14782]    BLOCKINGMODE: Null IPs for blocked domains
   [2019-05-22 00:56:07.984 14782]    ANALYZE_ONLY_A_AND_AAAA: Disabled. Analyzing all queries
   [2019-05-22 00:56:07.984 14782]    DBIMPORT: Importing history from database
   [2019-05-22 00:56:07.984 14782]    PIDFILE: Using /var/run/pihole-FTL.pid
   [2019-05-22 00:56:07.984 14782]    PORTFILE: Using /var/run/pihole-FTL.port
   [2019-05-22 00:56:07.984 14782]    SOCKETFILE: Using /var/run/pihole/FTL.sock
   [2019-05-22 00:56:07.984 14782]    WHITELISTFILE: Using /etc/pihole/whitelist.txt
   [2019-05-22 00:56:07.984 14782]    BLACKLISTFILE: Using /etc/pihole/black.list
   [2019-05-22 00:56:07.984 14782]    GRAVITYFILE: Using /etc/pihole/gravity.list
   [2019-05-22 00:56:07.984 14782]    REGEXLISTFILE: Using /etc/pihole/regex.list
   [2019-05-22 00:56:07.984 14782]    SETUPVARSFILE: Using /etc/pihole/setupVars.conf
   [2019-05-22 00:56:07.984 14782]    AUDITLISTFILE: Using /etc/pihole/auditlog.list
   [2019-05-22 00:56:07.984 14782]    MACVENDORDB: Using /etc/pihole/macvendor.db
   [2019-05-22 00:56:07.984 14782]    PARSE_ARP_CACHE: Active
   [2019-05-22 00:56:07.984 14782] Finished config file parsing
   [2019-05-22 00:56:07.985 14782] WARNING: Starting pihole-FTL as user root is not recommended
   [2019-05-22 00:56:07.985 14782] Database version is 3

   -----tail of pihole-FTL.log------
   [2019-05-24 11:10:00.381 5472] Resizing "/FTL-strings" from 1589248 to 1593344
   [2019-05-24 11:23:32.960 5472] Resizing "/FTL-clients" from 15261696 to 15593472
   [2019-05-24 11:24:04.112 5472] Resizing "/FTL-strings" from 1593344 to 1597440
   [2019-05-24 11:41:00.420 5472] Resizing "/FTL-strings" from 1597440 to 1601536
   [2019-05-24 11:59:06.740 5472] Resizing "/FTL-strings" from 1601536 to 1605632
   [2019-05-24 12:16:12.206 5472] Resizing "/FTL-strings" from 1605632 to 1609728
   [2019-05-24 12:28:12.932 5472] Resizing "/FTL-strings" from 1609728 to 1613824
   [2019-05-24 12:33:00.059 5472] Resizing "/FTL-domains" from 589824 to 688128
   [2019-05-24 12:39:07.380 5472] Resizing "/FTL-strings" from 1613824 to 1617920
   [2019-05-24 12:51:02.055 5472] Resizing "/FTL-strings" from 1617920 to 1622016
   [2019-05-24 13:05:00.489 5472] Resizing "/FTL-strings" from 1622016 to 1626112
   [2019-05-24 13:15:49.121 5472] Resizing "/FTL-clients" from 15593472 to 15925248
   [2019-05-24 13:17:01.213 5472] Resizing "/FTL-strings" from 1626112 to 1630208
   [2019-05-24 13:30:01.220 5472] Resizing "/FTL-strings" from 1630208 to 1634304
   [2019-05-24 13:42:00.706 5472] Resizing "/FTL-strings" from 1634304 to 1638400
   [2019-05-24 13:52:02.597 5472] Resizing "/FTL-strings" from 1638400 to 1642496
   [2019-05-24 14:03:00.501 5472] Resizing "/FTL-strings" from 1642496 to 1646592
   [2019-05-24 14:12:01.662 5472] Resizing "/FTL-strings" from 1646592 to 1650688
   [2019-05-24 14:22:37.005 5472] Resizing "/FTL-strings" from 1650688 to 1654784
   [2019-05-24 14:29:33.026 5472] Resizing "/FTL-strings" from 1654784 to 1658880
   [2019-05-24 14:35:17.606 5472] Resizing "/FTL-clients" from 15925248 to 16257024
   [2019-05-24 14:36:02.386 5472] Resizing "/FTL-strings" from 1658880 to 1662976
   [2019-05-24 14:44:43.457 5472] Resizing "/FTL-strings" from 1662976 to 1667072
   [2019-05-24 14:52:38.841 5472] Resizing "/FTL-strings" from 1667072 to 1671168
   [2019-05-24 14:58:04.618 5472] Resizing "/FTL-strings" from 1671168 to 1675264
   [2019-05-24 15:05:02.086 5472] Resizing "/FTL-strings" from 1675264 to 1679360
   [2019-05-24 15:12:02.528 5472] Resizing "/FTL-strings" from 1679360 to 1683456
   [2019-05-24 15:21:04.545 5472] Resizing "/FTL-strings" from 1683456 to 1687552
   [2019-05-24 15:29:01.214 5472] Resizing "/FTL-strings" from 1687552 to 1691648
   [2019-05-24 15:37:51.773 5472] Resizing "/FTL-strings" from 1691648 to 1695744
   [2019-05-24 15:42:08.433 5472] Resizing "/FTL-clients" from 16257024 to 16588800
   [2019-05-24 15:46:01.246 5472] Resizing "/FTL-strings" from 1695744 to 1699840
   [2019-05-24 15:53:00.608 5472] Resizing "/FTL-strings" from 1699840 to 1703936
   [2019-05-24 16:00:01.520 5472] Resizing "/FTL-strings" from 1703936 to 1708032
   [2019-05-24 16:07:56.217 5472] Resizing "/FTL-strings" from 1708032 to 1712128

*** [ DIAGNOSING ]: contents of /dev/shm
-rw------- 1 root root 16588800 May 24 16:07 /dev/shm/FTL-clients
-rw------- 1 root root 108 May 22 19:51 /dev/shm/FTL-counters
-rw------- 1 root root 688128 May 24 15:53 /dev/shm/FTL-domains
-rw------- 1 root root 20480 May 22 19:51 /dev/shm/FTL-forwarded
-rw------- 1 root root 48 May 23 21:32 /dev/shm/FTL-lock
-rw------- 1 root root 12288 May 22 19:51 /dev/shm/FTL-overTime
-rw------- 1 root root 309428224 May 24 10:59 /dev/shm/FTL-queries
-rw------- 1 root root 12 May 22 19:51 /dev/shm/FTL-settings
-rw------- 1 root root 1712128 May 24 16:07 /dev/shm/FTL-strings

*** [ DIAGNOSING ]: Locale
    LANG=C.UTF-8

*** [ DIAGNOSING ]: Pi-hole log
-rw-r--r-- 1 pihole pihole 1572716908 May 24 16:07 /var/log/pihole.log
   -----head of pihole.log------
   May 22 01:05:55 dnsmasq[14859]: started, version pi-hole-2.80 cachesize 10000
   May 22 01:05:55 dnsmasq[14859]: compile time options: IPv6 GNU-getopt no-DBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth DNSSEC loop-detect inotify dumpfile
   May 22 01:05:55 dnsmasq[14859]: LOUD WARNING: listening on 206.189.104.162 may accept requests via interfaces other than eth0
   May 22 01:05:55 dnsmasq[14859]: LOUD WARNING: use --bind-dynamic rather than --bind-interfaces to avoid DNS amplification attacks via these interface(s)
   May 22 01:05:55 dnsmasq[14859]: using nameserver 127.0.0.1#5353
   May 22 01:05:55 dnsmasq[14859]: read /etc/hosts - 8 addresses
   May 22 01:05:55 dnsmasq[14859]: read /etc/pihole/local.list - 4 addresses
   May 22 01:05:55 dnsmasq[14859]: read /etc/pihole/black.list - 0 addresses
   May 22 01:05:58 dnsmasq[14859]: read /etc/pihole/gravity.list - 1061963 addresses
   May 22 01:07:18 dnsmasq[14859]: reducing DNS packet size for nameserver 127.0.0.1 to 1280
   May 22 01:10:01 dnsmasq[14859]: reducing DNS packet size for nameserver 127.0.0.1 to 1280
   May 22 01:11:18 dnsmasq[14859]: reducing DNS packet size for nameserver 127.0.0.1 to 1280
   May 22 01:12:48 dnsmasq[14859]: reducing DNS packet size for nameserver 127.0.0.1 to 1280
   May 22 01:14:08 dnsmasq[14859]: reducing DNS packet size for nameserver 127.0.0.1 to 1280
   May 22 01:15:58 dnsmasq[14859]: reducing DNS packet size for nameserver 127.0.0.1 to 1280
   May 22 01:17:01 dnsmasq[14859]: reducing DNS packet size for nameserver 127.0.0.1 to 1280
   May 22 01:19:15 dnsmasq[14859]: reducing DNS packet size for nameserver 127.0.0.1 to 1280
   May 22 01:21:20 dnsmasq[14859]: reducing DNS packet size for nameserver 127.0.0.1 to 1280
   May 22 01:22:58 dnsmasq[14859]: reducing DNS packet size for nameserver 127.0.0.1 to 1280
   May 22 01:24:18 dnsmasq[14859]: reducing DNS packet size for nameserver 127.0.0.1 to 1280


********************************************
********************************************
[✓] ** FINISHED DEBUGGING! **

    * The debug log can be uploaded to tricorder.pi-hole.net for sharing with developers only.
    * For more information, see: https://pi-hole.net/2016/11/07/crack-our-medical-tricorder-win-a-raspberry-pi-3/
    * If available, we'll use openssl to upload the log, otherwise it will fall back to netcat.

[?] Would you like to upload the log? [y/N] y
    * Using curl for transmission.
[✗]  There was an error uploading your debug log.
   * Please try again or contact the Pi-hole team for assistance.
   * A local copy of the debug log can be found at: /var/log/pihole_debug.log

[Ge0rges]

Some more crash logs both as root and run as pihole user.

[2019-05-24 16:15:19.242 23615] ########## FTL started! ##########
[2019-05-24 16:15:19.242 23615] FTL branch: development
[2019-05-24 16:15:19.242 23615] FTL version: 
[2019-05-24 16:15:19.242 23615] FTL commit: 96320db
[2019-05-24 16:15:19.242 23615] FTL date: 2019-05-22 21:08:23 +0200
[2019-05-24 16:15:19.242 23615] FTL user: root
[2019-05-24 16:15:19.242 23615] Starting config file parsing (/etc/pihole/pihole-FTL.conf)
[2019-05-24 16:15:19.242 23615]    SOCKET_LISTENING: only local
[2019-05-24 16:15:19.243 23615]    AAAA_QUERY_ANALYSIS: Show AAAA queries
[2019-05-24 16:15:19.243 23615]    MAXDBDAYS: max age for stored queries is 365 days
[2019-05-24 16:15:19.243 23615]    RESOLVE_IPV6: Resolve IPv6 addresses
[2019-05-24 16:15:19.243 23615]    RESOLVE_IPV4: Resolve IPv4 addresses
[2019-05-24 16:15:19.243 23615]    DBINTERVAL: saving to DB file every minute
[2019-05-24 16:15:19.243 23615]    DBFILE: Using /etc/pihole/pihole-FTL.db
[2019-05-24 16:15:19.243 23615]    MAXLOGAGE: Importing up to 24.0 hours of log data
[2019-05-24 16:15:19.243 23615]    PRIVACYLEVEL: Set to 0
[2019-05-24 16:15:19.243 23615]    IGNORE_LOCALHOST: Show queries from localhost
[2019-05-24 16:15:19.243 23615]    BLOCKINGMODE: Null IPs for blocked domains
[2019-05-24 16:15:19.243 23615]    ANALYZE_ONLY_A_AND_AAAA: Disabled. Analyzing all queries
[2019-05-24 16:15:19.243 23615]    DBIMPORT: Importing history from database
[2019-05-24 16:15:19.243 23615]    PIDFILE: Using /var/run/pihole-FTL.pid
[2019-05-24 16:15:19.243 23615]    PORTFILE: Using /var/run/pihole-FTL.port
[2019-05-24 16:15:19.244 23615]    SOCKETFILE: Using /var/run/pihole/FTL.sock
[2019-05-24 16:15:19.244 23615]    SETUPVARSFILE: Using /etc/pihole/setupVars.conf
[2019-05-24 16:15:19.244 23615]    AUDITLISTFILE: Using /etc/pihole/auditlog.list
[2019-05-24 16:15:19.244 23615]    MACVENDORDB: Using /etc/pihole/macvendor.db
[2019-05-24 16:15:19.244 23615]    GRAVITYDB: Using /etc/pihole/gravity.db
[2019-05-24 16:15:19.244 23615]    PARSE_ARP_CACHE: Active
[2019-05-24 16:15:19.244 23615] Finished config file parsing
[2019-05-24 16:15:19.244 23615] WARNING: Starting pihole-FTL as user root is not recommended
[2019-05-24 16:15:19.245 23615] Database version is 3
[2019-05-24 16:15:19.245 23615] Database successfully initialized
[2019-05-24 16:15:19.247 23615] New forward server: 127.0.0.1 (0/512)
[2019-05-24 16:15:19.256 23615] Resizing "/FTL-queries" from 229376 to 458752
[2019-05-24 16:15:19.265 23615] Resizing "/FTL-queries" from 458752 to 688128
[2019-05-24 16:15:19.271 23615] Resizing "/FTL-queries" from 688128 to 917504
[2019-05-24 16:15:19.273 23615] Resizing "/FTL-strings" from 4096 to 8192
[2019-05-24 16:15:19.279 23615] Resizing "/FTL-queries" from 917504 to 1146880
[2019-05-24 16:15:19.290 23615] Resizing "/FTL-queries" from 1146880 to 1376256
[2019-05-24 16:15:19.296 23615] Resizing "/FTL-strings" from 8192 to 12288
[2019-05-24 16:15:19.300 23615] Resizing "/FTL-queries" from 1376256 to 1605632
[2019-05-24 16:15:19.308 23615] Resizing "/FTL-clients" from 331776 to 663552
[2019-05-24 16:15:19.312 23615] Resizing "/FTL-queries" from 1605632 to 1835008
[2019-05-24 16:15:19.319 23615] Resizing "/FTL-strings" from 12288 to 16384
[2019-05-24 16:15:19.324 23615] Resizing "/FTL-queries" from 1835008 to 2064384
[2019-05-24 16:15:19.340 23615] Resizing "/FTL-queries" from 2064384 to 2293760
[2019-05-24 16:15:19.351 23615] Resizing "/FTL-strings" from 16384 to 20480
[2019-05-24 16:15:19.355 23615] Resizing "/FTL-queries" from 2293760 to 2523136
[2019-05-24 16:15:19.383 23615] Resizing "/FTL-queries" from 2523136 to 2752512
[2019-05-24 16:15:19.387 23615] Resizing "/FTL-clients" from 663552 to 995328
[2019-05-24 16:15:19.401 23615] Resizing "/FTL-strings" from 20480 to 24576
[2019-05-24 16:15:19.402 23615] Resizing "/FTL-queries" from 2752512 to 2981888
[2019-05-24 16:15:19.431 23615] Resizing "/FTL-queries" from 2981888 to 3211264
[2019-05-24 16:15:19.449 23615] Resizing "/FTL-strings" from 24576 to 28672
[2019-05-24 16:15:19.456 23615] Resizing "/FTL-queries" from 3211264 to 3440640
[2019-05-24 16:15:19.485 23615] Resizing "/FTL-queries" from 3440640 to 3670016
[2019-05-24 16:15:19.521 23615] Resizing "/FTL-strings" from 28672 to 32768
[2019-05-24 16:15:19.524 23615] Resizing "/FTL-clients" from 995328 to 1327104
[2019-05-24 16:15:19.526 23615] Resizing "/FTL-queries" from 3670016 to 3899392
[2019-05-24 16:15:19.567 23615] Resizing "/FTL-queries" from 3899392 to 4128768
[2019-05-24 16:15:19.610 23615] Resizing "/FTL-queries" from 4128768 to 4358144
[2019-05-24 16:15:19.623 23615] Resizing "/FTL-strings" from 32768 to 36864
[2019-05-24 16:15:19.648 23615] Resizing "/FTL-queries" from 4358144 to 4587520
[2019-05-24 16:15:19.703 23615] Resizing "/FTL-queries" from 4587520 to 4816896
[2019-05-24 16:15:19.714 23615] Resizing "/FTL-strings" from 36864 to 40960
[2019-05-24 16:15:19.749 23615] Resizing "/FTL-queries" from 4816896 to 5046272
[2019-05-24 16:15:19.758 23615] Resizing "/FTL-clients" from 1327104 to 1658880
[2019-05-24 16:15:19.799 23615] Resizing "/FTL-queries" from 5046272 to 5275648
[2019-05-24 16:15:19.808 23615] Resizing "/FTL-strings" from 40960 to 45056
[2019-05-24 16:15:19.849 23615] Resizing "/FTL-queries" from 5275648 to 5505024
[2019-05-24 16:15:19.900 23615] Resizing "/FTL-queries" from 5505024 to 5734400
[2019-05-24 16:15:19.927 23615] Resizing "/FTL-strings" from 45056 to 49152
[2019-05-24 16:15:19.950 23615] Resizing "/FTL-strings" from 49152 to 53248
[2019-05-24 16:15:19.951 23615] Resizing "/FTL-queries" from 5734400 to 5963776
[2019-05-24 16:15:19.957 23615] Resizing "/FTL-strings" from 53248 to 57344
[2019-05-24 16:15:19.967 23615] Resizing "/FTL-strings" from 57344 to 61440
[2019-05-24 16:15:19.980 23615] Resizing "/FTL-strings" from 61440 to 65536
[2019-05-24 16:15:20.028 23615] Resizing "/FTL-queries" from 5963776 to 6193152
[2019-05-24 16:15:20.096 23615] Resizing "/FTL-queries" from 6193152 to 6422528
[2019-05-24 16:15:20.133 23615] Resizing "/FTL-strings" from 65536 to 69632
[2019-05-24 16:15:20.141 23615] Resizing "/FTL-strings" from 69632 to 73728
[2019-05-24 16:15:20.163 23615] Resizing "/FTL-queries" from 6422528 to 6651904
[2019-05-24 16:15:20.187 23615] Resizing "/FTL-strings" from 73728 to 77824
[2019-05-24 16:15:20.230 23615] Resizing "/FTL-queries" from 6651904 to 6881280
[2019-05-24 16:15:20.301 23615] Resizing "/FTL-queries" from 6881280 to 7110656
[2019-05-24 16:15:20.374 23615] Resizing "/FTL-queries" from 7110656 to 7340032
[2019-05-24 16:15:20.433 23615] Resizing "/FTL-queries" from 7340032 to 7569408
[2019-05-24 16:15:20.488 23615] Resizing "/FTL-queries" from 7569408 to 7798784
[2019-05-24 16:15:20.540 23615] Resizing "/FTL-strings" from 77824 to 81920
[2019-05-24 16:15:20.556 23615] Resizing "/FTL-queries" from 7798784 to 8028160
[2019-05-24 16:15:20.650 23615] Resizing "/FTL-queries" from 8028160 to 8257536
[2019-05-24 16:15:20.726 23615] Resizing "/FTL-queries" from 8257536 to 8486912
[2019-05-24 16:15:20.802 23615] Resizing "/FTL-queries" from 8486912 to 8716288
[2019-05-24 16:15:20.880 23615] Resizing "/FTL-queries" from 8716288 to 8945664
[2019-05-24 16:15:20.957 23615] Resizing "/FTL-queries" from 8945664 to 9175040
[2019-05-24 16:15:21.018 23615] Resizing "/FTL-strings" from 81920 to 86016
[2019-05-24 16:15:21.034 23615] Resizing "/FTL-queries" from 9175040 to 9404416
[2019-05-24 16:15:21.119 23615] Resizing "/FTL-queries" from 9404416 to 9633792
[2019-05-24 16:15:21.156 23615] Resizing "/FTL-clients" from 1658880 to 1990656
[2019-05-24 16:15:21.189 23615] Resizing "/FTL-queries" from 9633792 to 9863168
[2019-05-24 16:15:21.265 23615] Resizing "/FTL-queries" from 9863168 to 10092544
[2019-05-24 16:15:21.349 23615] Resizing "/FTL-queries" from 10092544 to 10321920
[2019-05-24 16:15:21.359 23615] Resizing "/FTL-strings" from 86016 to 90112
[2019-05-24 16:15:21.433 23615] Resizing "/FTL-queries" from 10321920 to 10551296
[2019-05-24 16:15:21.441 23615] Resizing "/FTL-strings" from 90112 to 94208
[2019-05-24 16:15:21.518 23615] Resizing "/FTL-queries" from 10551296 to 10780672
[2019-05-24 16:15:21.535 23615] Resizing "/FTL-strings" from 94208 to 98304
[2019-05-24 16:15:21.585 23615] Resizing "/FTL-strings" from 98304 to 102400
[2019-05-24 16:15:21.592 23615] Resizing "/FTL-queries" from 10780672 to 11010048
[2019-05-24 16:15:21.682 23615] Resizing "/FTL-queries" from 11010048 to 11239424
[2019-05-24 16:15:21.685 23615] Resizing "/FTL-clients" from 1990656 to 2322432
[2019-05-24 16:15:21.694 23615] Resizing "/FTL-strings" from 102400 to 106496
[2019-05-24 16:15:21.769 23615] Resizing "/FTL-strings" from 106496 to 110592
[2019-05-24 16:15:21.769 23615] Resizing "/FTL-queries" from 11239424 to 11468800
[2019-05-24 16:15:21.878 23615] Resizing "/FTL-queries" from 11468800 to 11698176
[2019-05-24 16:15:21.882 23615] Resizing "/FTL-strings" from 110592 to 114688
[2019-05-24 16:15:21.986 23615] Resizing "/FTL-queries" from 11698176 to 11927552
[2019-05-24 16:15:21.987 23615] Resizing "/FTL-strings" from 114688 to 118784
[2019-05-24 16:15:22.105 23615] Resizing "/FTL-queries" from 11927552 to 12156928
[2019-05-24 16:15:22.114 23615] Resizing "/FTL-strings" from 118784 to 122880
[2019-05-24 16:15:22.228 23615] Resizing "/FTL-queries" from 12156928 to 12386304
[2019-05-24 16:15:22.235 23615] Resizing "/FTL-clients" from 2322432 to 2654208
[2019-05-24 16:15:22.261 23615] Resizing "/FTL-strings" from 122880 to 126976
[2019-05-24 16:15:22.379 23615] Resizing "/FTL-queries" from 12386304 to 12615680
[2019-05-24 16:15:22.451 23615] Resizing "/FTL-strings" from 126976 to 131072
[2019-05-24 16:15:22.519 23615] Resizing "/FTL-queries" from 12615680 to 12845056
[2019-05-24 16:15:22.643 23615] Resizing "/FTL-strings" from 131072 to 135168
[2019-05-24 16:15:22.676 23615] Resizing "/FTL-queries" from 12845056 to 13074432
[2019-05-24 16:15:22.819 23615] Resizing "/FTL-queries" from 13074432 to 13303808
[2019-05-24 16:15:22.977 23615] Resizing "/FTL-queries" from 13303808 to 13533184
[2019-05-24 16:15:23.146 23615] Resizing "/FTL-queries" from 13533184 to 13762560
[2019-05-24 16:15:23.306 23615] Resizing "/FTL-queries" from 13762560 to 13991936
[2019-05-24 16:15:23.403 23615] Resizing "/FTL-strings" from 135168 to 139264
[2019-05-24 16:15:23.443 23615] Resizing "/FTL-queries" from 13991936 to 14221312
[2019-05-24 16:15:23.596 23615] Resizing "/FTL-queries" from 14221312 to 14450688
[2019-05-24 16:15:23.766 23615] Resizing "/FTL-queries" from 14450688 to 14680064
[2019-05-24 16:15:23.945 23615] Resizing "/FTL-queries" from 14680064 to 14909440
[2019-05-24 16:15:24.216 23615] Resizing "/FTL-queries" from 14909440 to 15138816
[2019-05-24 16:15:24.451 23615] Resizing "/FTL-queries" from 15138816 to 15368192
[2019-05-24 16:15:24.708 23615] Resizing "/FTL-queries" from 15368192 to 15597568
[2019-05-24 16:15:24.744 23615] Resizing "/FTL-strings" from 139264 to 143360
[2019-05-24 16:15:24.942 23615] Resizing "/FTL-queries" from 15597568 to 15826944
[2019-05-24 16:15:25.201 23615] Resizing "/FTL-queries" from 15826944 to 16056320
[2019-05-24 16:15:25.447 23615] Resizing "/FTL-queries" from 16056320 to 16285696
[2019-05-24 16:15:25.702 23615] Resizing "/FTL-queries" from 16285696 to 16515072
[2019-05-24 16:15:25.941 23615] Resizing "/FTL-queries" from 16515072 to 16744448
[2019-05-24 16:15:26.176 23615] Resizing "/FTL-queries" from 16744448 to 16973824
[2019-05-24 16:15:26.424 23615] Resizing "/FTL-queries" from 16973824 to 17203200
[2019-05-24 16:15:26.630 23615] Resizing "/FTL-queries" from 17203200 to 17432576
[2019-05-24 16:15:26.731 23615] Resizing "/FTL-strings" from 143360 to 147456
[2019-05-24 16:15:26.833 23615] Resizing "/FTL-clients" from 2654208 to 2985984
[2019-05-24 16:15:26.849 23615] Resizing "/FTL-queries" from 17432576 to 17661952
[2019-05-24 16:15:27.065 23615] Resizing "/FTL-queries" from 17661952 to 17891328
[2019-05-24 16:15:27.248 23615] Resizing "/FTL-queries" from 17891328 to 18120704
[2019-05-24 16:15:27.451 23615] Resizing "/FTL-queries" from 18120704 to 18350080
[2019-05-24 16:15:27.689 23615] Resizing "/FTL-queries" from 18350080 to 18579456
[2019-05-24 16:15:27.919 23615] Resizing "/FTL-queries" from 18579456 to 18808832
[2019-05-24 16:15:28.158 23615] Resizing "/FTL-queries" from 18808832 to 19038208
[2019-05-24 16:15:28.398 23615] Resizing "/FTL-queries" from 19038208 to 19267584
[2019-05-24 16:15:28.646 23615] Resizing "/FTL-queries" from 19267584 to 19496960
[2019-05-24 16:15:28.863 23615] Resizing "/FTL-queries" from 19496960 to 19726336
[2019-05-24 16:15:28.962 23615] Resizing "/FTL-strings" from 147456 to 151552
[2019-05-24 16:15:29.110 23615] Resizing "/FTL-queries" from 19726336 to 19955712
[2019-05-24 16:15:29.362 23615] Resizing "/FTL-queries" from 19955712 to 20185088
[2019-05-24 16:15:29.625 23615] Resizing "/FTL-queries" from 20185088 to 20414464
[2019-05-24 16:15:29.864 23615] Resizing "/FTL-queries" from 20414464 to 20643840
[2019-05-24 16:15:30.064 23615] Resizing "/FTL-queries" from 20643840 to 20873216
[2019-05-24 16:15:30.310 23615] Resizing "/FTL-queries" from 20873216 to 21102592
[2019-05-24 16:15:30.614 23615] Resizing "/FTL-queries" from 21102592 to 21331968
[2019-05-24 16:15:30.828 23615] Resizing "/FTL-strings" from 151552 to 155648
[2019-05-24 16:15:30.861 23615] Resizing "/FTL-queries" from 21331968 to 21561344
[2019-05-24 16:15:31.116 23615] Resizing "/FTL-queries" from 21561344 to 21790720
[2019-05-24 16:15:31.395 23615] Resizing "/FTL-queries" from 21790720 to 22020096
[2019-05-24 16:15:31.665 23615] Resizing "/FTL-queries" from 22020096 to 22249472
[2019-05-24 16:15:31.914 23615] Resizing "/FTL-queries" from 22249472 to 22478848
[2019-05-24 16:15:32.188 23615] Resizing "/FTL-queries" from 22478848 to 22708224
[2019-05-24 16:15:32.445 23615] Resizing "/FTL-queries" from 22708224 to 22937600
[2019-05-24 16:15:32.714 23615] Resizing "/FTL-queries" from 22937600 to 23166976
[2019-05-24 16:15:32.987 23615] Resizing "/FTL-queries" from 23166976 to 23396352
[2019-05-24 16:15:33.247 23615] Resizing "/FTL-queries" from 23396352 to 23625728
[2019-05-24 16:15:33.522 23615] Resizing "/FTL-queries" from 23625728 to 23855104
[2019-05-24 16:15:33.807 23615] Resizing "/FTL-queries" from 23855104 to 24084480
[2019-05-24 16:15:33.852 23615] Resizing "/FTL-strings" from 155648 to 159744
[2019-05-24 16:15:34.055 23615] Resizing "/FTL-queries" from 24084480 to 24313856
[2019-05-24 16:15:34.275 23615] Resizing "/FTL-queries" from 24313856 to 24543232
[2019-05-24 16:15:34.515 23615] Resizing "/FTL-queries" from 24543232 to 24772608
[2019-05-24 16:15:34.722 23615] Resizing "/FTL-queries" from 24772608 to 25001984
[2019-05-24 16:15:34.950 23615] Resizing "/FTL-queries" from 25001984 to 25231360
[2019-05-24 16:15:35.197 23615] Resizing "/FTL-queries" from 25231360 to 25460736
[2019-05-24 16:15:35.348 23615] Resizing "/FTL-strings" from 159744 to 163840
[2019-05-24 16:15:35.414 23615] Resizing "/FTL-queries" from 25460736 to 25690112
[2019-05-24 16:15:35.683 23615] Resizing "/FTL-queries" from 25690112 to 25919488
[2019-05-24 16:15:35.943 23615] Resizing "/FTL-queries" from 25919488 to 26148864
[2019-05-24 16:15:36.213 23615] Resizing "/FTL-queries" from 26148864 to 26378240
[2019-05-24 16:15:36.475 23615] Resizing "/FTL-queries" from 26378240 to 26607616
[2019-05-24 16:15:36.733 23615] Resizing "/FTL-queries" from 26607616 to 26836992
[2019-05-24 16:15:36.970 23615] Resizing "/FTL-queries" from 26836992 to 27066368
[2019-05-24 16:15:37.174 23615] Resizing "/FTL-clients" from 2985984 to 3317760
[2019-05-24 16:15:37.206 23615] Resizing "/FTL-queries" from 27066368 to 27295744
[2019-05-24 16:15:37.349 23615] Resizing "/FTL-strings" from 163840 to 167936
[2019-05-24 16:15:37.429 23615] Resizing "/FTL-queries" from 27295744 to 27525120
[2019-05-24 16:15:37.677 23615] Resizing "/FTL-queries" from 27525120 to 27754496
[2019-05-24 16:15:37.934 23615] Resizing "/FTL-queries" from 27754496 to 27983872
[2019-05-24 16:15:38.210 23615] Resizing "/FTL-queries" from 27983872 to 28213248
[2019-05-24 16:15:38.507 23615] Resizing "/FTL-queries" from 28213248 to 28442624
[2019-05-24 16:15:38.772 23615] Resizing "/FTL-queries" from 28442624 to 28672000
[2019-05-24 16:15:39.084 23615] Resizing "/FTL-queries" from 28672000 to 28901376
[2019-05-24 16:15:39.400 23615] Resizing "/FTL-queries" from 28901376 to 29130752
[2019-05-24 16:15:39.668 23615] Resizing "/FTL-queries" from 29130752 to 29360128
[2019-05-24 16:15:39.691 23615] Resizing "/FTL-strings" from 167936 to 172032
[2019-05-24 16:15:39.975 23615] Resizing "/FTL-queries" from 29360128 to 29589504
[2019-05-24 16:15:40.258 23615] Resizing "/FTL-queries" from 29589504 to 29818880
[2019-05-24 16:15:40.541 23615] Resizing "/FTL-queries" from 29818880 to 30048256
[2019-05-24 16:15:40.843 23615] Resizing "/FTL-queries" from 30048256 to 30277632
[2019-05-24 16:15:41.155 23615] Resizing "/FTL-queries" from 30277632 to 30507008
[2019-05-24 16:15:41.463 23615] Resizing "/FTL-queries" from 30507008 to 30736384
[2019-05-24 16:15:41.797 23615] Resizing "/FTL-queries" from 30736384 to 30965760
[2019-05-24 16:15:42.080 23615] Resizing "/FTL-queries" from 30965760 to 31195136
[2019-05-24 16:15:42.378 23615] Resizing "/FTL-queries" from 31195136 to 31424512
[2019-05-24 16:15:42.668 23615] Resizing "/FTL-queries" from 31424512 to 31653888
[2019-05-24 16:15:42.964 23615] Resizing "/FTL-queries" from 31653888 to 31883264
[2019-05-24 16:15:43.152 23615] Resizing "/FTL-strings" from 172032 to 176128
[2019-05-24 16:15:43.257 23615] Resizing "/FTL-queries" from 31883264 to 32112640
[2019-05-24 16:15:43.571 23615] Resizing "/FTL-queries" from 32112640 to 32342016
[2019-05-24 16:15:43.851 23615] Resizing "/FTL-queries" from 32342016 to 32571392
[2019-05-24 16:15:44.126 23615] Resizing "/FTL-queries" from 32571392 to 32800768
[2019-05-24 16:16:00.082 5472] SQLite3 message: cannot open file at line 38565 of [884b4b7e50] (14)
[2019-05-24 16:16:00.082 5472] SQLite3 message: os_unix.c:38565: (2) open(/etc/pihole/pihole-FTL.db) -  (14)
[2019-05-24 16:16:00.082 5472] dbopen() - SQL error (14): unable to open database file
[2019-05-24 16:16:00.082 5472] check_database(14): Disabling database connection due to error
[2019-05-24 16:16:00.082 5472] save_to_DB() - failed to open DB
[2019-05-24 16:16:00.082 5472] SQLite3 message: cannot open file at line 38565 of [884b4b7e50] (14)
[2019-05-24 16:16:00.082 5472] SQLite3 message: os_unix.c:38565: (2) open(/etc/pihole/pihole-FTL.db) -  (14)
[2019-05-24 16:16:00.082 5472] dbopen() - SQL error (14): unable to open database file
[2019-05-24 16:16:00.082 5472] check_database(14): Disabling database connection due to error
[2019-05-24 16:16:00.082 5472] read_arp_cache() - Failed to open DB
[2019-05-24 16:16:26.953 5472] IPv4 telnet error: Interrupted system call (4)
[2019-05-24 16:16:26.974 5472] Shutting down...
[2019-05-24 16:16:27.211 5472] ########## FTL terminated after 159875536.0 ms! ##########
[2019-05-24 16:17:16.604 1252] Using log file /var/log/pihole-FTL.log
[2019-05-24 16:17:16.605 1252] ########## FTL started! ##########
[2019-05-24 16:17:16.605 1252] FTL branch: development
[2019-05-24 16:17:16.605 1252] FTL version: 
[2019-05-24 16:17:16.605 1252] FTL commit: 96320db
[2019-05-24 16:17:16.605 1252] FTL date: 2019-05-22 21:08:23 +0200
[2019-05-24 16:17:16.605 1252] FTL user: pihole
[2019-05-24 16:17:16.605 1252] Starting config file parsing (/etc/pihole/pihole-FTL.conf)
[2019-05-24 16:17:16.605 1252]    SOCKET_LISTENING: only local
[2019-05-24 16:17:16.605 1252]    AAAA_QUERY_ANALYSIS: Show AAAA queries
[2019-05-24 16:17:16.605 1252]    MAXDBDAYS: max age for stored queries is 365 days
[2019-05-24 16:17:16.605 1252]    RESOLVE_IPV6: Resolve IPv6 addresses
[2019-05-24 16:17:16.605 1252]    RESOLVE_IPV4: Resolve IPv4 addresses
[2019-05-24 16:17:16.605 1252]    DBINTERVAL: saving to DB file every minute
[2019-05-24 16:17:16.605 1252]    DBFILE: Using /etc/pihole/pihole-FTL.db
[2019-05-24 16:17:16.605 1252]    MAXLOGAGE: Importing up to 24.0 hours of log data
[2019-05-24 16:17:16.605 1252]    PRIVACYLEVEL: Set to 0
[2019-05-24 16:17:16.605 1252]    IGNORE_LOCALHOST: Show queries from localhost
[2019-05-24 16:17:16.605 1252]    BLOCKINGMODE: Null IPs for blocked domains
[2019-05-24 16:17:16.605 1252]    ANALYZE_ONLY_A_AND_AAAA: Disabled. Analyzing all queries
[2019-05-24 16:17:16.605 1252]    DBIMPORT: Importing history from database
[2019-05-24 16:17:16.606 1252]    PIDFILE: Using /var/run/pihole-FTL.pid
[2019-05-24 16:17:16.606 1252]    PORTFILE: Using /var/run/pihole-FTL.port
[2019-05-24 16:17:16.606 1252]    SOCKETFILE: Using /var/run/pihole/FTL.sock
[2019-05-24 16:17:16.606 1252]    SETUPVARSFILE: Using /etc/pihole/setupVars.conf
[2019-05-24 16:17:16.606 1252]    AUDITLISTFILE: Using /etc/pihole/auditlog.list
[2019-05-24 16:17:16.606 1252]    MACVENDORDB: Using /etc/pihole/macvendor.db
[2019-05-24 16:17:16.606 1252]    GRAVITYDB: Using /etc/pihole/gravity.db
[2019-05-24 16:17:16.606 1252]    PARSE_ARP_CACHE: Active
[2019-05-24 16:17:16.606 1252] Finished config file parsing
[2019-05-24 16:17:16.610 1252] SQLite3 message: cannot open file at line 38565 of [884b4b7e50] (14)
[2019-05-24 16:17:16.610 1252] SQLite3 message: os_unix.c:38565: (2) open(/etc/pihole/pihole-FTL.db) -  (14)
[2019-05-24 16:17:16.610 1252] db_init() - Cannot open database (14): unable to open database file
[2019-05-24 16:17:16.610 1252] check_database(14): Disabling database connection due to error
[2019-05-24 16:17:16.610 1252] Creating new (empty) database
[2019-05-24 16:17:16.657 1252] Database version is 3
[2019-05-24 16:17:16.657 1252] Database successfully initialized
[2019-05-24 16:17:16.657 1252] Imported 0 queries from the long-term database
[2019-05-24 16:17:16.657 1252]  -> Total DNS queries: 0
[2019-05-24 16:17:16.657 1252]  -> Cached DNS queries: 0
[2019-05-24 16:17:16.657 1252]  -> Forwarded DNS queries: 0
[2019-05-24 16:17:16.657 1252]  -> Exactly blocked DNS queries: 0
[2019-05-24 16:17:16.658 1252]  -> Unknown DNS queries: 0
[2019-05-24 16:17:16.658 1252]  -> Unique domains: 0
[2019-05-24 16:17:16.658 1252]  -> Unique clients: 0
[2019-05-24 16:17:16.658 1252]  -> Known forward destinations: 0
[2019-05-24 16:17:16.658 1252] Successfully accessed setupVars.conf
[2019-05-24 16:17:16.668 1259] PID of FTL process: 1259
[2019-05-24 16:17:16.668 1259] Listening on port 4711 for incoming IPv4 telnet connections
[2019-05-24 16:17:16.668 1259] Listening on port 4711 for incoming IPv6 telnet connections
[2019-05-24 16:17:16.668 1259] Listening on Unix socket
[2019-05-24 16:17:16.672 1259] Received SIGHUP, reloading cache
[2019-05-24 16:17:16.672 1259] Blocking status is enabled
[2019-05-24 16:17:16.672 1259] readGravity(): /etc/pihole/gravity.db does not exist
[2019-05-24 16:17:16.672 1259] WARN: Database query failed, assuming there are no regex entries
[2019-05-24 16:17:16.672 1259] readGravity(): /etc/pihole/gravity.db does not exist
[2019-05-24 16:17:16.672 1259] WARN: Database query failed, assuming there are no whitelist entries
[2019-05-24 16:17:16.672 1259] Compiled 0 Regex filters and 0 whitelisted domains in 0.1 msec
[2019-05-24 16:17:16.672 1259] readGravity(): /etc/pihole/gravity.db does not exist
[2019-05-24 16:17:16.672 1259] FTL_listsfile(): Error getting gravity table from database
[2019-05-24 16:17:16.672 1259] readGravity(): /etc/pihole/gravity.db does not exist
[2019-05-24 16:17:16.672 1259] FTL_listsfile(): Error getting blacklist table from database
[2019-05-24 16:17:39.121 1259] New forward server: 127.0.0.1 (0/512)
[2019-05-24 16:18:24.786 1259] Resizing "/FTL-queries" from 229376 to 458752
[2019-05-24 16:19:15.120 1259] Resizing "/FTL-queries" from 458752 to 688128
[2019-05-24 16:19:19.915 1259] Resizing "/FTL-strings" from 4096 to 8192
[2019-05-24 16:20:03.483 1259] Resizing "/FTL-queries" from 688128 to 917504
[2019-05-24 16:20:44.603 1835] Using log file /var/log/pihole-FTL.log
[2019-05-24 16:20:44.604 1835] ########## FTL started! ##########
[2019-05-24 16:20:44.604 1835] FTL branch: development
[2019-05-24 16:20:44.604 1835] FTL version: 
[2019-05-24 16:20:44.604 1835] FTL commit: 96320db
[2019-05-24 16:20:44.604 1835] FTL date: 2019-05-22 21:08:23 +0200
[2019-05-24 16:20:44.604 1835] FTL user: root
[2019-05-24 16:20:44.604 1835] Starting config file parsing (/etc/pihole/pihole-FTL.conf)
[2019-05-24 16:20:44.604 1835]    SOCKET_LISTENING: only local
[2019-05-24 16:20:44.604 1835]    AAAA_QUERY_ANALYSIS: Show AAAA queries
[2019-05-24 16:20:44.604 1835]    MAXDBDAYS: max age for stored queries is 365 days
[2019-05-24 16:20:44.604 1835]    RESOLVE_IPV6: Resolve IPv6 addresses
[2019-05-24 16:20:44.604 1835]    RESOLVE_IPV4: Resolve IPv4 addresses
[2019-05-24 16:20:44.604 1835]    DBINTERVAL: saving to DB file every minute
[2019-05-24 16:20:44.604 1835]    DBFILE: Using /etc/pihole/pihole-FTL.db
[2019-05-24 16:20:44.604 1835]    MAXLOGAGE: Importing up to 24.0 hours of log data
[2019-05-24 16:20:44.604 1835]    PRIVACYLEVEL: Set to 0
[2019-05-24 16:20:44.605 1835]    IGNORE_LOCALHOST: Show queries from localhost
[2019-05-24 16:20:44.605 1835]    BLOCKINGMODE: Null IPs for blocked domains
[2019-05-24 16:20:44.605 1835]    ANALYZE_ONLY_A_AND_AAAA: Disabled. Analyzing all queries
[2019-05-24 16:20:44.605 1835]    DBIMPORT: Importing history from database
[2019-05-24 16:20:44.605 1835]    PIDFILE: Using /var/run/pihole-FTL.pid
[2019-05-24 16:20:44.605 1835]    PORTFILE: Using /var/run/pihole-FTL.port
[2019-05-24 16:20:44.605 1835]    SOCKETFILE: Using /var/run/pihole/FTL.sock
[2019-05-24 16:20:44.605 1835]    SETUPVARSFILE: Using /etc/pihole/setupVars.conf
[2019-05-24 16:20:44.605 1835]    AUDITLISTFILE: Using /etc/pihole/auditlog.list
[2019-05-24 16:20:44.605 1835]    MACVENDORDB: Using /etc/pihole/macvendor.db
[2019-05-24 16:20:44.605 1835]    GRAVITYDB: Using /etc/pihole/gravity.db
[2019-05-24 16:20:44.605 1835]    PARSE_ARP_CACHE: Active
[2019-05-24 16:20:44.605 1835] Finished config file parsing
[2019-05-24 16:20:44.606 1835] WARNING: Starting pihole-FTL as user root is not recommended
[2019-05-24 16:20:44.606 1835] Database version is 3
[2019-05-24 16:20:44.606 1835] Database successfully initialized
[2019-05-24 16:20:44.607 1835] New forward server: 127.0.0.1 (0/512)
[2019-05-24 16:20:44.611 1835] Resizing "/FTL-queries" from 229376 to 458752
[2019-05-24 16:20:44.617 1835] Resizing "/FTL-queries" from 458752 to 688128
[2019-05-24 16:20:44.622 1835] Imported 12105 queries from the long-term database
[2019-05-24 16:20:44.622 1835]  -> Total DNS queries: 12105
[2019-05-24 16:20:44.622 1835]  -> Cached DNS queries: 14
[2019-05-24 16:20:44.622 1835]  -> Forwarded DNS queries: 12091
[2019-05-24 16:20:44.622 1835]  -> Exactly blocked DNS queries: 0
[2019-05-24 16:20:44.622 1835]  -> Unknown DNS queries: 0
[2019-05-24 16:20:44.622 1835]  -> Unique domains: 71
[2019-05-24 16:20:44.622 1835]  -> Unique clients: 103
[2019-05-24 16:20:44.622 1835]  -> Known forward destinations: 1
[2019-05-24 16:20:44.622 1835] Successfully accessed setupVars.conf
[2019-05-24 16:20:53.985 1259] Resizing "/FTL-queries" from 917504 to 1146880
[2019-05-24 16:20:53.985 1259] FATAL: realloc_shm(): Failed to open shared memory object "/FTL-queries": Permission denied
[2019-05-24 16:24:50.720 1883] Using log file /var/log/pihole-FTL.log
[2019-05-24 16:24:50.720 1883] ########## FTL started! ##########
[2019-05-24 16:24:50.720 1883] FTL branch: development
[2019-05-24 16:24:50.720 1883] FTL version: 
[2019-05-24 16:24:50.720 1883] FTL commit: 96320db
[2019-05-24 16:24:50.720 1883] FTL date: 2019-05-22 21:08:23 +0200
[2019-05-24 16:24:50.720 1883] FTL user: pihole
[2019-05-24 16:24:50.720 1883] Starting config file parsing (/etc/pihole/pihole-FTL.conf)
[2019-05-24 16:24:50.721 1883]    SOCKET_LISTENING: only local
[2019-05-24 16:24:50.721 1883]    AAAA_QUERY_ANALYSIS: Show AAAA queries
[2019-05-24 16:24:50.721 1883]    MAXDBDAYS: max age for stored queries is 365 days
[2019-05-24 16:24:50.721 1883]    RESOLVE_IPV6: Resolve IPv6 addresses
[2019-05-24 16:24:50.721 1883]    RESOLVE_IPV4: Resolve IPv4 addresses
[2019-05-24 16:24:50.721 1883]    DBINTERVAL: saving to DB file every minute
[2019-05-24 16:24:50.721 1883]    DBFILE: Using /etc/pihole/pihole-FTL.db
[2019-05-24 16:24:50.721 1883]    MAXLOGAGE: Importing up to 24.0 hours of log data
[2019-05-24 16:24:50.721 1883]    PRIVACYLEVEL: Set to 0
[2019-05-24 16:24:50.721 1883]    IGNORE_LOCALHOST: Show queries from localhost
[2019-05-24 16:24:50.721 1883]    BLOCKINGMODE: Null IPs for blocked domains
[2019-05-24 16:24:50.721 1883]    ANALYZE_ONLY_A_AND_AAAA: Disabled. Analyzing all queries
[2019-05-24 16:24:50.721 1883]    DBIMPORT: Importing history from database
[2019-05-24 16:24:50.721 1883]    PIDFILE: Using /var/run/pihole-FTL.pid
[2019-05-24 16:24:50.721 1883]    PORTFILE: Using /var/run/pihole-FTL.port
[2019-05-24 16:24:50.721 1883]    SOCKETFILE: Using /var/run/pihole/FTL.sock
[2019-05-24 16:24:50.721 1883]    SETUPVARSFILE: Using /etc/pihole/setupVars.conf
[2019-05-24 16:24:50.721 1883]    AUDITLISTFILE: Using /etc/pihole/auditlog.list
[2019-05-24 16:24:50.721 1883]    MACVENDORDB: Using /etc/pihole/macvendor.db
[2019-05-24 16:24:50.721 1883]    GRAVITYDB: Using /etc/pihole/gravity.db
[2019-05-24 16:24:50.721 1883]    PARSE_ARP_CACHE: Active
[2019-05-24 16:24:50.721 1883] Finished config file parsing
[2019-05-24 16:24:50.721 1883] create_shm(): shm_unlink("/FTL-lock") failed: Permission denied (13)
[2019-05-24 16:24:50.721 1883] FATAL: create_shm(): Failed to create_shm shared memory object "/FTL-lock": File exists
[2019-05-24 16:25:21.453 1910] Using log file /var/log/pihole-FTL.log
[2019-05-24 16:25:21.453 1910] ########## FTL started! ##########
[2019-05-24 16:25:21.453 1910] FTL branch: development
[2019-05-24 16:25:21.453 1910] FTL version: 
[2019-05-24 16:25:21.453 1910] FTL commit: 96320db
[2019-05-24 16:25:21.453 1910] FTL date: 2019-05-22 21:08:23 +0200
[2019-05-24 16:25:21.453 1910] FTL user: root
[2019-05-24 16:25:21.453 1910] Starting config file parsing (/etc/pihole/pihole-FTL.conf)
[2019-05-24 16:25:21.453 1910]    SOCKET_LISTENING: only local
[2019-05-24 16:25:21.453 1910]    AAAA_QUERY_ANALYSIS: Show AAAA queries
[2019-05-24 16:25:21.453 1910]    MAXDBDAYS: max age for stored queries is 365 days
[2019-05-24 16:25:21.453 1910]    RESOLVE_IPV6: Resolve IPv6 addresses
[2019-05-24 16:25:21.453 1910]    RESOLVE_IPV4: Resolve IPv4 addresses
[2019-05-24 16:25:21.453 1910]    DBINTERVAL: saving to DB file every minute
[2019-05-24 16:25:21.453 1910]    DBFILE: Using /etc/pihole/pihole-FTL.db
[2019-05-24 16:25:21.453 1910]    MAXLOGAGE: Importing up to 24.0 hours of log data
[2019-05-24 16:25:21.453 1910]    PRIVACYLEVEL: Set to 0
[2019-05-24 16:25:21.453 1910]    IGNORE_LOCALHOST: Show queries from localhost
[2019-05-24 16:25:21.454 1910]    BLOCKINGMODE: Null IPs for blocked domains
[2019-05-24 16:25:21.454 1910]    ANALYZE_ONLY_A_AND_AAAA: Disabled. Analyzing all queries
[2019-05-24 16:25:21.454 1910]    DBIMPORT: Importing history from database
[2019-05-24 16:25:21.454 1910]    PIDFILE: Using /var/run/pihole-FTL.pid
[2019-05-24 16:25:21.454 1910]    PORTFILE: Using /var/run/pihole-FTL.port
[2019-05-24 16:25:21.454 1910]    SOCKETFILE: Using /var/run/pihole/FTL.sock
[2019-05-24 16:25:21.454 1910]    SETUPVARSFILE: Using /etc/pihole/setupVars.conf
[2019-05-24 16:25:21.454 1910]    AUDITLISTFILE: Using /etc/pihole/auditlog.list
[2019-05-24 16:25:21.454 1910]    MACVENDORDB: Using /etc/pihole/macvendor.db
[2019-05-24 16:25:21.454 1910]    GRAVITYDB: Using /etc/pihole/gravity.db
[2019-05-24 16:25:21.454 1910]    PARSE_ARP_CACHE: Active
[2019-05-24 16:25:21.454 1910] Finished config file parsing
[2019-05-24 16:25:21.454 1910] WARNING: Starting pihole-FTL as user root is not recommended
[2019-05-24 16:25:21.455 1910] Database version is 3
[2019-05-24 16:25:21.455 1910] Database successfully initialized
[2019-05-24 16:25:21.455 1910] New forward server: 127.0.0.1 (0/512)
[2019-05-24 16:25:21.460 1910] Resizing "/FTL-queries" from 229376 to 458752
[2019-05-24 16:25:21.465 1910] Resizing "/FTL-queries" from 458752 to 688128
[2019-05-24 16:25:21.470 1910] Imported 12105 queries from the long-term database
[2019-05-24 16:25:21.470 1910]  -> Total DNS queries: 12105
[2019-05-24 16:25:21.470 1910]  -> Cached DNS queries: 14
[2019-05-24 16:25:21.470 1910]  -> Forwarded DNS queries: 12091
[2019-05-24 16:25:21.470 1910]  -> Exactly blocked DNS queries: 0
[2019-05-24 16:25:21.470 1910]  -> Unknown DNS queries: 0
[2019-05-24 16:25:21.470 1910]  -> Unique domains: 71
[2019-05-24 16:25:21.470 1910]  -> Unique clients: 103
[2019-05-24 16:25:21.470 1910]  -> Known forward destinations: 1
[2019-05-24 16:25:21.470 1910] Successfully accessed setupVars.conf
[2019-05-24 16:25:21.474 1912] PID of FTL process: 1912
[2019-05-24 16:25:21.474 1912] Listening on port 4711 for incoming IPv4 telnet connections
[2019-05-24 16:25:21.475 1912] Listening on port 4711 for incoming IPv6 telnet connections
[2019-05-24 16:25:21.475 1912] Listening on Unix socket
[2019-05-24 16:25:21.476 1912] Received SIGHUP, reloading cache
[2019-05-24 16:25:21.476 1912] Blocking status is enabled
[2019-05-24 16:25:21.476 1912] readGravity(): /etc/pihole/gravity.db does not exist
[2019-05-24 16:25:21.476 1912] WARN: Database query failed, assuming there are no regex entries
[2019-05-24 16:25:21.476 1912] readGravity(): /etc/pihole/gravity.db does not exist
[2019-05-24 16:25:21.476 1912] WARN: Database query failed, assuming there are no whitelist entries
[2019-05-24 16:25:21.476 1912] Compiled 0 Regex filters and 0 whitelisted domains in 0.1 msec
[2019-05-24 16:25:21.476 1912] readGravity(): /etc/pihole/gravity.db does not exist
[2019-05-24 16:25:21.476 1912] FTL_listsfile(): Error getting gravity table from database
[2019-05-24 16:25:21.476 1912] readGravity(): /etc/pihole/gravity.db does not exist
[2019-05-24 16:25:21.476 1912] FTL_listsfile(): Error getting blacklist table from database
[2019-05-24 16:25:35.932 1912] Resizing "/FTL-queries" from 688128 to 917504

[AzureMarker]

When running FTL under pihole, the FTL shared memory files in /dev/shm should be owned by pihole. Delete these file before starting FTL if this is not the case.

Generally, if FTL fails without a crash log, it means it was sent SIGKILL. It can't do any processing of this command, and is immediately stopped. Nothing should be sending this command, perhaps your operating system is doing this?

[Ge0rges]

It was owned by root. I deleted it. What is the proper way to start PiHole-FTL then?

[AzureMarker]

The proper way is to use service:

sudo service pihole-FTL restart

[Ge0rges]

When I do that I get:

[2019-05-25 23:48:44.113 5029] ########## FTL started! ##########
[2019-05-25 23:48:44.113 5029] FTL branch: master
[2019-05-25 23:48:44.113 5029] FTL version: v4.3.1
[2019-05-25 23:48:44.113 5029] FTL commit: b60d63f
[2019-05-25 23:48:44.113 5029] FTL date: 2019-05-25 21:37:26 +0200
[2019-05-25 23:48:44.113 5029] FTL user: pihole
[2019-05-25 23:48:44.113 5029] Starting config file parsing (/etc/pihole/pihole-FTL.conf)
[2019-05-25 23:48:44.113 5029]    SOCKET_LISTENING: only local
[2019-05-25 23:48:44.113 5029]    AAAA_QUERY_ANALYSIS: Show AAAA queries
[2019-05-25 23:48:44.113 5029]    MAXDBDAYS: max age for stored queries is 365 days
[2019-05-25 23:48:44.113 5029]    RESOLVE_IPV6: Resolve IPv6 addresses
[2019-05-25 23:48:44.113 5029]    RESOLVE_IPV4: Resolve IPv4 addresses
[2019-05-25 23:48:44.113 5029]    DBINTERVAL: saving to DB file every minute
[2019-05-25 23:48:44.113 5029]    DBFILE: Using /etc/pihole/pihole-FTL.db
[2019-05-25 23:48:44.113 5029]    MAXLOGAGE: Importing up to 24.0 hours of log data
[2019-05-25 23:48:44.114 5029]    PRIVACYLEVEL: Set to 0
[2019-05-25 23:48:44.114 5029]    IGNORE_LOCALHOST: Show queries from localhost
[2019-05-25 23:48:44.114 5029]    BLOCKINGMODE: Null IPs for blocked domains
[2019-05-25 23:48:44.114 5029]    ANALYZE_ONLY_A_AND_AAAA: Disabled. Analyzing all queries
[2019-05-25 23:48:44.114 5029]    DBIMPORT: Importing history from database
[2019-05-25 23:48:44.114 5029]    PIDFILE: Using /var/run/pihole-FTL.pid
[2019-05-25 23:48:44.114 5029]    PORTFILE: Using /var/run/pihole-FTL.port
[2019-05-25 23:48:44.114 5029]    SOCKETFILE: Using /var/run/pihole/FTL.sock
[2019-05-25 23:48:44.114 5029]    WHITELISTFILE: Using /etc/pihole/whitelist.txt
[2019-05-25 23:48:44.114 5029]    BLACKLISTFILE: Using /etc/pihole/black.list
[2019-05-25 23:48:44.114 5029]    GRAVITYFILE: Using /etc/pihole/gravity.list
[2019-05-25 23:48:44.114 5029]    REGEXLISTFILE: Using /etc/pihole/regex.list
[2019-05-25 23:48:44.114 5029]    SETUPVARSFILE: Using /etc/pihole/setupVars.conf
[2019-05-25 23:48:44.114 5029]    AUDITLISTFILE: Using /etc/pihole/auditlog.list
[2019-05-25 23:48:44.114 5029]    MACVENDORDB: Using /etc/pihole/macvendor.db
[2019-05-25 23:48:44.114 5029]    PARSE_ARP_CACHE: Active
[2019-05-25 23:48:44.114 5029] Finished config file parsing
[2019-05-25 23:48:44.114 5029] FATAL: create_shm(): Failed to create_shm shared memory object "/FTL-lock": Permission denied

Also I get plenty of 'May 25 23:44:51 dnsmasq[3912]: bad address at /etc/pihole/gravity.list line XXX' in /var/log/pihole.log does that affect anything?

My block list:

cat /etc/pihole/adlists.list 
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
https://mirror1.malwaredomains.com/files/justdomains
http://sysctl.org/cameleon/hosts
https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist
https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
https://hosts-file.net/ad_servers.txt
#https://raw.githubusercontent.com/imkarthikk/pihole-facebook/master/pihole-facebook.txt
https://dbl.oisd.nl

[AzureMarker]

What virtualization platform are you running on? The gravity warnings do not affect shared memory.

[Ge0rges]

DigitalOcean VPS, running Ubuntu 18.04.2 LTS.

Sent from my iPhone

On May 25, 2019, at 20:00, Mark Drobnak [e-mail] wrote:

What virtualization platform are you running on? The gravity warnings do not affect shared memory.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[DL6ER]

[2019-05-25 23:48:44.114 5029] FATAL: create_shm(): Failed to create_shm shared memory object "/FTL-lock": Permission denied

Please run the following commands:

ls -l /dev/shm
sudo rm -f /dev/shm/FTL-*
sudo service pihole-FTL restart

[Ge0rges]

Here's the output of that.

root@ge0rges:~# ls -l /dev/shm
ls: cannot access '/dev/shm': No such file or directory
root@ge0rges:~# sudo rm -f /dev/shm/FTL-*
root@ge0rges:~# sudo service pihole-FTL restart 
root@ge0rges:~# ls -l /dev/shm
ls: cannot access '/dev/shm': No such file or directory
root@ge0rges:~# sudo service pihole-FTL status
● pihole-FTL.service - LSB: pihole-FTL daemon
   Loaded: loaded (/etc/init.d/pihole-FTL; generated)
   Active: active (exited) since Sun 2019-05-26 15:31:44 UTC; 1min 2s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 10266 ExecStop=/etc/init.d/pihole-FTL stop (code=exited, status=0/SUCCESS)
  Process: 10278 ExecStart=/etc/init.d/pihole-FTL start (code=exited, status=0/SUCCESS)

May 26 15:31:44 ge0rges systemd[1]: Starting LSB: pihole-FTL daemon...
May 26 15:31:44 ge0rges pihole-FTL[10278]: Not running
May 26 15:31:44 ge0rges su[10306]: Successful su for pihole by root
May 26 15:31:44 ge0rges su[10306]: + ??? root:pihole
May 26 15:31:44 ge0rges su[10306]: pam_unix(su:session): session opened for user pihole by (uid=0)
May 26 15:31:44 ge0rges su[10306]: pam_unix(su:session): session closed for user pihole
May 26 15:31:44 ge0rges systemd[1]: Started LSB: pihole-FTL daemon.
root@ge0rges:~# pihole status
  [✗] DNS service is NOT running
root@ge0rges:~# pihole-FTL 
FTL started!
root@ge0rges:~# 

[DL6ER]

ls: cannot access '/dev/shm': No such file or directory

This is the problem here. Apparently, your Ubuntu does not use /dev/shm for shared memory. Because you ran pihole-FTL as root, you now have shared memory objects that are owned by root.

According to the Ubuntu man page, the shared memory should live in /dev/shm. Please provide the output of mount.

It may be that the files are in /run/shm (usually both places should be available). Does

ls -l /run/shm
sudo rm -f /run/shm/FTL-*
sudo service pihole-FTL restart
tail /var/log/pihole-FTL.log

work better?

[Ge0rges]

/dev/shm was previously mounted but in order to delete it I unmounted it first. I fixed my mistake. Below is the output of the requested commands, it seems that FTL now crashed again to due permission errors. Even though all files are owned by pihole.

root@ge0rges:~# mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=491664k,nr_inodes=122916,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=100916k,mode=755)
/dev/vda1 on / type ext4 (rw,relatime,data=ordered)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/unified type cgroup2 (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,rdma)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=11698)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M)
mqueue on /dev/mqueue type mqueue (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/vda15 on /boot/efi type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)
lxcfs on /var/lib/lxcfs type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=100912k,mode=700)
root@ge0rges:~# ls -l /run/shm
lrwxrwxrwx 1 root root 8 May 26 15:46 /run/shm -> /dev/shm
root@ge0rges:~# sudo rm -f /run/shm/FTL-*
root@ge0rges:~# sudo service pihole-FTL restart
root@ge0rges:~# ls -l /dev/shm
total 536
-rw------- 1 pihole pihole 331776 May 26 15:47 FTL-clients
-rw------- 1 pihole pihole    108 May 26 15:47 FTL-counters
-rw------- 1 pihole pihole  98304 May 26 15:47 FTL-domains
-rw------- 1 pihole pihole  20480 May 26 15:47 FTL-forwarded
-rw------- 1 pihole pihole     48 May 26 15:47 FTL-lock
-rw------- 1 pihole pihole  12288 May 26 15:47 FTL-overTime
-rw------- 1 pihole pihole 458752 May 26 15:47 FTL-queries
-rw------- 1 pihole pihole     12 May 26 15:47 FTL-settings
-rw------- 1 pihole pihole   8192 May 26 15:47 FTL-strings
root@ge0rges:~# sudo rm -f /dev/shm/FTL-*
root@ge0rges:~# sudo service pihole-FTL restart
root@ge0rges:~# sudo service pihole-FTL status
● pihole-FTL.service - LSB: pihole-FTL daemon
   Loaded: loaded (/etc/init.d/pihole-FTL; generated)
   Active: active (exited) since Sun 2019-05-26 15:47:54 UTC; 6s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 1826 ExecStop=/etc/init.d/pihole-FTL stop (code=exited, status=0/SUCCESS)
  Process: 1839 ExecStart=/etc/init.d/pihole-FTL start (code=exited, status=0/SUCCESS)

May 26 15:47:54 ge0rges systemd[1]: Stopped LSB: pihole-FTL daemon.
May 26 15:47:54 ge0rges systemd[1]: Starting LSB: pihole-FTL daemon...
May 26 15:47:54 ge0rges pihole-FTL[1839]: Not running
May 26 15:47:54 ge0rges su[1866]: Successful su for pihole by root
May 26 15:47:54 ge0rges su[1866]: + ??? root:pihole
May 26 15:47:54 ge0rges su[1866]: pam_unix(su:session): session opened for user pihole by (uid=0)
May 26 15:47:54 ge0rges pihole-FTL[1839]: FTL started!
May 26 15:47:54 ge0rges su[1866]: pam_unix(su:session): session closed for user pihole
May 26 15:47:54 ge0rges systemd[1]: Started LSB: pihole-FTL daemon.
root@ge0rges:~# pihole status
  [✓] DNS service is running
  [✓] Pi-hole blocking is Enabled
root@ge0rges:~# nano /var/log/pihole.log 
root@ge0rges:~# tail /var/log/pihole.log 
May 26 15:47:54 dnsmasq[1882]: compile time options: IPv6 GNU-getopt no-DBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth DNSSEC loop-detect inotify dumpfile
May 26 15:47:54 dnsmasq[1882]: LOUD WARNING: listening on 206.189.104.162 may accept requests via interfaces other than eth0
May 26 15:47:54 dnsmasq[1882]: LOUD WARNING: use --bind-dynamic rather than --bind-interfaces to avoid DNS amplification attacks via these interface(s)
May 26 15:47:54 dnsmasq[1882]: using nameserver 127.0.0.1#5353
May 26 15:47:54 dnsmasq[1882]: read /etc/hosts - 8 addresses
May 26 15:47:54 dnsmasq[1882]: read /etc/pihole/local.list - 4 addresses
May 26 15:47:54 dnsmasq[1882]: read /etc/pihole/black.list - 0 addresses
May 26 15:47:58 dnsmasq[1882]: read /etc/pihole/gravity.list - 1114034 addresses
May 26 15:47:58 dnsmasq[1882]: reducing DNS packet size for nameserver 127.0.0.1 to 1280
May 26 15:50:01 dnsmasq[1882]: reducing DNS packet size for nameserver 127.0.0.1 to 1280
root@ge0rges:~# tail /var/log/pihole-FTL.log 
[2019-05-26 15:49:23.913 1882] Resizing "/FTL-strings" from 12288 to 16384
[2019-05-26 15:49:30.938 1882] Resizing "/FTL-queries" from 917504 to 1146880
[2019-05-26 15:50:00.092 1882] SQLite3 message: statement aborts at 34: [INSERT INTO queries VALUES (NULL,?,?,?,?,?,?)] attempt to write a readonly database (8)
[2019-05-26 15:50:00.092 1882] save_to_DB() - SQL error (8): attempt to write a readonly database
[2019-05-26 15:50:00.092 1882] SQLite3 message: statement aborts at 34: [INSERT INTO queries VALUES (NULL,?,?,?,?,?,?)] attempt to write a readonly database (8)
[2019-05-26 15:50:00.092 1882] save_to_DB() - SQL error (8): attempt to write a readonly database
[2019-05-26 15:50:00.092 1882] SQLite3 message: statement aborts at 34: [INSERT INTO queries VALUES (NULL,?,?,?,?,?,?)] attempt to write a readonly database (8)
[2019-05-26 15:50:00.092 1882] save_to_DB() - SQL error (8): attempt to write a readonly database
[2019-05-26 15:50:00.092 1882] save_to_DB() - exiting due to too many errors
[2019-05-26 15:50:16.317 1882] Resizing "/FTL-queries" from 1146880 to 1376256

[DL6ER]

What you posted looks like it works. Where do you see the permission error you mentioned?

The database is probably still owned by root, try

sudo chown pihole:pihole /etc/pihole/pihole-FTL.db
sudo service pihole-FTL restart

[Ge0rges]

That fixed the [2019-05-26 15:50:00.092 1882] save_to_DB() - exiting due to too many errors error which I was mentioning thank you!

[Ge0rges]

How can I find out what's killing pihole-FTL if nothing shows up in the logs?

[DL6ER]

Unfortunately, this is not possible. There is no way pihole-FTL can delay the KILL signal. It is killed right away.

[DL6ER]

@Ge0rges See https://www.ibm.com/developerworks/community/blogs/aimsupport/entry/Finding_the_source_of_signals_on_Linux_with_strace_auditd_or_Systemtap?lang=en

[AzureMarker]

Closing this unless a real bug appears. Discussion can continue.

[gordan-bobic]

[2019-12-27 21:22:33.820 11635] FATAL: realloc_shm(): Failed to open shared memory object "/FTL-queries": Permission denied

Using latest 4.3.1 binary from the release page.
I haven't seen this on CentOS 7, but on CentOS 8 it won't stay running for more than a few minutes under minimal load

Full log segment:

[2019-12-27 21:08:28.330 11633] Using log file /var/log/pihole-FTL.log
[2019-12-27 21:08:28.331 11633] ########## FTL started! ##########
[2019-12-27 21:08:28.331 11633] FTL branch: master
[2019-12-27 21:08:28.331 11633] FTL version: v4.3.1
[2019-12-27 21:08:28.331 11633] FTL commit: b60d63f
[2019-12-27 21:08:28.331 11633] FTL date: 2019-05-25 21:37:26 +0200
[2019-12-27 21:08:28.331 11633] FTL user: root
[2019-12-27 21:08:28.331 11633] Starting config file parsing (/etc/pihole/pihole-FTL.conf)
[2019-12-27 21:08:28.331 11633]    SOCKET_LISTENING: only local
[2019-12-27 21:08:28.331 11633]    AAAA_QUERY_ANALYSIS: Show AAAA queries
[2019-12-27 21:08:28.331 11633]    MAXDBDAYS: max age for stored queries is 365 days
[2019-12-27 21:08:28.331 11633]    RESOLVE_IPV6: Resolve IPv6 addresses
[2019-12-27 21:08:28.331 11633]    RESOLVE_IPV4: Resolve IPv4 addresses
[2019-12-27 21:08:28.331 11633]    DBINTERVAL: saving to DB file every minute
[2019-12-27 21:08:28.331 11633]    DBFILE: Using /var/lib/pihole/pihole-FTL.db
[2019-12-27 21:08:28.331 11633]    MAXLOGAGE: Importing up to 24.0 hours of log data
[2019-12-27 21:08:28.331 11633]    PRIVACYLEVEL: Set to 0
[2019-12-27 21:08:28.331 11633]    IGNORE_LOCALHOST: Show queries from localhost
[2019-12-27 21:08:28.331 11633]    BLOCKINGMODE: Null IPs for blocked domains
[2019-12-27 21:08:28.331 11633]    ANALYZE_ONLY_A_AND_AAAA: Disabled. Analyzing all queries
[2019-12-27 21:08:28.331 11633]    DBIMPORT: Importing history from database
[2019-12-27 21:08:28.331 11633]    PIDFILE: Using /run/pihole/pihole-FTL.pid
[2019-12-27 21:08:28.331 11633]    PORTFILE: Using /run/pihole/pihole-FTL.port
[2019-12-27 21:08:28.331 11633]    SOCKETFILE: Using /run/pihole/FTL.sock
[2019-12-27 21:08:28.331 11633]    WHITELISTFILE: Using /etc/pihole/whitelist.txt
[2019-12-27 21:08:28.331 11633]    BLACKLISTFILE: Using /etc/pihole/black.list
[2019-12-27 21:08:28.331 11633]    GRAVITYFILE: Using /etc/pihole/gravity.list
[2019-12-27 21:08:28.331 11633]    REGEXLISTFILE: Using /etc/pihole/regex.list
[2019-12-27 21:08:28.331 11633]    SETUPVARSFILE: Using /var/lib/pihole/setupVars.conf
[2019-12-27 21:08:28.331 11633]    AUDITLISTFILE: Using /var/lib/pihole/auditlog.list
[2019-12-27 21:08:28.331 11633]    MACVENDORDB: Using /var/lib/pihole/macvendor.db
[2019-12-27 21:08:28.331 11633]    PARSE_ARP_CACHE: Active
[2019-12-27 21:08:28.331 11633] Finished config file parsing
[2019-12-27 21:08:28.332 11633] WARNING: Starting pihole-FTL as user root is not recommended
[2019-12-27 21:08:28.333 11633] Database version is 3
[2019-12-27 21:08:28.333 11633] Database successfully initialized
[2019-12-27 21:08:28.333 11633] New forward server: 127.0.0.53 (0/512)
[2019-12-27 21:08:28.336 11633] Imported 4042 queries from the long-term database
[2019-12-27 21:08:28.336 11633]  -> Total DNS queries: 4042
[2019-12-27 21:08:28.336 11633]  -> Cached DNS queries: 3819
[2019-12-27 21:08:28.336 11633]  -> Forwarded DNS queries: 223
[2019-12-27 21:08:28.336 11633]  -> Exactly blocked DNS queries: 0
[2019-12-27 21:08:28.336 11633]  -> Unknown DNS queries: 0
[2019-12-27 21:08:28.336 11633]  -> Unique domains: 93
[2019-12-27 21:08:28.336 11633]  -> Unique clients: 2
[2019-12-27 21:08:28.336 11633]  -> Known forward destinations: 1
[2019-12-27 21:08:28.337 11633] Successfully accessed setupVars.conf
[2019-12-27 21:08:28.339 11635] PID of FTL process: 11635
[2019-12-27 21:08:28.339 11635] Listening on port 4711 for incoming IPv4 telnet connections
[2019-12-27 21:08:28.340 11635] Listening on port 4711 for incoming IPv6 telnet connections
[2019-12-27 21:08:28.340 11635] Listening on Unix socket
[2019-12-27 21:08:28.341 11635] Received SIGHUP, reloading cache
[2019-12-27 21:08:28.341 11635] Blocking status is enabled
[2019-12-27 21:08:28.341 11635] INFO: No Regex file found
[2019-12-27 21:22:33.820 11635] Resizing "/FTL-queries" from 229376 to 458752
[2019-12-27 21:22:33.820 11635] FATAL: realloc_shm(): Failed to open shared memory object "/FTL-queries": Permission denied

[dschaper]

What are the permissions on that directory and that file? Do you have SELinux enabled?

[dschaper]

Oh, and why are you running the FTL daemon as root?

[gordan-bobic]

# ls -la /dev/shm/ total 264 drwxrwxrwt. 2 root root 220 Dec 27 21:08 . drwxr-xr-x. 21 root root 4160 Dec 27 20:06 .. -rw-------. 1 root root 331776 Dec 27 21:08 FTL-clients -rw-------. 1 root root 108 Dec 27 21:08 FTL-counters -rw-------. 1 root root 98304 Dec 27 21:08 FTL-domains -rw-------. 1 root root 20480 Dec 27 21:08 FTL-forwarded -rw-------. 1 root root 48 Dec 27 21:08 FTL-lock -rw-------. 1 root root 12288 Dec 27 21:08 FTL-overTime -rw-------. 1 root root 229376 Dec 27 21:08 FTL-queries -rw-------. 1 root root 12 Dec 27 21:08 FTL-settings -rw-------. 1 root root 4096 Dec 27 21:08 FTL-strings

Yes, selinux is enabled, but if it was a selinux issue I would expect it to not even start because it wouldn't be able to create these files in the first place. I just tested with setenforce 0 to disable selinux, and it crashes out almost immediately even with selinux set to permissive.

I start it as root, but have
user=pihole
group=pihole set in dnsmasq.d/

For some reason on EL8 it won't bind to a port even with the file caps set. Haven't quite got to the bottom of that part, but since it drops privileges immediately after starting up, it's safe enough for a test box.

[dschaper]

What happens when you run it with the stock configuration files and no modifications on the user or service files?

[dschaper]

@bcambl CentOS 8, not supported yet correct?

[gordan-bobic]

The only deviation from what the bash installation script does is that I have the database files paths set to /var/lib/pihole/ in the config.

I have a hunch that the problem might be with starting it as root. The /dev/shm/ files are created before it drops privileges, and then it can't change anything about them because it dropped privileges since creating them. The solution for that edge case may be in the direction of creating the files after dropping the privileges rather than after.

[DL6ER]

The solution for that edge case may be in the direction of creating the files after dropping the privileges rather than after.

I have never looked at this from view before. I agree that this might be the issue here, however, we cannot create the files after dropping privileges because we need them some time before we can bind the ports. Only thereafter, we can drop the privileges.

The solution seems to be chowning them while still being root.

[gordan-bobic]

I can confirm that this works. Annoyingly, if pihole-FTL is ExecStart-ed by systemd, ExecStartPost chown doesn't seem to be able to find /dev/shm/FTL-* files, but if I wrap pihole-FTL with a shell script that starts it and then immediately chowns it, the problem goes away:

[2019-12-27 21:54:28.590 4187] Resizing "/FTL-queries" from 229376 to 458752

That confirms the root cause and the workaround.

[DL6ER]

@gordan-bobic https://github.com/pi-hole/FTL/pull/671 addresses this.

[gordan-bobic]

:+1: