FTL crash (findClientID+0x28)

[SirRufo]

Log file output [if available]

• pihole.FTL.log

[2020-05-15 10:40:01.957 7241] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-05-15 10:40:01.958 7241] ---------------------------->  FTL crashed!  <----------------------------
[2020-05-15 10:40:01.958 7241] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-05-15 10:40:01.958 7241] Please report a bug at https://github.com/pi-hole/FTL/issues
[2020-05-15 10:40:01.958 7241] and include in your report already the following details:
[2020-05-15 10:40:01.958 7241] FTL has been running for 5164 seconds
[2020-05-15 10:40:01.958 7241] FTL branch: master
[2020-05-15 10:40:01.959 7241] FTL version: v5.0
[2020-05-15 10:40:01.959 7241] FTL commit: 3d7c095
[2020-05-15 10:40:01.959 7241] FTL date: 2020-05-10 18:58:38 +0100
[2020-05-15 10:40:01.959 7241] FTL user: started as pihole, ended as pihole
[2020-05-15 10:40:01.960 7241] Compiled for arm (compiled on CI) using arm-linux-gnueabihf-gcc (crosstool-NG crosstool-ng-1.22.0-88-g8460611) 4.9.3
[2020-05-15 10:40:01.960 7241] Received signal: Segmentation fault
[2020-05-15 10:40:01.960 7241]      at address: 0xb6f81018
[2020-05-15 10:40:01.960 7241]      with code: SEGV_MAPERR (Address not mapped to object)
[2020-05-15 10:40:01.960 7241] Backtrace:
[2020-05-15 10:40:01.962 7241] B[0000]: 0x479704, /usr/bin/pihole-FTL(+0x2c704) [0x479704]
[2020-05-15 10:40:01.962 7241] B[0001]: 0xb6db3130, /lib/arm-linux-gnueabihf/libc.so.6(__default_rt_sa_restorer+0) [0xb6db3130]
[2020-05-15 10:40:01.962 7241] B[0002]: 0x478e28, /usr/bin/pihole-FTL(findClientID+0x28) [0x478e28]
[2020-05-15 10:40:01.962 7241] B[0003]: 0x46efe4, /usr/bin/pihole-FTL(parse_neighbor_cache+0x2e0) [0x46efe4]
[2020-05-15 10:40:01.962 7241] B[0004]: 0x4721bc, /usr/bin/pihole-FTL(DB_thread+0x158) [0x4721bc]
[2020-05-15 10:40:01.963 7241] ------ Listing content of directory /dev/shm ------
[2020-05-15 10:40:01.963 7241] File Mode User:Group  Filesize Filename
[2020-05-15 10:40:01.963 7241] rwxrwxrwx root:root 160 .
[2020-05-15 10:40:01.964 7241] rwxr-xr-x root:root 4K ..
[2020-05-15 10:40:01.964 7241] rw------- pihole:pihole 4K FTL-per-client-regex
[2020-05-15 10:40:02.051 7241] rw------- pihole:pihole 33K FTL-dns-cache
[2020-05-15 10:40:02.051 7241] rw------- pihole:pihole 29K FTL-overTime
[2020-05-15 10:40:02.051 7241] FTL-queries failed with No such file or directory (2)
[2020-05-15 10:40:02.052 7241] rw------- pihole:pihole 20K FTL-upstreams
[2020-05-15 10:40:02.053 7241] rw------- pihole:pihole 12 FTL-settings
[2020-05-15 10:40:02.053 7241] ---------------------------------------------------
[2020-05-15 10:40:02.053 7241] Thank you for helping us to improve our FTL engine!
[2020-05-15 10:40:02.053 7241] FTL terminated!

Device specifics

Hardware Type: rPi Zero W
OS: Raspbian GNU/Linux 10 (buster) 4.19.97+

It occurs randomly in the past and at the moment I restart the rPi every day and restart DNS every 2 hours

[yubiuser]

Same as here?
https://github.com/pi-hole/FTL/issues/757

What is the output of cat /var/log/pihole.log

[SirRufo]

I am not quite sure if it is the same issue. #757 is not arm version as this one

Nothing special in the

• pihole.log

May 15 09:14:02 dnsmasq[7241]: started, version pi-hole-2.81 cachesize 10000
May 15 09:14:02 dnsmasq[7241]: compile time options: IPv6 GNU-getopt no-DBus no-UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth DNSSEC l$
May 15 09:14:02 dnsmasq[7241]: using only locally-known addresses for domain use-application-dns.net
May 15 09:14:02 dnsmasq[7241]: using nameserver ****
May 15 09:14:02 dnsmasq[7241]: using nameserver ****
May 15 09:14:02 dnsmasq[7241]: read /etc/hosts - 5 addresses
May 15 09:14:02 dnsmasq[7241]: failed to load names from /etc/pihole/custom.list: No such file or directory
May 15 09:14:02 dnsmasq[7241]: read /etc/pihole/local.list - 4 addresses
May 15 10:33:08 dnsmasq[7241]: reducing DNS packet size for nameserver 192.168.246.1 to 1280
May 15 10:40:01 dnsmasq[7241]: exiting on receipt of SIGTERM
May 15 10:40:03 dnsmasq[13142]: started, version pi-hole-2.81 cachesize 10000
May 15 10:40:03 dnsmasq[13142]: compile time options: IPv6 GNU-getopt no-DBus no-UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth DNSSEC $
May 15 10:40:03 dnsmasq[13142]: using only locally-known addresses for domain use-application-dns.net
May 15 10:40:03 dnsmasq[13142]: using nameserver ****
May 15 10:40:03 dnsmasq[13142]: using nameserver ****
May 15 10:40:03 dnsmasq[13142]: read /etc/hosts - 5 addresses
May 15 10:40:03 dnsmasq[13142]: failed to load names from /etc/pihole/custom.list: No such file or directory
May 15 10:40:03 dnsmasq[13142]: read /etc/pihole/local.list - 4 addresses

[DL6ER]

I think this is another issue, maybe shared memory related.

Please share some more lines of the log above

[2020-05-15 10:40:01.957 7241] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-05-15 10:40:01.958 7241] ---------------------------->  FTL crashed!  <----------------------------
[2020-05-15 10:40:01.958 7241] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

[SirRufo]

Here is the lates crash with the whole log from starting

[2020-05-15 14:14:21.446 605] ########## FTL started! ##########
[2020-05-15 14:14:21.446 605] FTL branch: master
[2020-05-15 14:14:21.446 605] FTL version: v5.0
[2020-05-15 14:14:21.446 605] FTL commit: 3d7c095
[2020-05-15 14:14:21.447 605] FTL date: 2020-05-10 18:58:38 +0100
[2020-05-15 14:14:21.447 605] FTL user: pihole
[2020-05-15 14:14:21.447 605] Compiled for arm (compiled on CI) using arm-linux-gnueabihf-gcc (crosstool-NG crosstool-ng-1.22.0-88-g8460611) 4.9.3
[2020-05-15 14:14:21.447 605] Starting config file parsing (/etc/pihole/pihole-FTL.conf)
[2020-05-15 14:14:21.447 605]    SOCKET_LISTENING: only local
[2020-05-15 14:14:21.448 605]    AAAA_QUERY_ANALYSIS: Show AAAA queries
[2020-05-15 14:14:21.448 605]    MAXDBDAYS: max age for stored queries is 365 days
[2020-05-15 14:14:21.448 605]    RESOLVE_IPV6: Resolve IPv6 addresses
[2020-05-15 14:14:21.448 605]    RESOLVE_IPV4: Resolve IPv4 addresses
[2020-05-15 14:14:21.449 605]    DBINTERVAL: saving to DB file every minute
[2020-05-15 14:14:21.449 605]    DBFILE: Using /etc/pihole/pihole-FTL.db
[2020-05-15 14:14:21.449 605]    MAXLOGAGE: Importing up to 24.0 hours of log data
[2020-05-15 14:14:21.449 605] Notice: Increasing privacy level from 0 to 1
[2020-05-15 14:14:21.450 605]    PRIVACYLEVEL: Set to 1
[2020-05-15 14:14:21.450 605]    IGNORE_LOCALHOST: Show queries from localhost
[2020-05-15 14:14:21.450 605]    BLOCKINGMODE: Null IPs for blocked domains
[2020-05-15 14:14:21.450 605]    ANALYZE_ONLY_A_AND_AAAA: Disabled. Analyzing all queries
[2020-05-15 14:14:21.450 605]    DBIMPORT: Importing history from database
[2020-05-15 14:14:21.451 605]    PIDFILE: Using /var/run/pihole-FTL.pid
[2020-05-15 14:14:21.450 605]    DBIMPORT: Importing history from database
[2020-05-15 14:14:21.451 605]    PIDFILE: Using /var/run/pihole-FTL.pid
[2020-05-15 14:14:21.451 605]    PORTFILE: Using /var/run/pihole-FTL.port
[2020-05-15 14:14:21.451 605]    SOCKETFILE: Using /var/run/pihole/FTL.sock
[2020-05-15 14:14:21.451 605]    SETUPVARSFILE: Using /etc/pihole/setupVars.conf
[2020-05-15 14:14:21.455 605]    MACVENDORDB: Using /etc/pihole/macvendor.db
[2020-05-15 14:14:21.455 605]    GRAVITYDB: Using /etc/pihole/gravity.db
[2020-05-15 14:14:21.455 605]    PARSE_ARP_CACHE: Active
[2020-05-15 14:14:21.456 605]    CNAME_DEEP_INSPECT: Active
[2020-05-15 14:14:21.456 605]    DELAY_STARTUP: No delay requested.
[2020-05-15 14:14:21.456 605]    BLOCK_ESNI: Enabled, blocking _esni.{blocked domain}
[2020-05-15 14:14:21.457 605] Finished config file parsing
[2020-05-15 14:14:21.637 605] Database version is 5
[2020-05-15 14:14:21.638 605] Database successfully initialized
[2020-05-15 14:14:21.707 605] New upstream server: **** (0/1024)
[2020-05-15 14:14:21.712 605] New upstream server: **** (1/1024)
[2020-05-15 14:14:22.096 605] Resizing "/FTL-queries" from 229376 to 458752
[2020-05-15 14:14:22.371 605] Resizing "/FTL-queries" from 458752 to 688128
[2020-05-15 14:14:22.615 605] Resizing "/FTL-queries" from 688128 to 917504
[2020-05-15 14:14:22.788 605] Resizing "/FTL-queries" from 917504 to 1146880
[2020-05-15 14:14:23.005 605] Resizing "/FTL-queries" from 1146880 to 1376256
[2020-05-15 14:14:23.295 605] Resizing "/FTL-queries" from 1376256 to 1605632
[2020-05-15 14:14:23.573 605] Resizing "/FTL-queries" from 1605632 to 1835008
[2020-05-15 14:14:23.846 605] Resizing "/FTL-queries" from 1835008 to 2064384
[2020-05-15 14:14:24.183 605] Resizing "/FTL-queries" from 2064384 to 2293760
[2020-05-15 14:14:24.471 605] Resizing "/FTL-queries" from 2293760 to 2523136
[2020-05-15 14:14:24.736 605] Resizing "/FTL-queries" from 2523136 to 2752512
[2020-05-15 14:14:24.746 605] Imported 45116 queries from the long-term database
[2020-05-15 14:14:24.752 605]  -> Total DNS queries: 45116
[2020-05-15 14:14:24.752 605]  -> Cached DNS queries: 33952
[2020-05-15 14:14:24.752 605]  -> Forwarded DNS queries: 8532
[2020-05-15 14:14:24.752 605]  -> Blocked DNS queries: 2626
[2020-05-15 14:14:24.752 605]  -> Unknown DNS queries: 6
[2020-05-15 14:14:24.753 605]  -> Unique domains: 1
[2020-05-15 14:14:24.753 605]  -> Unique clients: 28
[2020-05-15 14:14:24.753 605]  -> Known forward destinations: 2
[2020-05-15 14:14:24.753 605] Successfully accessed setupVars.conf
[2020-05-15 14:14:24.808 607] PID of FTL process: 607
[2020-05-15 14:14:24.809 607] Listening on port 4711 for incoming IPv4 telnet connections
[2020-05-15 14:14:24.810 607] Listening on port 4711 for incoming IPv6 telnet connections
[2020-05-15 14:14:24.810 607] Listening on Unix socket
[2020-05-15 14:14:24.836 607] Reloading DNS cache
[2020-05-15 14:14:24.810 607] Listening on Unix socket
[2020-05-15 14:14:24.836 607] Reloading DNS cache
[2020-05-15 14:14:24.837 607] Blocking status is enabled
[2020-05-15 14:14:24.887 607] INFO: No regex whitelist entries found
[2020-05-15 14:14:25.167 607] Compiled 0 whitelist and 1 blacklist regex filters in 290.7 msec
[2020-05-15 14:16:01.198 607] Resizing "/FTL-strings" from 4096 to 8192
[2020-05-15 14:23:15.349 607] Resizing "/FTL-dns-cache" from 4096 to 8192
[2020-05-15 14:24:38.159 1506] Note: FTL forked to handle TCP requests
[2020-05-15 14:24:46.180 607] Resizing "/FTL-strings" from 8192 to 12288
[2020-05-15 14:32:24.866 607] Resizing "/FTL-queries" from 2752512 to 2981888
[2020-05-15 14:34:40.136 607] Resizing "/FTL-strings" from 12288 to 16384
[2020-05-15 14:34:40.142 607] Resizing "/FTL-dns-cache" from 8192 to 12288
[2020-05-15 14:40:47.476 607] Resizing "/FTL-strings" from 16384 to 20480
[2020-05-15 14:44:35.258 607] Resizing "/FTL-dns-cache" from 12288 to 16384
[2020-05-15 14:50:58.697 607] Resizing "/FTL-queries" from 2981888 to 3211264
[2020-05-15 15:03:58.527 607] Resizing "/FTL-strings" from 20480 to 24576
[2020-05-15 15:09:23.806 607] Resizing "/FTL-queries" from 3211264 to 3440640
[2020-05-15 15:12:27.649 607] Resizing "/FTL-dns-cache" from 16384 to 20480
[2020-05-15 15:14:38.368 5461] Note: FTL forked to handle TCP requests
[2020-05-15 15:20:00.655 607] Resizing "/FTL-dns-cache" from 20480 to 24576
[2020-05-15 15:20:16.028 607] Resizing "/FTL-strings" from 24576 to 28672
[2020-05-15 15:24:38.185 5947] Note: FTL forked to handle TCP requests
[2020-05-15 15:27:26.732 607] Resizing "/FTL-queries" from 3440640 to 3670016
[2020-05-15 15:47:46.888 607] Resizing "/FTL-queries" from 3670016 to 3899392
[2020-05-15 15:49:10.119 607] Resizing "/FTL-dns-cache" from 24576 to 28672
[2020-05-15 16:09:02.365 607] Resizing "/FTL-queries" from 3899392 to 4128768
[2020-05-15 16:25:58.232 8195] Note: FTL forked to handle TCP requests
[2020-05-15 16:30:14.327 607] Resizing "/FTL-queries" from 4128768 to 4358144
[2020-05-15 16:45:52.451 607] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-05-15 16:45:52.451 607] ---------------------------->  FTL crashed!  <----------------------------
[2020-05-15 16:45:52.451 607] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-05-15 16:45:52.451 607] Please report a bug at https://github.com/pi-hole/FTL/issues
[2020-05-15 16:45:52.452 607] and include in your report already the following details:
[2020-05-15 16:45:52.452 607] FTL has been running for 9091 seconds
[2020-05-15 16:45:52.452 607] FTL branch: master
[2020-05-15 16:45:52.452 607] FTL version: v5.0
[2020-05-15 16:45:52.452 607] FTL commit: 3d7c095
[2020-05-15 16:45:52.453 607] FTL date: 2020-05-10 18:58:38 +0100
[2020-05-15 16:45:52.453 607] FTL user: started as pihole, ended as pihole
[2020-05-15 16:45:52.453 607] Compiled for arm (compiled on CI) using arm-linux-gnueabihf-gcc (crosstool-NG crosstool-ng-1.22.0-88-g8460611) 4.9.3
[2020-05-15 16:45:52.453 607] Received signal: Segmentation fault
[2020-05-15 16:45:52.454 607]      at address: 0x7a66652d
[2020-05-15 16:45:52.453 607] Received signal: Segmentation fault
[2020-05-15 16:45:52.454 607]      at address: 0x7a66652d
[2020-05-15 16:45:52.454 607]      with code: SEGV_MAPERR (Address not mapped to object)
[2020-05-15 16:45:52.454 607] Backtrace:
[2020-05-15 16:45:52.455 607] B[0000]: 0x480704, /usr/bin/pihole-FTL(+0x2c704) [0x480704]
[2020-05-15 16:45:52.456 607] B[0001]: 0xb6dff130, /lib/arm-linux-gnueabihf/libc.so.6(__default_rt_sa_restorer+0) [0xb6dff130]
[2020-05-15 16:45:52.456 607] B[0002]: 0x4b6b2c, /usr/bin/pihole-FTL(+0x62b2c) [0x4b6b2c]
[2020-05-15 16:45:52.456 607] B[0003]: 0x4b8cfc, /usr/bin/pihole-FTL(main_dnsmasq+0x1238) [0x4b8cfc]
[2020-05-15 16:45:52.456 607] B[0004]: 0x472d04, /usr/bin/pihole-FTL(main+0xfc) [0x472d04]
[2020-05-15 16:45:52.456 607] B[0005]: 0xb6de9718, /lib/arm-linux-gnueabihf/libc.so.6(__libc_start_main+0x10c) [0xb6de9718]
[2020-05-15 16:45:52.457 607] ------ Listing content of directory /dev/shm ------
[2020-05-15 16:45:52.457 607] File Mode User:Group  Filesize Filename
[2020-05-15 16:45:52.457 607] rwxrwxrwx root:root 260 .
[2020-05-15 16:45:52.458 607] rwxr-xr-x root:root 4K ..
[2020-05-15 16:45:52.458 607] rw------- pihole:pihole 4K FTL-per-client-regex
[2020-05-15 16:45:52.459 607] rw------- pihole:pihole 29K FTL-dns-cache
[2020-05-15 16:45:52.460 607] rw------- pihole:pihole 29K FTL-overTime
[2020-05-15 16:45:52.460 607] rw------- pihole:pihole 4M FTL-queries
[2020-05-15 16:45:52.461 607] rw------- pihole:pihole 20K FTL-upstreams
[2020-05-15 16:45:52.461 607] rw------- pihole:pihole 643K FTL-clients
[2020-05-15 16:45:52.462 607] rw------- pihole:pihole 66K FTL-domains
[2020-05-15 16:45:52.463 607] rw------- pihole:pihole 29K FTL-strings
[2020-05-15 16:45:52.464 607] rw------- pihole:pihole 12 FTL-settings
[2020-05-15 16:45:52.465 607] rw------- pihole:pihole 124 FTL-counters
[2020-05-15 16:45:52.465 607] rw------- pihole:pihole 28 FTL-lock
[2020-05-15 16:45:52.465 607] ---------------------------------------------------
[2020-05-15 16:45:52.466 607] Thank you for helping us to improve our FTL engine!
[2020-05-15 16:45:52.466 607] FTL terminated!

[DL6ER]

Okay, so this time the crash happened in another place. Please follow the instructions at https://docs.pi-hole.net/ftldns/debugging/ so we can get a full backtrace next time the crash happens.

Sorry for the inconvenience caused by this!

[DL6ER]

For those who cannot attach the debugger (which would still give more information about the crash source): Please run

pihole checkout ftl new/addr2line_master

This should (hopefully) generate logs with extended information for us to check.

[SirRufo]

I am now on 5.0 with FTL development and for two days without any error

[SirRufo]

Ok, today I had another crash with the development branch:

[2020-06-17 03:17:53.009 1651] Resizing "/FTL-strings" from 126976 to 131072
[2020-06-17 03:37:53.219 22887] Note: FTL forked to handle TCP requests
[2020-06-17 04:07:53.642 1651] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-17 04:07:53.642 1651] ---------------------------->  FTL crashed!  <----------------------------
[2020-06-17 04:07:53.642 1651] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-17 04:07:53.643 1651] Please report a bug at https://github.com/pi-hole/FTL/issues
[2020-06-17 04:07:53.643 1651] and include in your report already the following details:
[2020-06-17 04:07:53.643 1651] FTL has been running for 234875 seconds
[2020-06-17 04:07:53.643 1651] FTL branch: development
[2020-06-17 04:07:53.643 1651] FTL version: vDev-6c32fb8
[2020-06-17 04:07:53.643 1651] FTL commit: 6c32fb8
[2020-06-17 04:07:53.644 1651] FTL date: 2020-06-04 15:39:51 +0200
[2020-06-17 04:07:53.644 1651] FTL user: started as pihole, ended as pihole
[2020-06-17 04:07:53.644 1651] Compiled for arm (compiled on CI) using arm-linux-gnueabihf-gcc (crosstool-NG crosstool-ng-1.22.0-8$
[2020-06-17 04:07:53.644 1651] Received signal: Segmentation fault
[2020-06-17 04:07:53.645 1651]      at address: 0x10876
[2020-06-17 04:07:53.645 1651]      with code: SEGV_MAPERR (Address not mapped to object)
[2020-06-17 04:07:53.645 1651] Backtrace:
[2020-06-17 04:07:53.647 1651] B[0000]: 0x451030, /usr/bin/pihole-FTL(+0x30030) [0x451030]
[2020-06-17 04:07:53.913 1651] L[0000]: /root/project/src/signals.c:94
[2020-06-17 04:07:53.922 1651] B[0001]: 0xb6d5d130, /lib/arm-linux-gnueabihf/libc.so.6(__default_rt_sa_restorer+0) [0xb6d5d130]
[2020-06-17 04:07:53.922 1651] B[0002]: 0x49d260, /usr/bin/pihole-FTL(answer_request+0xf4) [0x49d260]
[2020-06-17 04:07:54.000 1651] L[0002]: /root/project/src/dnsmasq/rfc1035.c:1373 (discriminator 3)
[2020-06-17 04:07:54.009 1651] B[0003]: 0x47bdb8, /usr/bin/pihole-FTL(receive_query+0x880) [0x47bdb8]
[2020-06-17 04:07:54.087 1651] L[0003]: /root/project/src/dnsmasq/forward.c:1635
[2020-06-17 04:07:54.094 1651] B[0004]: 0x46fad4, /usr/bin/pihole-FTL(+0x4ead4) [0x46fad4]
[2020-06-17 04:07:54.165 1651] L[0004]: /root/project/src/dnsmasq/dnsmasq.c:1786
[2020-06-17 04:07:54.172 1651] B[0005]: 0x471a68, /usr/bin/pihole-FTL(main_dnsmasq+0x1238) [0x471a68]
[2020-06-17 04:07:54.241 1651] L[0005]: /root/project/src/dnsmasq/dnsmasq.c:1206
[2020-06-17 04:07:54.248 1651] B[0006]: 0x441e3c, /usr/bin/pihole-FTL(main+0xfc) [0x441e3c]
[2020-06-17 04:07:54.304 1651] L[0006]: /root/project/src/main.c:95
[2020-06-17 04:07:54.310 1651] B[0007]: 0xb6d47718, /lib/arm-linux-gnueabihf/libc.so.6(__libc_start_main+0x10c) [0xb6d47718]
[2020-06-17 04:07:54.310 1651] ------ Listing content of directory /dev/shm ------
[2020-06-17 04:07:54.310 1651] File Mode User:Group  Filesize Filename
[2020-06-17 04:07:54.311 1651] rwxrwxrwx root:root 260 .
[2020-06-17 04:07:54.311 1651] rwxr-xr-x root:root 3K ..
[2020-06-17 04:07:54.312 1651] rw------- pihole:pihole 4K FTL-per-client-regex
[2020-06-17 04:07:54.312 1651] rw------- pihole:pihole 139K FTL-dns-cache
[2020-06-17 04:07:54.313 1651] rw------- pihole:pihole 29K FTL-overTime
[2020-06-17 04:07:54.314 1651] rw------- pihole:pihole 19M FTL-queries
[2020-06-17 04:07:54.314 1651] rw------- pihole:pihole 20K FTL-upstreams
[2020-06-17 04:07:54.315 1651] rw------- pihole:pihole 324K FTL-clients
[2020-06-17 04:07:54.316 1651] rw------- pihole:pihole 131K FTL-domains
[2020-06-17 04:07:54.316 1651] rw------- pihole:pihole 131K FTL-strings
[2020-06-17 04:07:54.317 1651] rw------- pihole:pihole 12 FTL-settings
[2020-06-17 04:07:54.317 1651] rw------- pihole:pihole 124 FTL-counters
[2020-06-17 04:07:54.318 1651] rw------- pihole:pihole 28 FTL-lock
[2020-06-17 04:07:54.318 1651] ---------------------------------------------------
[2020-06-17 04:07:54.318 1651] Thank you for helping us to improve our FTL engine!
[2020-06-17 04:07:54.319 1651] FTL terminated!

[SirRufo]

Ok, next crash after 11 hours running (after scheduled reboot)

The same as before

[2020-06-18 14:56:06.980 601] Resizing "/FTL-queries" from 18808832 to 19038208
[2020-06-18 14:56:39.516 601] Resizing "/FTL-dns-cache" from 45056 to 49152
[2020-06-18 15:17:53.389 601] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-18 15:17:53.389 601] ---------------------------->  FTL crashed!  <----------------------------
[2020-06-18 15:17:53.389 601] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2020-06-18 15:17:53.390 601] Please report a bug at https://github.com/pi-hole/FTL/issues
[2020-06-18 15:17:53.390 601] and include in your report already the following details:
[2020-06-18 15:17:53.390 601] FTL has been running for 40324 seconds
[2020-06-18 15:17:53.390 601] FTL branch: development
[2020-06-18 15:17:53.390 601] FTL version: vDev-6c32fb8
[2020-06-18 15:17:53.390 601] FTL commit: 6c32fb8
[2020-06-18 15:17:53.391 601] FTL date: 2020-06-04 15:39:51 +0200
[2020-06-18 15:17:53.391 601] FTL user: started as pihole, ended as pihole
[2020-06-18 15:17:53.391 601] Compiled for arm (compiled on CI) using arm-linux-gnueabihf-gcc (crosstool-NG crosstool-ng-1.22.0-88-g846$
[2020-06-18 15:17:53.391 601] Received signal: Segmentation fault
[2020-06-18 15:17:53.392 601]      at address: 0x7a66652d
[2020-06-18 15:17:53.392 601]      with code: SEGV_MAPERR (Address not mapped to object)
[2020-06-18 15:17:53.392 601] Backtrace:
[2020-06-18 15:17:53.394 601] B[0000]: 0x493030, /usr/bin/pihole-FTL(+0x30030) [0x493030]
[2020-06-18 15:17:53.659 601] L[0000]: /root/project/src/signals.c:94
[2020-06-18 15:17:53.668 601] B[0001]: 0xb6d67130, /lib/arm-linux-gnueabihf/libc.so.6(__default_rt_sa_restorer+0) [0xb6d67130]
[2020-06-18 15:17:53.668 601] B[0002]: 0x4b1898, /usr/bin/pihole-FTL(+0x4e898) [0x4b1898]
[2020-06-18 15:17:53.740 601] L[0002]: /root/project/src/dnsmasq/dnsmasq.c:1785
[2020-06-18 15:17:53.748 601] B[0003]: 0x4b3a68, /usr/bin/pihole-FTL(main_dnsmasq+0x1238) [0x4b3a68]
[2020-06-18 15:17:53.817 601] L[0003]: /root/project/src/dnsmasq/dnsmasq.c:1206
[2020-06-18 15:17:53.824 601] B[0004]: 0x483e3c, /usr/bin/pihole-FTL(main+0xfc) [0x483e3c]
[2020-06-18 15:17:53.880 601] L[0004]: /root/project/src/main.c:95
[2020-06-18 15:17:53.885 601] B[0005]: 0xb6d51718, /lib/arm-linux-gnueabihf/libc.so.6(__libc_start_main+0x10c) [0xb6d51718]
[2020-06-18 15:17:53.885 601] ------ Listing content of directory /dev/shm ------
[2020-06-18 15:17:53.885 601] File Mode User:Group  Filesize Filename
[2020-06-18 15:17:53.886 601] rwxrwxrwx root:root 260 .
[2020-06-18 15:17:53.886 601] rwxr-xr-x root:root 3K ..
[2020-06-18 15:17:53.887 601] rw------- pihole:pihole 4K FTL-per-client-regex
[2020-06-18 15:17:53.888 601] rw------- pihole:pihole 49K FTL-dns-cache
[2020-06-18 15:17:53.888 601] rw------- pihole:pihole 29K FTL-overTime
[2020-06-18 15:17:53.889 601] rw------- pihole:pihole 19M FTL-queries
[2020-06-18 15:17:53.889 601] rw------- pihole:pihole 20K FTL-upstreams
[2020-06-18 15:17:53.890 601] rw------- pihole:pihole 324K FTL-clients
[2020-06-18 15:17:53.891 601] rw------- pihole:pihole 66K FTL-domains
[2020-06-18 15:17:53.891 601] rw------- pihole:pihole 86K FTL-strings
[2020-06-18 15:17:53.892 601] rw------- pihole:pihole 12 FTL-settings
[2020-06-18 15:17:53.892 601] rw------- pihole:pihole 124 FTL-counters
[2020-06-18 15:17:53.893 601] rw------- pihole:pihole 28 FTL-lock
[2020-06-18 15:17:53.893 601] ---------------------------------------------------
[2020-06-18 15:17:53.893 601] Thank you for helping us to improve our FTL engine!
[2020-06-18 15:17:53.893 601] FTL terminated!

[DL6ER]

@SirRufo We have a potential fix for a rather severe dnsmasq-internal bug discovered by @fhriley

Please run

pihole checkout ftl fix/fhriley-fix_buf_overflow

and see if the crashes are resolved. As always, I very much appreciate testing as it is the only way to be sure we really got it fixed!

[SirRufo]

3 days running on fix/fhriley-fix_buf_overflow without any error ... look good so far

[DL6ER]

Pi-hole v5.1 has been released. Please get back on track with pihole checkout master and update.

If the issue persists with v5.1 (which we hope is not the case), please consider opening a new ticket for the sake of clarity. It will help us to help you best.