DietPi-Arr_to_RAM | Add enable/disable a systemd unit to do copy tasks automatically

[Dr0bac]

Hi @MichaIng this is not bug or feature request, just a question.

I want to make a service that will run a script that Fourdee made (this script will copy database files to ram, because this files don't let hdd to sleep)

https://github.com/MichaIng/DietPi/issues/2223#issuecomment-441082652

And I want that service to start before radarr and sonarr, what i need to change?

[MichaIng]

@Dr0bac
I will add the possibility to enable/disable a systemd service that does moving to RAM/disk automatically.

[Dr0bac]

Thank you very much!

So to clarify dietpi-arr_to_RAM 1 will backup database files to database.bak and than it will copy them to ram and make symlink back to sonarr/radarr database folder? this process must be ran before sonarr/radarr start?

dietpi-arr_to_RAM 0 will delete symlinks and copy database files back to sonarr/radarr database folders from ram on restart/shutdown process, and this must ran after sonarr/radarr close?

database.bak will be there if power outage happen, and this file i will update with sonarr/radarr post processing script after episode or movie is downloaded..

Also, can you make variable in this script where i can write/change sonarr/radarr database destination if i chose to bring these database folders back to sd card? and comment that line so i can understand where i need to put new folder path?

Again thank you very much!!!

[MichaIng]

@Dr0bac
Jep, I followed the discussion and creation of the script ;).

I will rework it slightly so that it will move the file to backup location on disk instead of copying, to reduce disk writes. It must run before Sonarr/Radarr/Lidarr start and after they stop. I will implement the server into the same boot stage as RAMdisk and RAMlog as they are bare basically doing very similar tasks.

Actually it could be made (on the long term) a very general script to move any files to RAM. We could implement a simple GUI that allows to enable/disable any pre-defined program (Sonarr/Radarr/Lidarr) or create an own slot and add related file paths. The script will then run through all enabled slots and does exactly what it does currently while looping through the enabled software file arrays:
On start:

• cp -a $DISK/file $RAM/file
• mv $DISK/file $DISK/file.bak
• ln -s $RAM/file $DISK/file

On stop:

• rm $DISK/file # symlink
• cp -a $RAM/file $DISK/file

However I will keep it simple for now.

---

For you a quick and dirty solution:

cat << _EOF_ > /etc/systemd/system/dietpi-arr_to_RAM.service
[Unit]
Description=DietPi-Arr_to_RAM
Requires=dietpi-ramdisk.service tmp.mount
After=dietpi-ramdisk.service dietpi-boot.service tmp.mount
Before=dietpi-postboot.service sonarr.service radarr.service lidarr.service

[Service]
Type=forking
RemainAfterExit=yes
ExecStart=/bin/bash -c '/DietPi/dietpi/misc/dietpi-arr_to_RAM 1 &>> /var/tmp/dietpi/logs/dietpi-arr_to_RAM.log'
ExecStop=/bin/bash -c '/DietPi/dietpi/misc/dietpi-arr_to_RAM 0 &>> /var/tmp/dietpi/logs/dietpi-arr_to_RAM.log'

[Install]
WantedBy=multi-user.target
_EOF_
systemctl daemon-reload
systemctl enable dietpi-arr_to_RAM

• It is currently not too clean since the script itself stops sonarr/radarr/lidarr when copying to RAM and starting them after finished. Both is obsolete when the service starts (as defined above) before dietpi-postboot since sonarr/radarr/lidarr are not yet running then and dietpi-postboot will start them afterwards. However it should not harm.
• Please do a backup of all databases before testing.
• You will find the result logs in /var/tmp/dietpi/logs/dietpi-arr_to_RAM.log. Check them and of course the Sonarr/Radarr/Lidarr service state before and after 2-3 reboots to verify it's working as expected.

[Dr0bac]

@MichaIng i tried few times without success..

in log file and when I look with ls -l in sonarr/radarr database folders i can see that only one sonarr file got to ram and created symlink nzbdrone.db -> /tmp/sonarr/nzbdrone.db in radarr folder nothing was changed..
Also sonarr is not loaded on restart, i can't access it..

Here is log file
dietpi-arr_to_RAM.log

[MichaIng]

@Dr0bac

/DietPi/dietpi/func/dietpi-globals: line 184: 757 Terminated systemctl start ${aFP_PROGRAM[$i]} (wd: /tmp/DietPi-Arr_to_RAM)

Hmm, not sure if the script kills the service start or the service start the script. Also currently the service is started after every single copied file. Of course it should only be started once, if at all. However as mentioned, these starts are obsolete anyway when running it from systemd service.

Also I currently can't find any Radarr files inside the script: https://github.com/MichaIng/DietPi/blob/master/dietpi/misc/dietpi-arr_to_RAM#L34-L45
Does it use the same names as Sonarr? Just running a test install, mono always taking looooong...

So finally did/does the script work as expected when running it directly from terminal?

[Dr0bac]

Yes sonarr and radarr use the same names for database files
nzbdrone.db
nzbdrone.db-shm
nzbdrone.db-wal

here is output when i run it manually

root@DietPi:~# /DietPi/dietpi/misc/dietpi-arr_to_RAM 1
[  OK  ] DietPi-Arr_to_RAM | Root access verified.
[  OK  ] DietPi-Arr_to_RAM | mkdir -p /tmp/sonarr
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/sonarr/nzbdrone.db /mnt/dietpi_userdata/sonarr/nzbdrone.db.bak
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/sonarr/nzbdrone.db /tmp/sonarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/sonarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | ln -sf /tmp/sonarr/nzbdrone.db /mnt/dietpi_userdata/sonarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm.bak
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm /tmp/sonarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | ln -sf /tmp/sonarr/nzbdrone.db-shm /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal.bak
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal /tmp/sonarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | ln -sf /tmp/sonarr/nzbdrone.db-wal /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | mkdir -p /tmp/radarr
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/radarr/nzbdrone.db /mnt/dietpi_userdata/radarr/nzbdrone.db.bak
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/radarr/nzbdrone.db /tmp/radarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/radarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | ln -sf /tmp/radarr/nzbdrone.db /mnt/dietpi_userdata/radarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/radarr/nzbdrone.db-shm /mnt/dietpi_userdata/radarr/nzbdrone.db-shm.bak
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/radarr/nzbdrone.db-shm /tmp/radarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/radarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | ln -sf /tmp/radarr/nzbdrone.db-shm /mnt/dietpi_userdata/radarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/radarr/nzbdrone.db-wal /mnt/dietpi_userdata/radarr/nzbdrone.db-wal.bak
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/radarr/nzbdrone.db-wal /tmp/radarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/radarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | ln -sf /tmp/radarr/nzbdrone.db-wal /mnt/dietpi_userdata/radarr/nzbdrone.db-wal
Failed to stop lidarr.service: Unit lidarr.service not loaded.
[  OK  ] DietPi-Arr_to_RAM | mkdir -p /tmp/lidarr
Failed to start lidarr.service: Unit lidarr.service not found.
Failed to start lidarr.service: Unit lidarr.service not found.
Failed to start lidarr.service: Unit lidarr.service not found.
Failed to start lidarr.service: Unit lidarr.service not found.
Failed to start lidarr.service: Unit lidarr.service not found.
Failed to start lidarr.service: Unit lidarr.service not found.

here is ls -l of sonarr database folder
```root@DietPi:/mnt/dietpi_userdata/sonarr# ls -l
total 3372
drwxrwxr-x 4 sonarr dietpi 4096 Jun 28 2018 Backups
-rwxrwxr-x 1 sonarr dietpi 465 Apr 7 18:47 config.xml
lrwxrwxrwx 1 sonarr dietpi 15 Feb 7 23:52 logs -> /var/log/sonarr
lrwxrwxrwx 1 sonarr dietpi 23 Feb 7 23:52 logs.db -> /var/log/sonarr/logs.db
lrwxrwxrwx 1 sonarr dietpi 27 Feb 7 23:52 logs.db-shm -> /var/log/sonarr/logs.db-shm
lrwxrwxrwx 1 sonarr dietpi 27 Feb 7 23:52 logs.db-wal -> /var/log/sonarr/logs.db-wal
drwxrwxr-x 4 sonarr dietpi 4096 Mar 29 20:51 MediaCover
lrwxrwxrwx 1 sonarr dietpi 23 Apr 7 18:47 nzbdrone.db -> /tmp/sonarr/nzbdrone.db
-rwxr-xr-x 1 root root 905216 Apr 7 18:47 nzbdrone.db.bak
lrwxrwxrwx 1 sonarr dietpi 27 Apr 7 18:47 nzbdrone.db-shm -> /tmp/sonarr/nzbdrone.db-shm
-rwxr-xr-x 1 root root 32768 Apr 7 18:47 nzbdrone.db-shm.bak
lrwxrwxrwx 1 sonarr dietpi 27 Apr 7 18:47 nzbdrone.db-wal -> /tmp/sonarr/nzbdrone.db-wal
-rwxr-xr-x 1 root root 2492632 Apr 7 18:47 nzbdrone.db-wal.bak
-rwxrwxr-x 1 sonarr dietpi 4 Apr 7 18:47 nzbdrone.pid
drwxrwxr-x 2 sonarr dietpi 4096 Nov 4 20:21 UpdateLogs

and from radarr database folder
```root@DietPi:/mnt/dietpi_userdata/radarr# ls -l
total 380
drwxrwxr-x 4 radarr dietpi   4096 Nov 20 22:54 Backups
-rwxrwxr-x 1 radarr dietpi    466 Apr  7 18:47 config.xml
lrwxrwxrwx 1 radarr dietpi     15 Feb  7 23:52 logs -> /var/log/radarr
lrwxrwxrwx 1 radarr dietpi     23 Feb  7 23:52 logs.db -> /var/log/radarr/logs.db
lrwxrwxrwx 1 radarr dietpi     27 Feb  7 23:52 logs.db-shm -> /var/log/radarr/logs.db-shm
lrwxrwxrwx 1 radarr dietpi     27 Feb  7 23:52 logs.db-wal -> /var/log/radarr/logs.db-wal
drwxrwxr-x 6 radarr dietpi   4096 Apr  2 10:31 MediaCover
lrwxrwxrwx 1 radarr dietpi     23 Apr  7 18:47 nzbdrone.db -> /tmp/radarr/nzbdrone.db
-rwxr-xr-x 1 root   root   262144 Apr  7 18:47 nzbdrone.db.bak
lrwxrwxrwx 1 radarr dietpi     27 Apr  7 18:47 nzbdrone.db-shm -> /tmp/radarr/nzbdrone.db-shm
-rwxr-xr-x 1 root   root    32768 Apr  7 18:47 nzbdrone.db-shm.bak
lrwxrwxrwx 1 radarr dietpi     27 Apr  7 18:47 nzbdrone.db-wal -> /tmp/radarr/nzbdrone.db-wal
-rwxr-xr-x 1 root   root    70072 Apr  7 18:47 nzbdrone.db-wal.bak
-rwxrwxr-x 1 radarr dietpi      4 Apr  7 18:47 nzbdrone.pid
drwxrwxr-x 2 radarr dietpi   4096 Nov 20 22:54 UpdateLogs

sonarr and radarr are working well

here is script execution on shutdown

root@DietPi:~# /DietPi/dietpi/misc/dietpi-arr_to_RAM 0
[  OK  ] DietPi-Arr_to_RAM | Root access verified.
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/sonarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | cp /tmp/sonarr/nzbdrone.db /mnt/dietpi_userdata/sonarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | rm /tmp/sonarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | cp /tmp/sonarr/nzbdrone.db-shm /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | rm /tmp/sonarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | cp /tmp/sonarr/nzbdrone.db-wal /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | rm /tmp/sonarr/nzbdrone.db-wal
[ INFO ] DietPi-Arr_to_RAM | /tmp/sonarr/lidarr.db does not exist.
[ INFO ] DietPi-Arr_to_RAM | /tmp/sonarr/lidarr.db-shm does not exist.
[ INFO ] DietPi-Arr_to_RAM | /tmp/sonarr/lidarr.db-wal does not exist.
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/radarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | cp /tmp/radarr/nzbdrone.db /mnt/dietpi_userdata/radarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | rm /tmp/radarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/radarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | cp /tmp/radarr/nzbdrone.db-shm /mnt/dietpi_userdata/radarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | rm /tmp/radarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/radarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | cp /tmp/radarr/nzbdrone.db-wal /mnt/dietpi_userdata/radarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | rm /tmp/radarr/nzbdrone.db-wal
[ INFO ] DietPi-Arr_to_RAM | /tmp/radarr/lidarr.db does not exist.
[ INFO ] DietPi-Arr_to_RAM | /tmp/radarr/lidarr.db-shm does not exist.
[ INFO ] DietPi-Arr_to_RAM | /tmp/radarr/lidarr.db-wal does not exist.
Failed to stop lidarr.service: Unit lidarr.service not loaded.
[ INFO ] DietPi-Arr_to_RAM | /tmp/lidarr/nzbdrone.db does not exist.
[ INFO ] DietPi-Arr_to_RAM | /tmp/lidarr/nzbdrone.db-shm does not exist.
[ INFO ] DietPi-Arr_to_RAM | /tmp/lidarr/nzbdrone.db-wal does not exist.
[ INFO ] DietPi-Arr_to_RAM | /tmp/lidarr/lidarr.db does not exist.
[ INFO ] DietPi-Arr_to_RAM | /tmp/lidarr/lidarr.db-shm does not exist.
[ INFO ] DietPi-Arr_to_RAM | /tmp/lidarr/lidarr.db-wal does not exist.

radarr database folder
```root@DietPi:/mnt/dietpi_userdata/radarr# ls -l
total 928
drwxrwxr-x 4 radarr dietpi 4096 Nov 20 22:54 Backups
-rwxrwxr-x 1 radarr dietpi 466 Apr 7 18:47 config.xml
lrwxrwxrwx 1 radarr dietpi 15 Feb 7 23:52 logs -> /var/log/radarr
lrwxrwxrwx 1 radarr dietpi 23 Feb 7 23:52 logs.db -> /var/log/radarr/logs.db
lrwxrwxrwx 1 radarr dietpi 27 Feb 7 23:52 logs.db-shm -> /var/log/radarr/logs.db-shm
lrwxrwxrwx 1 radarr dietpi 27 Feb 7 23:52 logs.db-wal -> /var/log/radarr/logs.db-wal
drwxrwxr-x 6 radarr dietpi 4096 Apr 2 10:31 MediaCover
-rwxr-xr-x 1 radarr dietpi 262144 Apr 7 18:51 nzbdrone.db
-rwxr-xr-x 1 root root 262144 Apr 7 18:47 nzbdrone.db.bak
-rwxr-xr-x 1 radarr dietpi 32768 Apr 7 18:51 nzbdrone.db-shm
-rwxr-xr-x 1 root root 32768 Apr 7 18:47 nzbdrone.db-shm.bak
-rwxr-xr-x 1 radarr dietpi 263712 Apr 7 18:51 nzbdrone.db-wal
-rwxr-xr-x 1 root root 70072 Apr 7 18:47 nzbdrone.db-wal.bak
-rwxrwxr-x 1 radarr dietpi 4 Apr 7 18:47 nzbdrone.pid
drwxrwxr-x 2 radarr dietpi 4096 Nov 20 22:54 UpdateLogs

and sonarr

root@DietPi:/mnt/dietpi_userdata/sonarr# ls -l
total 6872
drwxrwxr-x 4 sonarr dietpi 4096 Jun 28 2018 Backups
-rwxrwxr-x 1 sonarr dietpi 465 Apr 7 18:47 config.xml
lrwxrwxrwx 1 sonarr dietpi 15 Feb 7 23:52 logs -> /var/log/sonarr
lrwxrwxrwx 1 sonarr dietpi 23 Feb 7 23:52 logs.db -> /var/log/sonarr/logs.db
lrwxrwxrwx 1 sonarr dietpi 27 Feb 7 23:52 logs.db-shm -> /var/log/sonarr/logs.db-shm
lrwxrwxrwx 1 sonarr dietpi 27 Feb 7 23:52 logs.db-wal -> /var/log/sonarr/logs.db-wal
drwxrwxr-x 4 sonarr dietpi 4096 Mar 29 20:51 MediaCover
-rwxr--r-- 1 sonarr dietpi 811008 Apr 7 18:51 nzbdrone.db
-rwxr-xr-x 1 root root 905216 Apr 7 18:47 nzbdrone.db.bak
-rwxr-xr-x 1 sonarr dietpi 32768 Apr 7 18:51 nzbdrone.db-shm
-rwxr-xr-x 1 root root 32768 Apr 7 18:47 nzbdrone.db-shm.bak
-rwxr-xr-x 1 sonarr dietpi 2739832 Apr 7 18:51 nzbdrone.db-wal
-rwxr-xr-x 1 root root 2492632 Apr 7 18:47 nzbdrone.db-wal.bak
-rwxrwxr-x 1 sonarr dietpi 4 Apr 7 18:47 nzbdrone.pid
drwxrwxr-x 2 sonarr dietpi 4096 Nov 4 20:21 UpdateLogs
```

[MichaIng]

@Dr0bac
Okay indeed the issue with the systemd service seems to be inferences with the program service starts. I made some enhancements and prepared it to allow better execution during boot: https://github.com/MichaIng/DietPi/pull/2698
Will test proper function outside of service first and then implement service to be toggled via:
/DietPi/dietpi/misc/dietpi-arr_to-RAM enable|disable

[MichaIng]

@Dr0bac
Finished. If you want to test:

wget https://raw.githubusercontent.com/MichaIng/DietPi/arr_to_ram/dietpi/misc/dietpi-arr_to_RAM -O /DietPi/dietpi/misc/dietpi-arr_to_RAM
chmod +x /DietPi/dietpi/misc/dietpi-arr_to_RAM

Raw test:

/DietPi/dietpi/misc/dietpi-arr_to_RAM 1 # Link to RAM
/DietPi/dietpi/misc/dietpi-arr_to_RAM 0 # Restore to disk

To enable service that links to RAM on boot and restores to disk on shutdown automatically:

/DietPi/dietpi/misc/dietpi-arr_to_RAM enable

And of course to disable this:

/DietPi/dietpi/misc/dietpi-arr_to_RAM disable

If it passes testing and gone through fine tuning in case, I will add an alias as well so that it can be called via dietpi-arr_to_RAM (without full path) from terminal, when Sonarr, Radarr and/or Lidarr is found on login.

• Add info about this to online docs.

[Dr0bac]

@MichaIng after i downloaded file

wget https://raw.githubusercontent.com/MichaIng/DietPi/arr_to_ram/dietpi/misc/dietpi-arr_to_RAM -O /DietPi/dietpi/misc/dietpi-arr_to_RAM
chmod +x /DietPi/dietpi/misc/dietpi-arr_to_RAM

i enabled service /DietPi/dietpi/misc/dietpi-arr_to_RAM enable
and after one minute i got this output

root@DietPi:~# /DietPi/dietpi/misc/dietpi-arr_to_RAM enable
Job for dietpi-arr_to_RAM.service failed because a timeout was exceeded.
See "systemctl status dietpi-arr_to_RAM.service" and "journalctl -xe" for details.
2019-04-09 00:26:56 | DietPi-Arr_to_RAM: [ERROR] An issue has occurred. Please check the log for details: /var/tmp/dietpi/logs/dietpi-arr_to_RAM.log

after second try i checked systemctl status
```root@DietPi:~# systemctl status dietpi-arr_to_RAM.service
● dietpi-arr_to_RAM.service - DietPi-Arr_to_RAM
Loaded: loaded (/etc/systemd/system/dietpi-arr_to_RAM.service; enabled; vendor preset: enabled)
Active: failed (Result: timeout) since Tue 2019-04-09 00:30:53 CEST; 1min 15s ago
Process: 2379 ExecStart=/bin/dash -c /DietPi/dietpi/misc/dietpi-arr_to_RAM 1 2>&1 >> /var/tmp/dietpi/logs/dietpi-arr_to_RAM.log (code=killed, signal=TERM)

Apr 09 00:29:23 DietPi systemd[1]: Starting DietPi-Arr_to_RAM...
Apr 09 00:30:53 DietPi systemd[1]: dietpi-arr_to_RAM.service: Start operation timed out. Terminating.
Apr 09 00:30:53 DietPi systemd[1]: Failed to start DietPi-Arr_to_RAM.
Apr 09 00:30:53 DietPi systemd[1]: dietpi-arr_to_RAM.service: Unit entered failed state.
Apr 09 00:30:53 DietPi systemd[1]: dietpi-arr_to_RAM.service: Failed with result 'timeout'.

and var/log - it says already linked because i tried dietpi-arr_to_RAM enable two times

2019-04-09 00:29:23 | DietPi-Arr_to_RAM: Radarr detected
2019-04-09 00:29:23 | DietPi-Arr_to_RAM: Stopping Radarr service...
2019-04-09 00:29:23 | DietPi-Arr_to_RAM: Linking files to RAM...
2019-04-09 00:29:23 | DietPi-Arr_to_RAM: /mnt/dietpi_userdata/radarr/nzbdrone.db already linked to RAM (/tmp/radarr_db_link/nzbdrone.db). Skipping this file...
2019-04-09 00:29:23 | DietPi-Arr_to_RAM: /mnt/dietpi_userdata/radarr/nzbdrone.db-shm already linked to RAM (/tmp/radarr_db_link/nzbdrone.db-shm). Skipping this file...
2019-04-09 00:29:23 | DietPi-Arr_to_RAM: /mnt/dietpi_userdata/radarr/nzbdrone.db-wal already linked to RAM (/tmp/radarr_db_link/nzbdrone.db-wal). Skipping this file...

after that i hit `/DietPi/dietpi/misc/dietpi-arr_to_RAM 0 # Restore to disk` and restared system and everything loaded fine

root@DietPi:~# systemctl status dietpi-arr_to_RAM.service
● dietpi-arr_to_RAM.service - DietPi-Arr_to_RAM
Loaded: loaded (/etc/systemd/system/dietpi-arr_to_RAM.service; enabled; vendor preset: enabled)
Active: active (exited) since Tue 2019-04-09 00:35:25 CEST; 3min 2s ago
Process: 309 ExecStart=/bin/dash -c /DietPi/dietpi/misc/dietpi-arr_to_RAM 1 2>&1 >> /var/tmp/dietpi/logs/dietpi-arr_to_RAM.log (code=exited, status=0/SUCCESS)
CGroup: /system.slice/dietpi-arr_to_RAM.service

Apr 09 00:35:24 DietPi systemd[1]: Starting DietPi-Arr_to_RAM...
Apr 09 00:35:25 DietPi systemd[1]: Started DietPi-Arr_to_RAM.

2019-04-09 00:35:24 | DietPi-Arr_to_RAM: Radarr detected
2019-04-09 00:35:24 | DietPi-Arr_to_RAM: Linking files to RAM...
2019-04-09 00:35:24 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/radarr/nzbdrone.db to RAM (/tmp/radarr_db_link/nzbdrone.db)...
2019-04-09 00:35:24 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/radarr/nzbdrone.db-shm to RAM (/tmp/radarr_db_link/nzbdrone.db-shm)...
2019-04-09 00:35:24 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/radarr/nzbdrone.db-wal to RAM (/tmp/radarr_db_link/nzbdrone.db-wal)...
2019-04-09 00:35:24 | DietPi-Arr_to_RAM: Sonarr detected
2019-04-09 00:35:24 | DietPi-Arr_to_RAM: Linking files to RAM...
2019-04-09 00:35:24 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/sonarr/nzbdrone.db to RAM (/tmp/sonarr_db_link/nzbdrone.db)...
2019-04-09 00:35:25 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm to RAM (/tmp/sonarr_db_link/nzbdrone.db-shm)...
2019-04-09 00:35:25 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal to RAM (/tmp/sonarr_db_link/nzbdrone.db-wal)...

```

[MichaIng]

@Dr0bac
Many thanks for testing. Were files already in RAM from prior service tests two days ago?

Damn that currently stopping the service overwrites the log file, so we have no chance to check the initial error. Usually the last log is everything you need, if on last shutdown restoring to disk worked and on current boot loading into RAM as well. But while testing perhaps we should keep all logs 😉. However good that it works now. Will also run some more tests on different systems.

[Dr0bac]

@MichaIng no, files were on disk.. i brought them back with dietpi-arr_to_RAM 0 before downloading new dietpi-arr_to_RAM file.. i deleted rm /etc/systemd/system/dietpi-arr_to_RAM.service few days ago when service didn't work normaly, and i see that the new service have the same name dietpi-arr_to_RAM.service, maybe that was the problem..i restarted few times dietpi between these two actions..

I saw logs after i enabled service and before restarted dietpi, and i saw that only radarr files were moved to ram..maybe problem was with sonarr service didn't stop..

2019-04-09 00:29:23 | DietPi-Arr_to_RAM: Radarr detected
2019-04-09 00:29:23 | DietPi-Arr_to_RAM: Stopping Radarr service...
2019-04-09 00:29:23 | DietPi-Arr_to_RAM: Linking files to RAM...
2019-04-09 00:29:23 | DietPi-Arr_to_RAM: /mnt/dietpi_userdata/radarr/nzbdrone.db already linked to RAM (/tmp/radarr_db_link/nzbdrone.db). Skipping this file...
2019-04-09 00:29:23 | DietPi-Arr_to_RAM: /mnt/dietpi_userdata/radarr/nzbdrone.db-shm already linked to RAM (/tmp/radarr_db_link/nzbdrone.db-shm). Skipping this file...
2019-04-09 00:29:23 | DietPi-Arr_to_RAM: /mnt/dietpi_userdata/radarr/nzbdrone.db-wal already linked to RAM (/tmp/radarr_db_link/nzbdrone.db-wal). Skipping this file...

There is no mention of sonarr, i tried two times and saw logs, and both times logs was only displaying radarr actions.. i will try later today to disable service, and enable it again to see what will happen..

Edit: i quess it has something to do with sonarr
I disabled /DietPi/dietpi/misc/dietpi-arr_to_RAM disable everything went ok

2019-04-09 14:52:40 | DietPi-Arr_to_RAM: Radarr detected
2019-04-09 14:52:40 | DietPi-Arr_to_RAM: Stopping Radarr service...
2019-04-09 14:52:40 | DietPi-Arr_to_RAM: Restoring files from RAM (/tmp/radarr_db_link) to disk (/mnt/dietpi_userdata/radarr)...
2019-04-09 14:52:44 | DietPi-Arr_to_RAM: Sonarr detected
2019-04-09 14:52:44 | DietPi-Arr_to_RAM: Stopping Sonarr service...
2019-04-09 14:52:44 | DietPi-Arr_to_RAM: Restoring files from RAM (/tmp/sonarr_db_link) to disk (/mnt/dietpi_userdata/sonarr)...

and after restart i enabled it again, and again same error

root@DietPi:~# /DietPi/dietpi/misc/dietpi-arr_to_RAM enable
Created symlink /etc/systemd/system/multi-user.target.wants/dietpi-arr_to_RAM.service → /etc/systemd/system/dietpi-arr_to_RAM.service.
Job for dietpi-arr_to_RAM.service failed because a timeout was exceeded.
See "systemctl status dietpi-arr_to_RAM.service" and "journalctl -xe" for details.
2019-04-09 14:58:21 | DietPi-Arr_to_RAM: [ERROR] An issue has occurred. Please check the log for details: /var/tmp/dietpi/logs/dietpi-arr_to_RAM.log
root@DietPi:~# nano /var/tmp/dietpi/logs/dietpi-arr_to_RAM.log

and in log file there is nothing about sonarr

2019-04-09 14:56:51 | DietPi-Arr_to_RAM: Radarr detected
2019-04-09 14:56:51 | DietPi-Arr_to_RAM: Stopping Radarr service...
2019-04-09 14:56:51 | DietPi-Arr_to_RAM: Linking files to RAM...
2019-04-09 14:56:51 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/radarr/nzbdrone.db to RAM (/tmp/radarr_db_link/nzbdrone.db)...
2019-04-09 14:56:51 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/radarr/nzbdrone.db-shm to RAM (/tmp/radarr_db_link/nzbdrone.db-shm)...
2019-04-09 14:56:51 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/radarr/nzbdrone.db-wal to RAM (/tmp/radarr_db_link/nzbdrone.db-wal)...

and again after bringing radarr files from ram and restart, everything went ok

[MichaIng]

@Dr0bac
Hmm before Sonarr loop even really starts, "Sonarr detected" would be printed.
Possibly starting Radarr service after it's loop has finished fails. Perhaps the script runs too fast, so Radarr has not 100% finished before we attempt start.

Could you check systemctl status radarr when running into the issue with enabling the service?

In case we need to skip restarting the services or handling them at all and instead skip programs when we find their processes active, print an info instead that these need to be stopped manually first and restarted afterwards. Or simply add add some a sleep time after stopping a service? On boot this should not matter since those are not yet running, so just plays a role then arr_to_RAM is called manually.

[MichaIng]

A little fine tuning: https://github.com/MichaIng/DietPi/commit/8d29d9409d5885ccf31b8ea3ea49d036513b2756

[Dr0bac]

@MichaIng same problem..

root@DietPi:~# systemctl status dietpi-arr_to_RAM.service
● dietpi-arr_to_RAM.service - DietPi-Arr_to_RAM
   Loaded: loaded (/etc/systemd/system/dietpi-arr_to_RAM.service; enabled; vendor preset: enabled)
   Active: failed (Result: timeout) since Wed 2019-04-10 05:11:14 CEST; 1min 14s ago
  Process: 1872 ExecStart=/bin/dash -c /DietPi/dietpi/misc/dietpi-arr_to_RAM 1 2>&1 >> /var/tmp/dietpi/logs/dietpi-arr_to_RAM.log (code=killed, signal=TERM)

Apr 10 05:09:44 DietPi systemd[1]: Starting DietPi-Arr_to_RAM...
Apr 10 05:11:14 DietPi systemd[1]: dietpi-arr_to_RAM.service: Start operation timed out. Terminating.
Apr 10 05:11:14 DietPi systemd[1]: Failed to start DietPi-Arr_to_RAM.
Apr 10 05:11:14 DietPi systemd[1]: dietpi-arr_to_RAM.service: Unit entered failed state.
Apr 10 05:11:14 DietPi systemd[1]: dietpi-arr_to_RAM.service: Failed with result 'timeout'.

radarr service

root@DietPi:~# systemctl status radarr
● radarr.service - Radarr Daemon (DietPi)
   Loaded: loaded (/etc/systemd/system/radarr.service; disabled; vendor preset: enabled)
   Active: active (running) since Wed 2019-04-10 05:11:14 CEST; 2min 4s ago
 Main PID: 1994 (mono)
   CGroup: /system.slice/radarr.service
           └─1994 /usr/bin/mono -O=-aot /opt/Radarr/Radarr.exe -nobrowser -data=/mnt/dietpi_userdata/radarr

Apr 10 05:11:14 DietPi systemd[1]: Started Radarr Daemon (DietPi).
Apr 10 05:11:16 DietPi mono[1994]: [Info] Bootstrap: Starting Radarr - /opt/Radarr/Radarr.exe - Version 0.2.0.1293
Apr 10 05:11:20 DietPi mono[1994]: [Info] AppFolderInfo: Data directory is being overridden to [/mnt/dietpi_userdata/radarr]
Apr 10 05:11:21 DietPi mono[1994]: [Info] Router: Application mode: Interactive
Apr 10 05:11:22 DietPi mono[1994]: [Info] MigrationLogger: *** Migrating data source=/mnt/dietpi_userdata/radarr/nzbdrone.db;cache size=-20000;datetimekind=Utc;journal mode=Wal;pooling=True
;version=3 ***
Apr 10 05:11:23 DietPi mono[1994]: [Info] MigrationLogger: *** Migrating data source=/mnt/dietpi_userdata/radarr/logs.db;cache size=-20000;datetimekind=Utc;journal mode=Wal;pooling=True;ver
sion=3 ***
Apr 10 05:11:23 DietPi mono[1994]: [Info] OwinHostController: Listening on the following URLs:
Apr 10 05:11:23 DietPi mono[1994]: [Info] OwinHostController:   http://*:7878/
Apr 10 05:11:24 DietPi mono[1994]: [Info] NancyBootstrapper: Starting NzbDrone API
Apr 10 05:11:29 DietPi mono[1994]: [Info] CommandExecutor: Starting 2 threads for tasks.

log file

2019-04-10 05:09:44 | DietPi-Arr_to_RAM: Radarr detected
2019-04-10 05:09:44 | DietPi-Arr_to_RAM: Stopping Radarr service...
2019-04-10 05:09:44 | DietPi-Arr_to_RAM: Linking Radarr files to RAM...
2019-04-10 05:09:44 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/radarr/nzbdrone.db to RAM (/tmp/radarr_db_link/nzbdrone.db)...
2019-04-10 05:09:44 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/radarr/nzbdrone.db-shm to RAM (/tmp/radarr_db_link/nzbdrone.db-shm)...
2019-04-10 05:09:44 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/radarr/nzbdrone.db-wal to RAM (/tmp/radarr_db_link/nzbdrone.db-wal)...
2019-04-10 05:09:44 | DietPi-Arr_to_RAM: Linked Radarr files to RAM.
2019-04-10 05:09:44 | DietPi-Arr_to_RAM: Restarting Radarr service...

it's interesting that radarr service started at the same time when script failed
Apr 10 05:11:14 DietPi systemd[1]: Started Radarr Daemon (DietPi).
Apr 10 05:11:14 DietPi systemd[1]: dietpi-arr_to_RAM.service: Failed with result 'timeout'.
maybe radarr is waiting for dietpi-arr_to_RAM to finish

[MichaIng]

@Dr0bac
So it is indeed Radarr (re)start that hangs.

Could you please try the following:
sed -i '/systemctl restart/i\sleep 1' /DietPi/dietpi/misc/dietpi-arr_to_RAM

[Dr0bac]

@MichaIng I entered this code above in terminal and brought files from ram to disk and disabled service, and after restart tried to enable again
Again same output.. radar started after script service failed..

Also i noticed this when bring files back from ram

root@DietPi:~# /DietPi/dietpi/misc/dietpi-arr_to_RAM 0
2019-04-10 17:17:43 | DietPi-Arr_to_RAM: Radarr detected
2019-04-10 17:17:43 | DietPi-Arr_to_RAM: Stopping Radarr service...
2019-04-10 17:17:43 | DietPi-Arr_to_RAM: Restoring files from RAM (/tmp/radarr_db_link) to disk (/mnt/dietpi_userdata/radarr)...
/DietPi/dietpi/misc/dietpi-arr_to_RAM: line 189: Restored ${^i} files from RAM to disk.: bad substitution

and i tried second time because only radarr files were brought back

root@DietPi:~# /DietPi/dietpi/misc/dietpi-arr_to_RAM 0
2019-04-10 17:18:28 | DietPi-Arr_to_RAM: Radarr detected
2019-04-10 17:18:28 | DietPi-Arr_to_RAM: Stopping Radarr service...
2019-04-10 17:18:28 | DietPi-Arr_to_RAM: Restoring files from RAM (/tmp/radarr_db_link) to disk (/mnt/dietpi_userdata/radarr)...
2019-04-10 17:18:28 | DietPi-Arr_to_RAM: /tmp/radarr_db_link not found. Skipping this program...
2019-04-10 17:18:28 | DietPi-Arr_to_RAM: Sonarr detected
2019-04-10 17:18:28 | DietPi-Arr_to_RAM: Stopping Sonarr service...
2019-04-10 17:18:28 | DietPi-Arr_to_RAM: Restoring files from RAM (/tmp/sonarr_db_link) to disk (/mnt/dietpi_userdata/sonarr)...
/DietPi/dietpi/misc/dietpi-arr_to_RAM: line 189: Restored ${^i} files from RAM to disk.: bad substitution

[MichaIng]

@Dr0bac
Ah lol I added a syntax error accidentally. To fix:
sed -i 's/\^i/i\^/g' /DietPi/dietpi/misc/dietpi-arr_to_RAM
However it should ignore the syntax error AFAIK, does it never pass it?

EDIT Ah no actually everything went fine. The error otherwise just sais that files have been moved successfully. So your storing to disk went well. I fixed the syntax meanwhile so the desired output is shown.

Okay but the sleep for one second does not resolve the issue that Radarr is not starting up. Very strange is that it times out our service but afterwards actually did or starts up fine...

Ahh now I have an idea what might be the reason:

• Our service has the Before= setting to start before Sonarr, Radarr and Lidarr. This is just to assure that on boot this service never needs to stop and restart Sonarr/Radarr/Lidarr, in case controlled by systemd.
• But this order (Before=/After=) should only play a role, if a service is actually controlled (started) by systemd. For Sonarr, Radarr and Lidarr this is not required since otherwise DietPi boot scripts do that.
• I guess that you have enabled Radarr. So what happens is that when dietpi-arr_to_RAM.service starts, itself starts Radarr. But according to the rules, Radarr needs to wait for dietpi-arr_to_RAM.service to finish, which never happens, since dietpi-arr_to_RAM waits for Radarr again... So a snake bites it's own tail :laughing:.

You can verify by disabling Radarr service which is then started manually by DietPi-Postboot script at a later boot stage. If a service is disable, this Before/After order has no effect.

Run: systemctl disable radarr to verify this being the issue

---

But since the whole Before/After order was added as failsafe, in case user have enabled these services to make systemd starting them on boot, and exactly this is not possible, I need to adjust this somehow.

Hmm not sure how to do best:

• It is comfortable, if enabling the service as well starts it directly, so moved files to RAM, right?
• As well when disabling the service, we want files to be restored to disk in case.
• But if enabling the service moves files to RAM directly, the service needs to be started (systemctl start dietpi-arr_to_RAM), otherwise on shutdown it cannot be stopped, thus restoring to disk on first shutdown would not be done. So when enabling the service, we cannot link the files to RAM internally without starting the service via systemctl as well.
• Of course we could skip starting Sonarr/Radarr/Lidarr after files have been moved to RAM. On boot this is not required anyway since either DietPi-Postboot starts them at later stage or systemd as well afterwards due to Before/After order. But if we do not restart Radarr/Sonarr/Lidarr, user would need to do that manually, which is as well not too comforable. :thinking:.

However I will think about this after verified that it is indeed the issue.

[Dr0bac]

@MichaIng why not just put a message for user to restart dietpi, and don't touch files at this time. On reboot everything will be fine..
or maybe if user enter /DietPi/dietpi/misc/dietpi-arr_to_RAM enable if service doesn't exist to just do /DietPi/dietpi/misc/dietpi-arr_to_RAM 1 and after that is done service can be created and enabled..that way radarr will not wait for script to finish, because it will be already started..
i just tried to enter /DietPi/dietpi/misc/dietpi-arr_to_RAM 1 after i disabled service.. and it went great..

i disable radarr and sonarr systemctl disable radarr and reboot dietpi to try to enable arr_to_RAM, but after reboot radarr is again enabled.. sonarr is disabled after reboot
if i disable radarr and enable dietpi_arr_to_RAM without rebooting dietpi, same problem happen like before

[MichaIng]

@Dr0bac
Hmm strange. On newest tests I ran into the issue as well, regardless if services were enabled/disabled. I thought I tried exactly the same yesterday and didn't face it. Systemd indeed seems to respect the Before/After order as well when services are (re)started concurrently outside of boot/shutdown sequence.

• Removing Before=sonarr.service radarr.service lidarr.service resolves the issue, which verifies the reason.

Found a solution: https://manpages.debian.org/stretch/systemd/systemctl.1.en.html

    Do not synchronously wait for the requested operation to finish. If this is not specified, the job will be 
    verified, enqueued and systemctl will wait until the unit's start-up is completed. By passing this 
    argument, it is only verified and enqueued. This option may not be combined with --wait.

• With this we can queue Sonarr/Radarr/Lidarr start but DietPi-Arr_to_RAM will go on and finish, Sonarr/Radarr/Lidarr then directly afterwards from queue.
• Works well here now. Will update the code.

[Dr0bac]

@MichaIng great job! Tried few times with no errors.. Thank you for taking your time to work on this feature!

Edit: just one question.. What will happen on power outage..will script then use .bak files on next system start? So can i update these .bak files from time to time with crontab?

Also i tried to move MediaCover folder to sdcard and create symlink to radarr database..
But when i add new movies in radarr, folder with new cover images is not created..what is proper way to symlink a folder?

I tried

ln -sf /home/radarr_posters/MediaCover/ /mnt/dietpi_userdata/radarr/MediaCover

And

ln -sf /home/radarr_posters /mnt/dietpi_userdata/radarr/MediaCover

I want to move covers on sdcard because every time i open radarr or sonarr, it looks for covers on hdd and that wake it.. it should not harm to be on sdcard because it's usualy just read these covers..

[MichaIng]

@Dr0bac

Tried few times with no errors..

Perfect, many thanks for testing!

What will happen on power outage..will script then use .bak files on next system start? So can i update these .bak files from time to time with crontab?

Jep the script will automatically recover and use the backups. I had exactly the same idea with the cron job to update backups regularly. Actually most elegant would be to implement a systemd timer attached to the systemd service. So the whole job only runs if the service is active. I have to dig inside a bid, never really dealt with systemd timers.

what is proper way to symlink a folder?

I guess the issue is the permissions:

chown -R radarr:dietpi /home/radarr_posters
chmod -R 664 /home/radarr_posters

• 664 permissions only, in case some media player accesses and should be able to edit/remove them as well, or place some flags and such.

Take care that the initial dir where you want the symlink instead is removed, otherwise symlink is placed inside, even if MediaCover is already a symlink 😉.

rm /mnt/dietpi_userdata/radarr/MediaCover
ln -s /home/radarr_posters /mnt/dietpi_userdata/radarr/MediaCover

[Dr0bac]

@MichaIng you don't need to make a timer, i'm gonna update .bak files when radarr/sonarr finish movie downloads..at this time hdd is already waked..

I don't want to wake hdd if i don't need to

Only if there is easy way to update script to detect when hdd wake up to update files..
If is hard to implement, don't worry then..

[MichaIng]

@Dr0bac
I reopen the issue just to allow discussing further tests during beta phase etc.

i'm gonna update .bak files when radarr/sonarr finish movie downloads..at this time hdd is already waked..

Reasonable. How do you want to achieve that? Manually after downloading a movie or is there some kind of trigger that we can hook in? I believe there is no lightweight way to hook into disk-active-anyway status. We would need to check this regularly e.g. every 5 minutes and I am not keen to implement such a regular background job.

Perhaps for now it is easiest to add updating the backup as daily cron job. During that usually there are anyway much maintenaince tasks done that lead to external drive wakeup. But yeah, I would do that optionally as separate command: enable_cron/disable_cron e.g.

Hmm, actually an update_backup command would be great as well to update backups from RAM without the need to restore to disk completely and move to RAM again... So what the cron job does can be done manually as well.

Haha, project is growing :smile:.

[Dr0bac]

@MichaIng Radarr/sonarr have post processing events after movie/episode are downloaded and there we can attach scripts..

One more way to not wake up hdd is to place .bak files on sd card..

I read somewhere something about inotify, can this be used to trigger service to update .bak files?

Edit: inotify is using kernel or similar, it's not demanding if i remember..

Edit2: or maybe we can send signal to service when program that is using hdd are doing some tasks.. for example plex or torrent client or radarr/sonarr but don't sure how that can be achieved

Edit3: but i will be happy with that update_backup command :)

[MichaIng]

@Dr0bac
Radarr/sonarr have post processing events after movie/episode are downloaded and there we can attach scripts..

Perfect, will check on test install, then:

• Add bak update command to the script.
• Add enable/disable bak update command to script.
• Add bak update command to Radarr/Sonarr/Lidarr post-download event, when running enable bak update.
• Remove post-download job when disabling bak update.
• Disable bak update when disabling the Link to RAM boot service as well.

---

Edit: inotify is using kernel or similar, it's not demanding if i remember..

Good to have this in mind for future ideas, however post-download event as above is simplest solution then.

Edit2: or maybe we can send signal to service when program that is using hdd are doing some tasks.. for example plex or torrent client or radarr/sonarr but don't sure how that can be achieved

This would require to add a hook to every possible program that uses the HDD, much too complicated.

Edit3: but i will be happy with that update_backup command :)

Jep that as basis, will start after dinner to implement the above 😃.

[Dr0bac]

@MichaIng hey, i see that you worked on the script.. is it finished? Can i test it, or you gonna work more on it? Thanks :smile:

[MichaIng]

@Dr0bac
Jep just did some final polishing and passed my tests successfully.

I added the possibility to update the backups via:

/DietPi/dietpi/misc/dietpi-arr_to_RAM 2

Since unix permissions are required to do so, and Sonarr/Radarr/Lidarr run as their own limited user, it was required to allow this per-program. As this is generally a nice feature and to have everything more consistent, I added the possibility to do all tasks per-program optionally, e.g.:

/DietPi/dietpi/misc/dietpi-arr_to_RAM 2 radarr

will only update the Radarr databases. The same can be done to only link certain program databases to RAM or back to disk. The boot service still applies to all programs. This would have been too complicated for now an generally you either want that for all or non of the programs. Linking a single program to RAM or back most likely is only interesting for debug reasons or to fix errors, restore individual backups and such anyway.

Since systemd service start/stop requires root permissions, only the database backup updates can be done as sonarr/radarr/lidarr program user, and of course only for the single program. So what you can/need to do now is using the "Connect" > "Custom script" feature and add:
Command: /DietPi/dietpi/misc/dietpi-arr_to_RAM
Agrument: 2 radarr

Just tested as well. Hitting "Test" does not really execute the script, not sure if it just checks existence and excitability? 🤔 But when adding any test resource with enable "On grab" trigger, then it indeed updates the database backup automatically. So should work with finished downloads/renames/updates well.

Btw: I wanted to implement automated adding the custom script trigger via the script itself. But this information is within the database files and those cannot be manipulated from outside. There is an API available to perform certain tasks, but at least currently this does not include changing these settings. So this needs to be added manually. However not too hard through the GUI, I will add the info to our online docs, once released.

---

Running /DietPi/dietpi/misc/dietpi-arr_to_RAM without argument or wrong (combination) will show you nice usage info now which should explain all features as well. Hope this is understandable, otherwise I am of course opened to polish/clarify some wording, if required:

root@VM-Stretch:/mnt/dietpi_userdata/radarr# /DietPi/dietpi/misc/dietpi-arr_to_RAM 2 radarrwhoopsie
2019-04-13 19:16:28 | DietPi-Arr_to_RAM: [ERROR] Invalid input program (radarrwhoopsie). Aborting...

Usage: /DietPi/dietpi/misc/dietpi-arr_to_RAM <command> [<program>]
Available commands:
  1 [<program>]         Link (program) database(s) to RAM
  2 [<program>]         Update (program) database backup(s)
  0 [<program>]         Store (program) database(s) back to disk
  enable                Enable Link to RAM on boot
  disable               Disable Link to RAM on boot
Supported programs:
  <empty>               Apply to all supported and installed programs
  sonarr                Apply to Sonarr database only
  radarr                Apply to Radarr database only
  lidarr                Apply to Lidarr database only

[Dr0bac]

@MichaIng yes i quess test button only confirm that required input fields are checked..
Thank you, great work!!

about your desire to implement automated backup files.. radarr/sonarr releases variables on events, so this can be helpfull?
https://github.com/Radarr/Radarr/wiki/Custom-Post-Processing-Scripts
but with this current way it's already perfect :smile:

Edit: just tasted.. Well done, beautiful script :smile:

[MichaIng]

@Dr0bac
Sadly this does not include adding the script itself. Still:

Scripts are added to Radarr via the Connect Settings page.

The environment variables are passed with the command, so e.g. our script can read those to derive what exactly has been grabbed/updated/downloaded etc. The example scripts are just what can be added to make 3rd party software (media players) recognize and react to the change.

What we would require is an API (via web page port or some binary or whatever) to actually add the script with configured triggers and arguments. But still couldn't find such.

but with this current way it's already perfect 😄
Edit: just tasted.. Well done, beautiful script 😄

Many thanks! Jep I think for now this works well. Future ideas like a simple GUI and/or allow to add custom programs+files to be linked to RAM is something for future releases. Current stage (+bug fixing/wording/fine tuning, if required) is what will be shipped with v6.23 😃.

[MichaIng]

Merged it finally into dev branch. Added info to Sonarr/Radarr/Lidarr users on update and detailed info to our online docs/forum: https://github.com/MichaIng/DietPi/pull/2698

• https://dietpi.com/phpbb/viewtopic.php?f=8&t=5828 (This is linked from Sonarr/Radarr/Lidarr docs as well)

[Fourdee]

Notes:

• Sonarr enabled. After reboot, does not auto mount to RAM:
• Does not automatically start once enabled.

root@Fuzon:~# cat /var/tmp/dietpi/logs/dietpi-arr_to_RAM.log 
2019-04-21 14:47:30 | DietPi-Arr_to_RAM: Sonarr detected
2019-04-21 14:47:30 | DietPi-Arr_to_RAM: Storing database from RAM (/tmp/sonarr_db_link) back to disk (/mnt/dietpi_userdata/sonarr)...
2019-04-21 14:47:30 | DietPi-Arr_to_RAM: Stored Sonarr database from RAM back to disk.
root@Fuzon:~# ls -lha /mnt/dietpi_userdata/sonarr/
total 17M
drwxr-xr-x  4 sonarr root   4.0K Apr 21 14:48 .
drwxrwxr-x 10 dietpi dietpi 4.0K Apr 20 18:34 ..
drwxrwxr-x  5 sonarr dietpi 4.0K Apr 20 18:35 Backups
-rwxrwxr-x  1 sonarr dietpi  464 Apr 21 14:48 config.xml
lrwxrwxrwx  1 sonarr dietpi   15 Apr 20 18:34 logs -> /var/log/sonarr
lrwxrwxrwx  1 sonarr dietpi   23 Apr 20 18:34 logs.db -> /var/log/sonarr/logs.db
lrwxrwxrwx  1 sonarr dietpi   27 Apr 20 18:34 logs.db-shm -> /var/log/sonarr/logs.db-shm
lrwxrwxrwx  1 sonarr dietpi   27 Apr 20 18:34 logs.db-wal -> /var/log/sonarr/logs.db-wal
drwxrwxr-x 46 sonarr dietpi 4.0K Apr 21 14:47 MediaCover
-rwxrwxr-x  1 sonarr dietpi 6.2M Apr 21 14:48 nzbdrone.db
-rwxrwxr-x  1 sonarr dietpi 6.1M Apr 21 13:33 nzbdrone.db.bak
-rwxrwxr-x  1 sonarr dietpi  32K Apr 21 14:44 nzbdrone.db-shm.bak
-rwxrwxr-x  1 sonarr dietpi 4.0M Apr 21 14:44 nzbdrone.db-wal.bak
-rwxrwxr-x  1 sonarr dietpi    4 Apr 21 14:47 nzbdrone.pid

[MichaIng]

@Fourdee
How did you enable? dietpi-arr_to_RAM enable
Hmm as the log shows data stored to disk on shutdown, the service was active at least. No output about any attempt on boot.

systemctl status dietpi-arr_to_RAM would be interesting then to verify it's enabled and check whether some dependency failed, boot order or something wrong.

[MichaIng]

I reverted the changes due to comments: https://github.com/MichaIng/DietPi/commit/54faaaac606dbc6d0c25af2e962cf3f5d93676d6#diff-0545e5f960140b2e9723fda692ce3e84

• Will retest with "enabled" (systemd controlled) Sonarr/Radarr/Lidarr

• https://github.com/MichaIng/DietPi/blob/dev/dietpi/misc/dietpi-arr_to_RAM

[MichaIng]

Okay sonarr behaves very strange:

root@VM-Jessie:/mnt/dietpi_userdata/sonarr# dietpi-arr_to_RAM 1 && l
2019-04-27 19:39:13 | DietPi-Arr_to_RAM: Sonarr detected
2019-04-27 19:39:13 | DietPi-Arr_to_RAM: Stopping Sonarr service...
2019-04-27 19:39:13 | DietPi-Arr_to_RAM: Linking Sonarr database to RAM...
2019-04-27 19:39:13 | DietPi-Arr_to_RAM: /mnt/dietpi_userdata/sonarr/nzbdrone.db already linked to RAM (/tmp/sonarr_db_link/nzbdrone.db). Skipping this file...
2019-04-27 19:39:13 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm to RAM (/tmp/sonarr_db_link/nzbdrone.db-shm)...
2019-04-27 19:39:13 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal to RAM (/tmp/sonarr_db_link/nzbdrone.db-wal)...
2019-04-27 19:39:13 | DietPi-Arr_to_RAM: Linked Sonarr database to RAM.
2019-04-27 19:39:13 | DietPi-Arr_to_RAM: Enqueuing sonarr service start(s)...
total 564
-rw-r--r-- 1 sonarr dietpi    306 Apr 27 19:37 config.xml
lrwxrwxrwx 1 sonarr dietpi     15 Apr 27 17:31 logs -> /var/log/sonarr
lrwxrwxrwx 1 sonarr dietpi     23 Apr 27 17:31 logs.db -> /var/log/sonarr/logs.db
lrwxrwxrwx 1 sonarr dietpi     27 Apr 27 17:31 logs.db-shm -> /var/log/sonarr/logs.db-shm
-rw-r--r-- 1 sonarr dietpi 318624 Apr 27 19:37 logs.db-wal
lrwxrwxrwx 1 sonarr root       31 Apr 27 19:07 nzbdrone.db -> /tmp/sonarr_db_link/nzbdrone.db
-rw-r--r-- 1 sonarr dietpi 189440 Apr 27 19:04 nzbdrone.db.bak
lrwxrwxrwx 1 sonarr root       35 Apr 27 19:39 nzbdrone.db-shm -> /tmp/sonarr_db_link/nzbdrone.db-shm
-rw-r--r-- 1 sonarr dietpi  32768 Apr 27 19:38 nzbdrone.db-shm.bak
lrwxrwxrwx 1 sonarr root       35 Apr 27 19:39 nzbdrone.db-wal -> /tmp/sonarr_db_link/nzbdrone.db-wal
-rw-r--r-- 1 sonarr dietpi  22040 Apr 27 19:39 nzbdrone.db-wal.bak
-rw-r--r-- 1 sonarr dietpi      4 Apr 27 19:37 nzbdrone.pid
root@VM-Jessie:/mnt/dietpi_userdata/sonarr# l
total 564
-rw-r--r-- 1 sonarr dietpi    306 Apr 27 19:37 config.xml
lrwxrwxrwx 1 sonarr dietpi     15 Apr 27 17:31 logs -> /var/log/sonarr
lrwxrwxrwx 1 sonarr dietpi     23 Apr 27 17:31 logs.db -> /var/log/sonarr/logs.db
lrwxrwxrwx 1 sonarr dietpi     27 Apr 27 17:31 logs.db-shm -> /var/log/sonarr/logs.db-shm
-rw-r--r-- 1 sonarr dietpi 318624 Apr 27 19:37 logs.db-wal
lrwxrwxrwx 1 sonarr root       31 Apr 27 19:07 nzbdrone.db -> /tmp/sonarr_db_link/nzbdrone.db
-rw-r--r-- 1 sonarr dietpi 189440 Apr 27 19:04 nzbdrone.db.bak
lrwxrwxrwx 1 sonarr root       35 Apr 27 19:39 nzbdrone.db-shm -> /tmp/sonarr_db_link/nzbdrone.db-shm
-rw-r--r-- 1 sonarr dietpi  32768 Apr 27 19:38 nzbdrone.db-shm.bak
lrwxrwxrwx 1 sonarr root       35 Apr 27 19:39 nzbdrone.db-wal -> /tmp/sonarr_db_link/nzbdrone.db-wal
-rw-r--r-- 1 sonarr dietpi  22040 Apr 27 19:39 nzbdrone.db-wal.bak
-rw-r--r-- 1 sonarr dietpi      4 Apr 27 19:39 nzbdrone.pid
root@VM-Jessie:/mnt/dietpi_userdata/sonarr# l
total 624
-rw-r--r-- 1 sonarr dietpi    306 Apr 27 19:37 config.xml
lrwxrwxrwx 1 sonarr dietpi     15 Apr 27 17:31 logs -> /var/log/sonarr
lrwxrwxrwx 1 sonarr dietpi     23 Apr 27 17:31 logs.db -> /var/log/sonarr/logs.db
lrwxrwxrwx 1 sonarr dietpi     27 Apr 27 17:31 logs.db-shm -> /var/log/sonarr/logs.db-shm
-rw-r--r-- 1 sonarr dietpi 318624 Apr 27 19:37 logs.db-wal
lrwxrwxrwx 1 sonarr root       31 Apr 27 19:07 nzbdrone.db -> /tmp/sonarr_db_link/nzbdrone.db
-rw-r--r-- 1 sonarr dietpi 189440 Apr 27 19:04 nzbdrone.db.bak
-rw-r--r-- 1 sonarr dietpi  32768 Apr 27 19:39 nzbdrone.db-shm
-rw-r--r-- 1 sonarr dietpi  32768 Apr 27 19:38 nzbdrone.db-shm.bak
-rw-r--r-- 1 sonarr dietpi  25184 Apr 27 19:39 nzbdrone.db-wal
-rw-r--r-- 1 sonarr dietpi  22040 Apr 27 19:39 nzbdrone.db-wal.bak
-rw-r--r-- 1 sonarr dietpi      4 Apr 27 19:39 nzbdrone.pid

• Arr_to_RAM succeeds, all files linked to RAM, but shm and wal files are removed an re-created.
• I guess this is part of the startup/data migration:

Apr 27 19:39:13 VM-Jessie systemd[1]: Starting Sonarr (NzbDrone) Daemon (DietPi)...
Apr 27 19:39:13 VM-Jessie systemd[1]: Started Sonarr (NzbDrone) Daemon (DietPi).
Apr 27 19:39:14 VM-Jessie mono[1465]: [Info] Bootstrap: Starting Sonarr - /opt/NzbDrone/NzbDrone.exe - Version 2.0.0.5321
Apr 27 19:39:16 VM-Jessie mono[1465]: [Info] AppFolderInfo: Data directory is being overridden to [/mnt/dietpi_userdata/sonarr]
Apr 27 19:39:17 VM-Jessie mono[1465]: [Info] Router: Application mode: Interactive
Apr 27 19:39:17 VM-Jessie mono[1465]: [Info] MigrationLogger: *** Migrating data source=/mnt/dietpi_userdata/sonarr/nzbdrone.db;cache size=-10000;datetimekind=Utc;journal mode=Wal;pooling=True;version=3 ***
Apr 27 19:39:18 VM-Jessie mono[1465]: [Info] MigrationLogger: *** Migrating data source=/mnt/dietpi_userdata/sonarr/logs.db;cache size=-10000;datetimekind=Utc;journal mode=Wal;pooling=True;version=3 ***
Apr 27 19:39:18 VM-Jessie mono[1465]: [Info] OwinHostController: Listening on the following URLs:
Apr 27 19:39:18 VM-Jessie mono[1465]: [Info] OwinHostController:   http://*:8989/
Apr 27 19:39:18 VM-Jessie mono[1465]: [Info] NancyBootstrapper: Starting Web Server
Apr 27 19:39:21 VM-Jessie mono[1465]: [Warn] MonoDebugCheck: Mono is not running with --debug switch

However aside from that, the Before=sonarr.service works well and database (itself) linked to RAM as expected, with and without sonarr.service being enabled:

Apr 27 19:07:02 VM-Jessie systemd[1]: Started DietPi-RAMdisk.
Apr 27 19:07:02 VM-Jessie systemd[1]: Starting DietPi-Arr_to_RAM...
Apr 27 19:07:02 VM-Jessie systemd[1]: Started DietPi-Arr_to_RAM.
Apr 27 19:07:02 VM-Jessie systemd[1]: Starting Sonarr (NzbDrone) Daemon (DietPi)...
Apr 27 19:07:02 VM-Jessie systemd[1]: Started Sonarr (NzbDrone) Daemon (DietPi).
Apr 27 19:07:02 VM-Jessie systemd[1]: Starting DietPi-PreBoot...
Apr 27 19:07:03 VM-Jessie systemd[1]: Started DietPi-PreBoot.

[MichaIng]

Sonarr-only issue: EDIT: Ah nope, waiting some seconds more and Lidarr/Radarr behave the same...

root@VM-Jessie:/mnt/dietpi_userdata# dietpi-arr_to_RAM 1
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Lidarr detected
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Stopping Lidarr service...
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Linking Lidarr database to RAM...
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/lidarr/lidarr.db to RAM (/tmp/lidarr_db_link/lidarr.db)...
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/lidarr/lidarr.db-shm to RAM (/tmp/lidarr_db_link/lidarr.db-shm)...
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/lidarr/lidarr.db-wal to RAM (/tmp/lidarr_db_link/lidarr.db-wal)...
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Linked Lidarr database to RAM.
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Radarr detected
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Stopping Radarr service...
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Linking Radarr database to RAM...
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/radarr/nzbdrone.db to RAM (/tmp/radarr_db_link/nzbdrone.db)...
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/radarr/nzbdrone.db-shm to RAM (/tmp/radarr_db_link/nzbdrone.db-shm)...
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/radarr/nzbdrone.db-wal to RAM (/tmp/radarr_db_link/nzbdrone.db-wal)...
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Linked Radarr database to RAM.
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Sonarr detected
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Stopping Sonarr service...
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Linking Sonarr database to RAM...
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: /mnt/dietpi_userdata/sonarr/nzbdrone.db already linked to RAM (/tmp/sonarr_db_link/nzbdrone.db). Skipping this file...
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm not found. Recovering from backup first (/mnt/dietpi_userdata/sonarr/nzbdrone.db-shm.bak)...
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm to RAM (/tmp/sonarr_db_link/nzbdrone.db-shm)...
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal not found. Recovering from backup first (/mnt/dietpi_userdata/sonarr/nzbdrone.db-wal.bak)...
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Linking /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal to RAM (/tmp/sonarr_db_link/nzbdrone.db-wal)...
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Linked Sonarr database to RAM.
2019-04-27 19:47:46 | DietPi-Arr_to_RAM: Enqueuing lidarr radarr sonarr service start(s)...
root@VM-Jessie:/mnt/dietpi_userdata# l radarr/
total 688
-rw-r--r-- 1 radarr dietpi     46 Apr 27 19:47 config.xml
lrwxrwxrwx 1 radarr dietpi     15 Apr 27 19:47 logs -> /var/log/radarr
lrwxrwxrwx 1 radarr dietpi     23 Apr 27 19:47 logs.db -> /var/log/radarr/logs.db
lrwxrwxrwx 1 radarr dietpi     27 Apr 27 19:47 logs.db-shm -> /var/log/radarr/logs.db-shm
-rw-r--r-- 1 radarr dietpi 102736 Apr 27 19:47 logs.db-wal
lrwxrwxrwx 1 radarr root       31 Apr 27 19:47 nzbdrone.db -> /tmp/radarr_db_link/nzbdrone.db
-rw-r--r-- 1 radarr dietpi   1024 Apr 27 19:47 nzbdrone.db.bak
lrwxrwxrwx 1 radarr root       35 Apr 27 19:47 nzbdrone.db-shm -> /tmp/radarr_db_link/nzbdrone.db-shm
-rw-r--r-- 1 radarr dietpi  32768 Apr 27 19:47 nzbdrone.db-shm.bak
lrwxrwxrwx 1 radarr root       35 Apr 27 19:47 nzbdrone.db-wal -> /tmp/radarr_db_link/nzbdrone.db-wal
-rw-r--r-- 1 radarr dietpi 549184 Apr 27 19:47 nzbdrone.db-wal.bak
-rw-r--r-- 1 radarr dietpi      4 Apr 27 19:47 nzbdrone.pid
root@VM-Jessie:/mnt/dietpi_userdata# l lidarr/
total 124
-rw-r--r-- 1 lidarr dietpi   280 Apr 27 19:47 config.xml
lrwxrwxrwx 1 lidarr root      29 Apr 27 19:47 lidarr.db -> /tmp/lidarr_db_link/lidarr.db
-rw-r--r-- 1 lidarr dietpi  1024 Apr 27 19:47 lidarr.db.bak
lrwxrwxrwx 1 lidarr root      33 Apr 27 19:47 lidarr.db-shm -> /tmp/lidarr_db_link/lidarr.db-shm
-rw-r--r-- 1 lidarr dietpi 32768 Apr 27 19:47 lidarr.db-shm.bak
lrwxrwxrwx 1 lidarr root      33 Apr 27 19:47 lidarr.db-wal -> /tmp/lidarr_db_link/lidarr.db-wal
-rw-r--r-- 1 lidarr dietpi  5272 Apr 27 19:47 lidarr.db-wal.bak
-rw-r--r-- 1 lidarr dietpi     4 Apr 27 19:47 lidarr.pid
lrwxrwxrwx 1 lidarr dietpi    15 Apr 27 19:47 logs -> /var/log/lidarr
lrwxrwxrwx 1 lidarr dietpi    23 Apr 27 19:47 logs.db -> /var/log/lidarr/logs.db
lrwxrwxrwx 1 lidarr dietpi    27 Apr 27 19:47 logs.db-shm -> /var/log/lidarr/logs.db-shm
-rw-r--r-- 1 lidarr dietpi 73392 Apr 27 19:47 logs.db-wal
root@VM-Jessie:/mnt/dietpi_userdata# l sonarr/
total 632
-rwxrwxr-x 1 sonarr dietpi    306 Apr 27 19:47 config.xml
lrwxrwxrwx 1 sonarr dietpi     15 Apr 27 17:31 logs -> /var/log/sonarr
lrwxrwxrwx 1 sonarr dietpi     23 Apr 27 17:31 logs.db -> /var/log/sonarr/logs.db
lrwxrwxrwx 1 sonarr dietpi     27 Apr 27 17:31 logs.db-shm -> /var/log/sonarr/logs.db-shm
-rwxrwxr-x 1 sonarr dietpi 324912 Apr 27 19:47 logs.db-wal
lrwxrwxrwx 1 sonarr dietpi     31 Apr 27 19:07 nzbdrone.db -> /tmp/sonarr_db_link/nzbdrone.db
-rwxrwxr-x 1 sonarr dietpi 189440 Apr 27 19:04 nzbdrone.db.bak
-rw-r--r-- 1 sonarr dietpi  32768 Apr 27 19:47 nzbdrone.db-shm
-rwxrwxr-x 1 sonarr dietpi  32768 Apr 27 19:38 nzbdrone.db-shm.bak
-rw-r--r-- 1 sonarr dietpi  27280 Apr 27 19:47 nzbdrone.db-wal
-rwxrwxr-x 1 sonarr dietpi  22040 Apr 27 19:39 nzbdrone.db-wal.bak
-rwxrwxr-x 1 sonarr dietpi      4 Apr 27 19:47 nzbdrone.pid

...

root@VM-Jessie:/mnt/dietpi_userdata# l lidarr/
total 136
-rw-r--r-- 1 lidarr dietpi   307 Apr 27 19:47 config.xml
lrwxrwxrwx 1 lidarr root      29 Apr 27 19:47 lidarr.db -> /tmp/lidarr_db_link/lidarr.db
-rw-r--r-- 1 lidarr dietpi  1024 Apr 27 19:47 lidarr.db.bak
-rw-r--r-- 1 lidarr dietpi 32768 Apr 27 19:47 lidarr.db-shm.bak
-rw-r--r-- 1 lidarr dietpi  5272 Apr 27 19:47 lidarr.db-wal.bak
-rw-r--r-- 1 lidarr dietpi     4 Apr 27 19:47 lidarr.pid
lrwxrwxrwx 1 lidarr dietpi    15 Apr 27 19:47 logs -> /var/log/lidarr
lrwxrwxrwx 1 lidarr dietpi    23 Apr 27 19:47 logs.db -> /var/log/lidarr/logs.db
lrwxrwxrwx 1 lidarr dietpi    27 Apr 27 19:47 logs.db-shm -> /var/log/lidarr/logs.db-shm
-rw-r--r-- 1 lidarr dietpi 85968 Apr 27 19:48 logs.db-wal
root@VM-Jessie:/mnt/dietpi_userdata# l radarr/
total 964
-rw-r--r-- 1 radarr dietpi    307 Apr 27 19:47 config.xml
lrwxrwxrwx 1 radarr dietpi     15 Apr 27 19:47 logs -> /var/log/radarr
lrwxrwxrwx 1 radarr dietpi     23 Apr 27 19:47 logs.db -> /var/log/radarr/logs.db
lrwxrwxrwx 1 radarr dietpi     27 Apr 27 19:47 logs.db-shm -> /var/log/radarr/logs.db-shm
-rw-r--r-- 1 radarr dietpi 347968 Apr 27 19:47 logs.db-wal
lrwxrwxrwx 1 radarr root       31 Apr 27 19:47 nzbdrone.db -> /tmp/radarr_db_link/nzbdrone.db
-rw-r--r-- 1 radarr dietpi   1024 Apr 27 19:47 nzbdrone.db.bak
-rw-r--r-- 1 radarr dietpi  32768 Apr 27 19:49 nzbdrone.db-shm
-rw-r--r-- 1 radarr dietpi  32768 Apr 27 19:47 nzbdrone.db-shm.bak
-rw-r--r-- 1 radarr dietpi   5272 Apr 27 19:48 nzbdrone.db-wal
-rw-r--r-- 1 radarr dietpi 549184 Apr 27 19:47 nzbdrone.db-wal.bak
-rw-r--r-- 1 radarr dietpi      4 Apr 27 19:47 nzbdrone.pid

[MichaIng]

The same happens with the logs shm/wal files. Hum that is bad, all services completely remove and recreate those files. I currently see no chance to prevent this. Hardlinks are only possible within the same file system. Mounts for each file individually is an overkill IMO?

That is very bad since this breaks the whole idea of the script. The db file is not what is regularly accessed but indeed those shm/wal files...

I manually re-created the symlink while Radarr running and see if they get overwritten again by times or only on service start.

---

Ah, the logs shm/wal file just got overwritten very quickly, lets see what happens to the database.

Jep, there is even a background tasks that recreates these files:

Component | Message | Time
-- | -- | --
PreDBService | Starting PreDB Sync | 8:05pm
Database | Main database compressed | 8:05pm
Database | Vacuuming Main database | 8:05pm
Database | Log database compressed | 8:05pm
Database | Vacuuming Log database | 8:05pm

Leads to:

root@VM-Jessie:/mnt/dietpi_userdata/radarr# l
total 212
-rw-r--r-- 1 radarr dietpi   420 Apr 27 20:05 config.xml
lrwxrwxrwx 1 radarr dietpi    15 Apr 27 19:47 logs -> /var/log/radarr
lrwxrwxrwx 1 radarr dietpi    23 Apr 27 19:47 logs.db -> /var/log/radarr/logs.db
-rw-r--r-- 1 radarr dietpi 32768 Apr 27 20:06 logs.db-shm
-rw-r--r-- 1 radarr dietpi     0 Apr 27 20:06 logs.db-wal
lrwxrwxrwx 1 radarr root      31 Apr 27 19:51 nzbdrone.db -> /tmp/radarr_db_link/nzbdrone.db
-rw-r--r-- 1 radarr dietpi 98304 Apr 27 19:48 nzbdrone.db.bak
-rw-r--r-- 1 radarr dietpi 32768 Apr 27 20:06 nzbdrone.db-shm
-rw-r--r-- 1 radarr dietpi 32768 Apr 27 19:50 nzbdrone.db-shm.bak
-rw-r--r-- 1 radarr dietpi     0 Apr 27 20:06 nzbdrone.db-wal
-rw-r--r-- 1 radarr dietpi 10512 Apr 27 19:50 nzbdrone.db-wal.bak
-rw-r--r-- 1 radarr dietpi     3 Apr 27 19:51 nzbdrone.pid

[MichaIng]

@Dr0bac @Fourdee
I wonder if this is new, perhaps some change in mono, since I never recognized it before. Can you verify this is the case for your installs as well?

[Dr0bac]

My files are on ram, but i didnt update anything lately? What you want for me to do? To do apt-get update / upgrade and check again?

[MichaIng]

@Dr0bac
I actually tested on Jessie. Will test on Stretch and Buster now as well. If it behaves the same there, yeah G_AGUP && G_AGUG and retest would be nice, to have mono on current version. And otherwise in case Sonarr/Radarr/Lidarr update, if available via web UI.

[MichaIng]

Okay indeed it's a Jessie-only issue. Tested on Stretch and Buster and database + log shm/wal links survive time and maintenance task. @Dr0bac so no need for you to run tests, but thanks for offering!

Jessie ships mono v5.18.1.0 while Stretch + Buster ship mono v5.20.1.19. Since all three software titles are affected and their versions are also the same across distro versions, it must be related to either mono or some underlying libraries/functions.

Hmm, no sure now how to handle. As said without these files staying in RAM the scripts intention is broken. Currently I see no solution but only supporting it for ... besides ... there is a preview/beta branch that ships v5.20.1 as well. Will try it.

---

EDIT

Okay not related to mono version either, so must be some underlying Jessie library/function thing. Very strange, however I am lucky since there are nearly no users affected then. So we will skip loading and block this script on Jessie.

[Dr0bac]

@MichaIng thanks for testing this out! So if users are on latest dietpi update they will be on strech? So no woring, because script is preloaded only on latest dietpi update?
Also you changed something few days back in script because of issues? Are functionality remain the same?

[MichaIng]

@Dr0bac
Ah, nope it's not about DietPi version but about the major version of the underlying Debian/Raspbian OS. Jessie is the old version, Stretch the current stable release and Buster is in the last steps of release, expected early summer. Due to this upcoming release the support for the old Jessie system is fading, causing problems on several edges.

This OS version is something we cannot influence sind upgrading a full production systems nearly always causes problems, APT errors, breaks existing software and setups/settings and leaves the system in a not ideal state with some outdated/deprecated settings/files/services and such. In short. If you are not experienced and in mood to tinker with the OS, you should never do a distro upgrade, so DietPi does never as well 😉.

Aside from that (on Stretch and Buster) the script runs fine according to my tests.

Also you changed something few days back in script because of issues? Are functionality remain the same?

Everything the same, no important changes were done.

[Dr0bac]

@MichaIng i noticed that sometimes my hdd wake up when i'm not using raspberry pi..
i searched programs that can tell me what files are accessed or what processes are doing this, and i found fatrace - https://piware.de/2012/02/fatrace-report-system-wide-file-access-events/
but it doesn't work, i get error every time Cannot initialize fanotify: Invalid argument
when i paste error in google search i get
https://bugs.launchpad.net/fatrace/+bug/1755787
so i guess this program don't work with raspberry pi..

What solution you recommend to see what file are accessed on hdd and which program is doing this?

[MichaIng]

@Dr0bac
Quick very simple idea is iotop, similar to top but disk I/O instead of CPU usage. But no real monitoring AFAIK.

G_AGI iotop
iotop -Pa

- a shows the accumulated usage instead of current, so comes close to a monitoring.
-P to show processes instead of single threads.

---

~fatrace and fanotify seem to be veeeery old unmaintained, so most likely they are not compatible with modern kernel versions and possibly systemd-logind vs rsyslog (fatrace), but not sure.~ Ah nope maintained in Debian repo: https://packages.debian.org/buster/fatrace

[Dr0bac]

Tried htop, it can't check custom folder, or custom hdd, only system wide read/wrtite. Tried inotifywait, can check custom folder acess but don't display which process use it..
Tried fatrace newer package from buster, same error, sent email to programer of fatrace, he said

Can you please check if your kernel has fanotify enabled? On most systems you
can check like this:

grep CONFIG_FANOTIFY /boot/config-`uname -r`

It should be "Y" or at least "M".

I tried but is says no such file or directory?
So i figure it out that he means about kernel file.. in config.txt in boot directory there are only specific dietpi options..

[MichaIng]

@Dr0bac
Yeah indeed iotop just tracks global I/O. A very simple tool with not much features.

/boot/config-$(uname -r) exists on x86 systems but not on RPi.

Here is the RPi kernel source config: https://github.com/raspberrypi/linux/blob/rpi-4.14.y/arch/arm/configs/bcm2709_defconfig
It contains CONFIG_FANOTIFY=y

Ah are you actually on RPi or which SBC/device? 😄

[Dr0bac]

@MichaIng yes i'm on Raspberry pi 3 model B