DietPi-Software | /var/log sonarr and related applications

[Fourdee]

https://github.com/Fourdee/DietPi/issues/2067#issuecomment-435872398

Sonarr, Radarr, Lidarr, Jackett

[MichaIng]

Perhaps there is a way to disable database logging. Would make sense at least to do either the one or the other. Would be great anyway, since SQLite database logging is quite I/O intensive. Every single change needs a single large file to be rewritten... Okay with single large log files it's the same, but those can usually be cleared easily, thus kept in RAM.

[Dr0bac]

So, i watched log files to see how often they are changed..
And real problem is in

nzbdrone.db-wal and
nzbdrone.db-shm

They are changing every minute

My Sonarr check for tv show releases every 20 minutes (default is 15 minutes) and that's is logged in

logs.db-wal
logs.db-shm and
updates/sonarr.txt

I screenshoted a sonarr tasks and you can see what sonarr does and how often..
I guess check for finished download are stored in nzbdrone.db-wal but not sure, just guessing

So sonarr.txt is not big problem, nzbdrone.db files are

[Fourdee]

@Dr0bac

👍 Thank you, much appreciated.

Perhaps there is a way to disable database logging.

@MichaIng Yep, would be the idea solution 👍

Failing that, check contents of the .db files, and, see if we could reset them every 1 hour for location in /var/log.

[Fourdee]

Test, removal of all logs and db during runtime (eg: we can then log to /var/log for all and clear on demand)

rm -R /mnt/dietpi_userdata/sonarr/logs*
mkdir -p /mnt/dietpi_userdata/sonarr/logs
chown -R sonarr:dietpi /mnt/dietpi_userdata/sonarr/logs
dietpi-services restart

root@Fuzon:~# ls -lha /mnt/dietpi_userdata/sonarr/
total 12M
drwxrwxrwx 1 root root 4.0K Nov 16 19:28 .
drwxrwxrwx 1 root root 4.0K Nov 15 13:09 ..
drwxrwxrwx 1 root root    0 Nov 15 16:32 Backups
-rwxrwxrwx 1 root root  465 Nov 10 23:35 config.xml
-rwxrwxrwx 1 root root 1.1M Nov 16 16:29 .fuse_hidden0000461800002a94
-rwxrwxrwx 1 root root 4.5M Nov 16 19:28 .fuse_hidden0000461a00002a96
-rwxrwxrwx 1 root root  32K Nov 16 19:28 .fuse_hidden0000461b00002a95
drwxrwxrwx 1 root root  392 Nov 16 19:28 logs
-rwxrwxrwx 1 root root 4.0K Nov 16 19:28 logs.db
-rwxrwxrwx 1 root root  32K Nov 16 19:28 logs.db-shm
-rwxrwxrwx 1 root root    0 Nov 16 19:28 logs.db-wal

• 🈯️ Check GUI logs page, all appears fine.
  Only issue, requires a restart everytime we want to clear logs. This can take a long time on lower end SBC's and the performance gain of reduced IO is offset by this.

[Fourdee]

Test logs to /dev/null

systemctl stop sonarr
rm -R /mnt/dietpi_userdata/sonarr/logs*
ln -sf /dev/null /mnt/dietpi_userdata/sonarr/logs
ln -sf /dev/null /mnt/dietpi_userdata/sonarr/logs.db
ln -sf /dev/null /mnt/dietpi_userdata/sonarr/logs.db-shm
ln -sf /dev/null /mnt/dietpi_userdata/sonarr/logs.db-wal
systemctl start sonarr

• 🈴 Check GUI logs page, causes application to crash.

[Fourdee]

Ok, lets go with .txt logs only to /var/log/sonarr?

Appears .db always logs everything, regardless of log level.

You can enable Trace or Debug logging in Settings on the General tab. Sonarr does not need to restarted for the change to take effect. This change only effects the log files, not the logging database.

[Dr0bac]

@Fourdee

Logs can be cleared from sonarr web ui.. maybe we can trigger that every ??? minutes within config file or similar..

Clearing Logs

You can clear log files and the logs database directly from the UI, under System -> Logs -> Files and System -> Logs respectively

Also is there a way to store those nzbdrone.db and logs.db files in ram without deleting it every ??? minutes..those files dont take to much space..one is 30 kb, another is 330 kb..i can track if those files get larger in time..

[Fourdee]

@Dr0bac

Logs can be cleared from sonarr web ui.. maybe we can trigger that every ??? minutes within config file or similar..

Great idea, but I don't believe we can, lack any API/CLI for web control outside of its webpage.
No documentation on the available config.xml options either from what I can see.

Also is there a way to store those nzbdrone.db and logs.db files in ram without deleting it every ??? minutes..those files dont take to much space..one is 30 kb, another is 330 kb..i can track if those files get larger in time..

Could work, we can move to /var/log and exclude the db files from clear. Only issue is memory usage. Although, we have a 2GB - RAM swapfile, so we should be covered.

Hmm, but if system left on for X days, could cause a out of memory error. Hmm, needs more investigation.

-rwxrwxrwx 1 root root 152K Nov 17 16:30 logs.db
-rwxrwxrwx 1 root root  32K Nov 17 17:17 logs.db-shm
-rwxrwxrwx 1 root root 4.0M Nov 17 17:16 logs.db-wal
drwxrwxrwx 1 root root 4.0K Nov  9 16:26 MediaCover
-rwxrwxrwx 1 root root 5.8M Nov 17 17:29 nzbdrone.db
-rwxrwxrwx 1 root root  32K Nov 17 17:30 nzbdrone.db-shm
-rwxrwxrwx 1 root root  21K Nov 17 17:29 nzbdrone.db-wal
-rwxrwxrwx 1 root root    4 Nov 16 20:46 nzbdrone.pid

[Fourdee]

Radarr/Sonarr/Lidarr:
🈯️ Install
🈯️ log clear + UI fine
🈯️ Reinstall

[Dr0bac]

@Fourdee i asked with others about this issue sonar devs..and here is conversation, if you want to understand more about problem

https://github.com/Sonarr/Sonarr/issues/2373

[MichaIng]

@Dr0bac
I also added my two cents there. Since SQLite it the database system of choice, which will not change fast or ever, at least some more flexibility for end user would be great: Allow to choose txt-only logging and allow to choose own background job schedule.

But the devs are right, that it's also about the chosen drive and server system in general. Such things as a database should be neither stored on an SDcard (for wear reasons), nor on an external HDD that is otherwise not/rarely used, thus will spin up and down then regularly. The latter also results in increased HDD wear.

• Best would be either an SSD, which allows enough I/O to survive this task for years and on the other hand does not need to spin up and down.
• On the other hand, when having a server with many software and regular disk I/O intense tasks (database, logging, downloading, whatever), then it's most properly best to disable spindown of external HDD and leave it running 24/7. This increases performance as well (delay for drive to spin up), reduces risk that some software might break/throw errors, since drive is not ready fast enough, and reduces start/stop wear on HDD. Of course this requires a server HDD which is made for this. NAS HDDs might survive as well, but are actually not made for 24/7 _access_, only 24/7 _uptime_.

[Dr0bac]

@MichaIng thanks for suggestions.. I download/watch movies pretty rarely..external hdd is spinning maybe 2-4 hours in one day, some days it's not spinning at all.. this is good for power consumption..that's the main reason i bought raspberry pi..because is very cheap to leave it on power 24/7.. also i'm afraid my hdd will not last few years with constant read/write access 24/7, my hdd is cheap external hdd

https://www.wd.com/products/portable-storage/wd-elements-portable.html#WDBUZG0010BBK-NESN

Maybe i can buy some cheap flash drive and connect to raspberry pi and put all those disk access heavy files on that..is that good idea? Is flash drive better than micro sd card for that purpose?

[MichaIng]

@Dr0bac
Ah yeah, this drive doesn't look like one that is made for 24/7 uptime. You can have luck for years or unluck with this, no chance to predict, I guess 😉. But if you have sensitive/important data on the drive, that should never get lost, I guess it's better to use something different.

Me e.g. use a WD Red (~4-5 years old now) to store cloud files databases and such. MariaDB caching and stuff like that prevents it from spinning up more than once an hour (moreless), and spinning down after 10 minutes (by HDD docking station, can't be controlled...). But at least this HDD is "officially" made for 24/7 uptime, warranty and I/O cycles/overall written data values are significantly hight than on cheaper kind of HDDs (and power consumption lower): https://www.wd.com/products/internal-storage/wd-red.html#WD30EFRX

An USB flash drive would survive most properly longer than an micro SDcard jep. In many cases it is adviced to move the whole root fs to USB flash on SBCs instead of using a micro SDcard at all (or for boot partition only). But I have no numbers in mind, guess a cheap USB flash can be worse than a quality micro SD still 😉.

However, whatever solution you have, and it's always about the price/costs for the setup as well of course, simply do regular backups, keep most important files in constant sync with a client system (cloud solution) and such, so whatever happens to your drives, you can always recover the system from last backup and most importantly have your data on a client system still available, if server drive crashes (and the other way round).

[Fourdee]

.db* files also moved to /var/log. They are not cleared, excluded from dietpi-logclear.

Tests:

• 🈯️ All GUI + Reboot to clear .db files.

root@DietPi:~# ls -lha /var/log/sonarr/
total 1.3M
drwxr-xr-x 2 sonarr dietpi  120 Nov 18 18:19 .
drwxr-xr-x 7 root   root    280 Nov 18 18:14 ..
-rw-r--r-- 1 sonarr dietpi 4.0K Nov 18 18:19 logs.db
-rw-r--r-- 1 sonarr dietpi  32K Nov 18 18:19 logs.db-shm
-rw-r--r-- 1 sonarr dietpi 1.3M Nov 18 19:30 logs.db-wal
-rw-r--r-- 1 sonarr dietpi  317 Nov 18 19:30 sonarr.txt

root@DietPi:~# ls -lha /mnt/dietpi_userdata/sonarr/
total 564K
drwxrwxr-x 2 sonarr dietpi 4.0K Nov 18 19:38 .
drwxrwxr-x 9 dietpi dietpi 4.0K Nov 18 18:14 ..
-rw-r--r-- 1 sonarr dietpi  306 Nov 18 18:19 config.xml
lrwxrwxrwx 1 sonarr dietpi   15 Nov 18 18:14 logs -> /var/log/sonarr
lrwxrwxrwx 1 sonarr dietpi   23 Nov 18 18:14 logs.db -> /var/log/sonarr/logs.db
lrwxrwxrwx 1 sonarr dietpi   27 Nov 18 18:14 logs.db-shm -> /var/log/sonarr/logs.db-shm
lrwxrwxrwx 1 sonarr dietpi   27 Nov 18 18:14 logs.db-wal -> /var/log/sonarr/logs.db-wal
-rw-r--r-- 1 sonarr dietpi 340K Nov 18 19:38 nzbdrone.db
-rw-r--r-- 1 sonarr dietpi  32K Nov 18 19:41 nzbdrone.db-shm
-rw-r--r-- 1 sonarr dietpi 170K Nov 18 19:41 nzbdrone.db-wal
-rw-r--r-- 1 sonarr dietpi    4 Nov 18 18:19 nzbdrone.pid

[Dr0bac]

@Fourdee thank you for moving log files to ram storage.. i will track how it's behave.. But you didnt include most active ones in your code..

-rw-r--r-- 1 sonarr dietpi 32K Nov 18 19:41 nzbdrone.db-shm
-rw-r--r-- 1 sonarr dietpi 170K Nov 18 19:41 nzbdrone.db-wal
...........................are the worst ones, they are updating every minute

lrwxrwxrwx 1 sonarr dietpi 27 Nov 18 18:14 logs.db-shm
lrwxrwxrwx 1 sonarr dietpi 27 Nov 18 18:14 logs.db-wal
...........................these update when sonarr search for new episodes (every 15 minutes) - you included them
...........................also with 2 abobe files, this file update every 15 minutes
-rw-r--r-- 1 sonarr dietpi 340K Nov 18 19:38 nzbdrone.db

lrwxrwxrwx 1 sonarr dietpi 15 Nov 18 18:14 logs
..........................txt files in this folder also update every 15 minutes - you included it

lrwxrwxrwx 1 sonarr dietpi 23 Nov 18 18:14 logs.db
........................this file update rarely, last time it's updated few hours ago, but ok..

can you include
nzbdrone.db-shm - update every minute
nzbdrone.db-wal - update every minute
nzbdrone.db - update every 15 minutes

[Fourdee]

@Dr0bac

nzbdrone.db-shm - update every minute
nzbdrone.db-wal - update every minute
nzbdrone.db - update every 15 minutes

I believe these are the database files for added series, status etc? Not log files.

We cant really place these in /var/log, if I'am correct, a reboot will clear all user settings + added series.

[Dr0bac]

You are correct.. i stoped radarr service and moved these files from appdata and started radarr after that..settings were reverted to default state also movies were disapeared..
its a bummer, nzbdrone.db-shm and nzbdrone.db-wal are worst ones in appdata folder

[Dr0bac]

One last request.. i always download movies and tv shows manually, also i have script that delete movies from radarr upon successuful download..so in most times these database files don't need to be up to date..
Can you tell me how can i put these 3 files in ram
And to backup these files from ram to sd card 2 times a day..and after restart before radarr service start to copy back to ram these files? Also can copy to backup folder be fired when raspberry pi is in shutdown process?

Edit: or better yet, i can make a script to backup these files from ram every time after movie is downloaded with radarr/sonarr post processing script, that way i will always have latest important database file after restart

Edit2:

So first i need to put nzbdrone files to ram.. i don't know how to do that..
After that i need to create a script that copy these files to backup folder after movie is downloaded

#!/bin/bash

    cp /var/log/radarr/nzbdrone.db /home/dietpi/radarr-database-backup/nzbdrone.db
    cp /var/log/radarr/nzbdrone.db-shm /home/dietpi/radarr-database-backup/nzbdrone.db-shm
    cp /var/log/radarr/nzbdrone.db-wal /home/dietpi/radarr-database-backup/nzbdrone.db-wal

and then make that script executable chmod +x script name

and also i can put that script to be fired up during shutdown/restart like someone described here
https://unix.stackexchange.com/questions/48973/execute-a-command-before-shutdown

and after restart/power on i can make reverse script to copy nzbdrone files from backup folder to ram destination with crontab -e
@reboot /script name
does reboot command in crontab execute before radarr service start?

That's all i need.. can you help me with first task, to move nzbdrone files to ram?

Edit3: I looked trough your changes and is this command for file to be moved to ram?

ln -sf /var/log/sonarr/logs.db $G_FP_DIETPI_USERDATA/sonarr/logs.db

In my case i need to write

ln -sf /var/log/sonarr/nzbdrone.db /mnt/dietpi_userdata/sonarr/nzbdrone.db
ln -sf /var/log/sonarr/nzbdrone.db-shm /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm
ln -sf /var/log/sonarr/nzbdrone.db-wal /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal

But before that i must move these files first to var/log?
So nzbdrone files in dietpi_userdata will be shortcuts to /var/log files?
and i see that you exluded .db files from deleting in var/log folder.. so i don't need to worry about that?
is this all i need to do or I'm missing something? @Fourdee @MichaIng

[Dr0bac]

@Fourdee
I tested database symlinks and everything works, can i just ask you few questions, before i automate everything

I will copy nzbdrone files to var/log/sonarr with crontab reboot option, i see in your code that you created sonarr folder in /var/log for log files.. So i need to know:

1. does crontab reboot is happening after sonarr folder is created in var/log, i ask to know if i need to create folder with script? (i guess that sonarr folder is deleted in ram on every reboot)
   and does crontab reboot is happening before sonarr is started?

2. And i will put script in /etc/rc0.d and /etc/rc6.d to backup nzbdrone files before device is turned off
   does scripts are fired in those folders before ram is cleared or after? what operation come first when device is shutdown process

Thanks

[MichaIng]

@Dr0bac
DietPi-RAMlog will create an empty file structure of /var/log on disk on shutdown and restores it on boot. So no need to manually re-create the folder :smiley:.
But I am not sure what you mean by crontab reboot option? You mean a reboot initiated by a cron script?

I am pretty sure the rcX.d scripts are initiated before any unmount is done, thus also before /var/log is cleared (related tmpfs is unmounted). But I am not 100% sure how strict/reliable this is.
Your aim is to preserve the sonarr logs through reboot? The issue with this is that /var/log content is cleared every hour anyway, so no point to backup anything from there anyway. You can use the other logging mode that stores logs to disk before clearing, then you also won't need any manual backup.

[Dr0bac]

@MichaIng no i dont want to reboot with script
I want to add script to crontab -e to execute after reboot, script will copy data files from backup folder to var/log/sonarr
But i saw that Fourdee added code to exclude .db files from deleting in /var/log? See this commit.. Am I right?
6f758f0b68b6025c7a9d0b0e2a0fd794903242bc

[Fourdee]

@Dr0bac

But i saw that Fourdee added code to exclude .db files from deleting in /var/log? See this commit.. Am I right?
6f758f0

Correct, any file with *.db* will be excluded from the hourly log clear.

If crontab runs after dietpi-services, should be fine.

You are trying to achieve moving the core database to RAM during boot, then save to disk during shutdown? Great idea, but the only issue to worry about, is, if you have a power failure, database will revert back to its previous state.

Hmm, the best solution would be to create a service that runs before sonarr.service to copy the files, then save on shutdown.

Was thinking about doing this the other day, If I get a chance later i'll create one.

[MichaIng]

Ah jep now I fully understand. Makes sense, so backup .db log on shutdown and recover on boot. Couldn't we just add this to sonarr service ExecPreStart/Stop? But would run then on every dietpi-service handling... Perhaps recover only, if /var/log .db files are empty (on boot). But not sure how to have the backup only done on shutdown, if required.

Otherwise optional additional service that is handled by systemd.

[Dr0bac]

@Fourdee yes, on power failure latest database will be lost..and system will restore database from last sucessuful restart/shutdown after next power on..
That's why i will put same backup script to sonarr post processing script (script will be triggered on episode download) i download everything manually so if power failure happen, i will not lose anything, but some users who have everything automated will have trouble if this happen!?
Is script in crontab -e that look like this

@reboot /home/dietpi/restore.sh

Is trigered before or after service sonarr start? I need to be triggered before sonarr

@MichaIng i don't need to backup log files, i want to backup nzbdrone.db files witch are database,and they are the reasons why disk is spinning every minute.. If they are lost, all custom settings and progress are lost.. but yes..you understand what's i'm aiming for..i just fear if power lost is happen, some users will be confused with their sonarr downloading some episodes again and similar

[MichaIng]

Not sure how well cron can handle service dependencies. Never used this for (re)boot/shutdown scripts. Generally I would use systemd units for this.
Trigger additional db backups on sonarr downloads and/or regularly via cron job, to prevent data loss on lower loss.

Since dietpi-services starts the cron service last, I guess it is with current implementation not possible to have it starting something before any other service was started. Inside DietPi-Services it would need a reorder. So jep, stay with systemd unit that starts after dietpi-ramlog.service and before dietpi-postboot.service at latest.

[Fourdee]

Core script done:
https://raw.githubusercontent.com/Fourdee/DietPi/dev/dietpi/misc/dietpi-arr_to_RAM

Must be run manually to transfer to RAM:

dietpi-arr_to_RAM 1

Save to disk before shutdown:

dietpi-arr_to_RAM 0

May edit to a service, if time permits.

---

Test:

root@DietPi:~# /DietPi/dietpi/dietpi-arr_to_RAM 1
[  OK  ] DietPi-Arr_to_RAM | Root access verified.
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/sonarr/nzbdrone.db /mnt/dietpi_userdata/sonarr/nzbdrone.db.bak
[ .... ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/sonarr/nzbdrone.db /tmp/son[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/sonarr/nzbdrone.db /tmp/sonarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/sonarr/nzbdrone.db
[ .... ] DietPi-Arr_to_RAM | ln -sf /tmp/sonarr/nzbdrone.db /mnt/dietpi_userdata[  OK  ] DietPi-Arr_to_RAM | ln -sf /tmp/sonarr/nzbdrone.db /mnt/dietpi_userdata/sonarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm.bak
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm /tmp/sonarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | ln -sf /tmp/sonarr/nzbdrone.db-shm /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal.bak
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal /tmp/sonarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | ln -sf /tmp/sonarr/nzbdrone.db-wal /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/radarr/nzbdrone.db /mnt/dietpi_userdata/radarr/nzbdrone.db.bak
[ .... ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/radarr/nzbdrone.db /tmp/rad[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/radarr/nzbdrone.db /tmp/radarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/radarr/nzbdrone.db
[ .... ] DietPi-Arr_to_RAM | ln -sf /tmp/radarr/nzbdrone.db /mnt/dietpi_userdata[  OK  ] DietPi-Arr_to_RAM | ln -sf /tmp/radarr/nzbdrone.db /mnt/dietpi_userdata/radarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/radarr/nzbdrone.db-shm /mnt/dietpi_userdata/radarr/nzbdrone.db-shm.bak
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/radarr/nzbdrone.db-shm /tmp/radarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/radarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | ln -sf /tmp/radarr/nzbdrone.db-shm /mnt/dietpi_userdata/radarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/radarr/nzbdrone.db-wal /mnt/dietpi_userdata/radarr/nzbdrone.db-wal.bak
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/radarr/nzbdrone.db-wal /tmp/radarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/radarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | ln -sf /tmp/radarr/nzbdrone.db-wal /mnt/dietpi_userdata/radarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/lidarr/lidarr.db /mnt/dietpi_userdata/lidarr/lidarr.db.bak
[ .... ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/lidarr/lidarr.db /tmp/lidar[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/lidarr/lidarr.db /tmp/lidarr/lidarr.db
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/lidarr/lidarr.db
[ .... ] DietPi-Arr_to_RAM | ln -sf /tmp/lidarr/lidarr.db /mnt/dietpi_userdata/l[  OK  ] DietPi-Arr_to_RAM | ln -sf /tmp/lidarr/lidarr.db /mnt/dietpi_userdata/lidarr/lidarr.db
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/lidarr/lidarr.db-shm /mnt/dietpi_userdata/lidarr/lidarr.db-shm.bak
[ .... ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/lidarr/lidarr.db-shm /tmp/l[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/lidarr/lidarr.db-shm /tmp/lidarr/lidarr.db-shm
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/lidarr/lidarr.db-shm
[  OK  ] DietPi-Arr_to_RAM | ln -sf /tmp/lidarr/lidarr.db-shm /mnt/dietpi_userdata/lidarr/lidarr.db-shm
[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/lidarr/lidarr.db-wal /mnt/dietpi_userdata/lidarr/lidarr.db-wal.bak
[ .... ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/lidarr/lidarr.db-wal /tmp/l[  OK  ] DietPi-Arr_to_RAM | cp /mnt/dietpi_userdata/lidarr/lidarr.db-wal /tmp/lidarr/lidarr.db-wal
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/lidarr/lidarr.db-wal
[  OK  ] DietPi-Arr_to_RAM | ln -sf /tmp/lidarr/lidarr.db-wal /mnt/dietpi_userdata/lidarr/lidarr.db-wal


root@DietPi:~# /DietPi/dietpi/dietpi-arr_to_RAM 0
[  OK  ] DietPi-Arr_to_RAM | Root access verified.
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/sonarr/nzbdrone.db
[ .... ] DietPi-Arr_to_RAM | cp /tmp/sonarr/nzbdrone.db /mnt/dietpi_userdata/son[  OK  ] DietPi-Arr_to_RAM | cp /tmp/sonarr/nzbdrone.db /mnt/dietpi_userdata/sonarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | rm /tmp/sonarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | cp /tmp/sonarr/nzbdrone.db-shm /mnt/dietpi_userdata/sonarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | rm /tmp/sonarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | cp /tmp/sonarr/nzbdrone.db-wal /mnt/dietpi_userdata/sonarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | rm /tmp/sonarr/nzbdrone.db-wal
[ INFO ] DietPi-Arr_to_RAM | /tmp/sonarr/lidarr.db does not exist.
[ INFO ] DietPi-Arr_to_RAM | /tmp/sonarr/lidarr.db-shm does not exist.
[ INFO ] DietPi-Arr_to_RAM | /tmp/sonarr/lidarr.db-wal does not exist.
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/radarr/nzbdrone.db
[ .... ] DietPi-Arr_to_RAM | cp /tmp/radarr/nzbdrone.db /mnt/dietpi_userdata/rad[  OK  ] DietPi-Arr_to_RAM | cp /tmp/radarr/nzbdrone.db /mnt/dietpi_userdata/radarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | rm /tmp/radarr/nzbdrone.db
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/radarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | cp /tmp/radarr/nzbdrone.db-shm /mnt/dietpi_userdata/radarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | rm /tmp/radarr/nzbdrone.db-shm
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/radarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | cp /tmp/radarr/nzbdrone.db-wal /mnt/dietpi_userdata/radarr/nzbdrone.db-wal
[  OK  ] DietPi-Arr_to_RAM | rm /tmp/radarr/nzbdrone.db-wal
[ INFO ] DietPi-Arr_to_RAM | /tmp/radarr/lidarr.db does not exist.
[ INFO ] DietPi-Arr_to_RAM | /tmp/radarr/lidarr.db-shm does not exist.
[ INFO ] DietPi-Arr_to_RAM | /tmp/radarr/lidarr.db-wal does not exist.
[ INFO ] DietPi-Arr_to_RAM | /tmp/lidarr/nzbdrone.db does not exist.
[ INFO ] DietPi-Arr_to_RAM | /tmp/lidarr/nzbdrone.db-shm does not exist.
[ INFO ] DietPi-Arr_to_RAM | /tmp/lidarr/nzbdrone.db-wal does not exist.
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/lidarr/lidarr.db
[ .... ] DietPi-Arr_to_RAM | cp /tmp/lidarr/lidarr.db /mnt/dietpi_userdata/lidar[  OK  ] DietPi-Arr_to_RAM | cp /tmp/lidarr/lidarr.db /mnt/dietpi_userdata/lidarr/lidarr.db
[  OK  ] DietPi-Arr_to_RAM | rm /tmp/lidarr/lidarr.db
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/lidarr/lidarr.db-shm
[ .... ] DietPi-Arr_to_RAM | cp /tmp/lidarr/lidarr.db-shm /mnt/dietpi_userdata/l[  OK  ] DietPi-Arr_to_RAM | cp /tmp/lidarr/lidarr.db-shm /mnt/dietpi_userdata/lidarr/lidarr.db-shm
[  OK  ] DietPi-Arr_to_RAM | rm /tmp/lidarr/lidarr.db-shm
[  OK  ] DietPi-Arr_to_RAM | rm /mnt/dietpi_userdata/lidarr/lidarr.db-wal
[ .... ] DietPi-Arr_to_RAM | cp /tmp/lidarr/lidarr.db-wal /mnt/dietpi_userdata/l[  OK  ] DietPi-Arr_to_RAM | cp /tmp/lidarr/lidarr.db-wal /mnt/dietpi_userdata/lidarr/lidarr.db-wal
[  OK  ] DietPi-Arr_to_RAM | rm /tmp/lidarr/lidarr.db-wal

[Dr0bac]

@Fourdee Thank you for making this. Can you make some changes. In current way script behave like this

first we backup nzbdrone.db to nzbdrone.db.bak
cp /mnt/dietpi_userdata/sonarr/nzbdrone.db /mnt/dietpi_userdata/sonarr/nzbdrone.db.bak

then we copy nzbdrone.db to ram and delete it from dietpi_userdata
cp /mnt/dietpi_userdata/sonarr/nzbdrone.db /tmp/sonarr/nzbdrone.db
rm /mnt/dietpi_userdata/sonarr/nzbdrone.db

then we link it from ram to dietpi_userdata
ln -sf /tmp/sonarr/nzbdrone.db /mnt/dietpi_userdata/sonarr/nzbdrone.db

on shutdown process we remove symlink from dietpi_userdata
rm /mnt/dietpi_userdata/sonarr/nzbdrone.db

copy nzbdrone.db back from ram to dietpi_userdata
cp /tmp/sonarr/nzbdrone.db /mnt/dietpi_userdata/sonarr/nzbdrone.db

and remove nzbdrone.db from ram
rm /tmp/sonarr/nzbdrone.db

problem is, if power loss happen, on next power on we will have symlink in dietpi_userdata.. and process will start with copying symlink file from dietpi_userdata to to .bak file and to ram
That way we will lose backup .bak file.. and sonarr database will be empty symlink file..

maybe is better to in shutdown process we copy nzbdrone.db from ram to backup file nzbdrone.db.bak

and in power on process we can delete symlink from dietpi_userdata and copy nzbdrone.db.bak to ram tmp/sonarr/nzbdrone.db and create symlink again (or we don't need to recreate symlink again every time?)

if we delete symlink during shutdown process, when power loss happen, that symlink will stay in current build of script after next power on.. so maybe it's better to delete it on power on script..

that's way if power loss happen, we will have symlink again when system start, but when script start, we will copy .bak backup file to ram folder, and not symlink..

I hope you understand what i'm trying to say

Also with .bak backup i can update backup periodicaly (after sonarr download episode) with sonarr post procesing script. because, system can be on for months, and in that time we will not update backup file just with shutdown process.. that way sonarr database will always be up to date

[Fourdee]

@Dr0bac

problem is, if power loss happen, on next power on we will have symlink in dietpi_userdata.. and process will start with copying symlink file from dietpi_userdata to to .bak file and to ram

Hmm, should not occur:

• Script will only copy from /mnt/dietpi_userdata/sonarr/nzbdrone.db if its not a symlink. Else, if it is a symlink, it go down the backup route.
• /tmp is cleared during power off state

elif [[ ! -L $FP_SOURCE && -f $FP_SOURCE ]]; then #if not a symlink

                    Copy_To_Ram

                # - recover + copy to RAM
                elif [[ -f ${FP_SOURCE}.bak ]]; then

                    rm $FP_SOURCE &> /dev/null

                    G_ERROR_HANDLER_INFO_ONLY=1 G_RUN_CMD cp ${FP_SOURCE}.bak $FP_SOURCE
                    Copy_To_Ram

                fi

[Fourdee]

@Dr0bac

with sonarr post procesing script. because, system can be on for months, and in that time we will not update backup file just with shutdown process.. that way sonarr database will always be up to date

Yep 👍 was thinking about a background script which updates the .bak on a hourly basis.

Although, could just run a cron job that does following every hour:

• Downside is service restarts during the process, but handled via same script.

dietpi-arr_to_RAM 0
dietpi-arr_to_RAM 1

[Dr0bac]

@Fourdee thank you for clarify to me..i was reading your output message..

No need for sonarr service to stop every hour, we can just put in crontab

/tmp/sonarr/nzbdrone.db /mnt/dietpi_userdata/sonarr/nzbdrone.db.bak

nzbdrone.db.bak is inactive so no harm to change it every hour

But i will put that command in sonarr post processing script, because i dont want to wake my hdd every hour.. when post processing script happen, my hdd will be already wake, because episode was just downloaded.

But with that route, we need on system start to copy .bak file to /tmp/ every time..

[Dr0bac]

@Fourdee hi..
You said few months ago that you will make startup service from this script when you have time..maybe you forgot about it? If you have time now, can you try?
Also can you change script, its better to always copy nzbdrone.db.bak file to var/log after power on.. because i want to update that bak. file after movie is downloaded so database can always be fresh if i got power loss..

Thanks