Dietpi: General | Recheck $FP_WORK_DIR

[MichaIng]

Creating a bug report/issue:

Required Information:

• DietPi version | 6.13 testing branch
• Distro version | stretch

Additional Information (if applicable):

• Software title | DietPi-Globals, observed within DietPi-Drive_Manager

Steps to reproduce:

1. Move dietpi_userdata via DietPi-Drive_Manager

Expected behaviour:

• After move has finished, drive manager current drive edit should show up as before.

Actual behaviour:

• Several error messages about missing scrape files appear, drive manager whip menu without much content

Extra details:

• Error messages:

[ INFO ] DietPi-Set_userdata | /mnt/dietpi_userdata is already symlinked to target directory.

[  OK  ]  Completed
cp: cannot create regular file '.fstab': No such file or directory
grep: .fstab: No such file or directory
grep: .fstab: No such file or directory
grep: .fstab: No such file or directory
grep: .fstab: No such file or directory
grep: .fstab: No such file or directory
grep: .fstab: No such file or directory
/DietPi/dietpi/dietpi-drive_manager: line 160: .fstab: No such file or directory
[ INFO ] DietPi-Drive_Manager | Detecting drives, please wait...
[  OK  ] DietPi-Drive_Manager | Detecting currently mounted drives, please wait...
/DietPi/dietpi/dietpi-drive_manager: line 192: .dietpi-drive_manager_df_tmp: No such file or directory
/DietPi/dietpi/dietpi-drive_manager: line 117: .dietpi-drive_manager_df_tmp: No such file or directory
rm: cannot remove '.dietpi-drive_manager_df_tmp': No such file or directory
[  OK  ] DietPi-Drive_Manager | Detecting unmounted drives, please wait...
/DietPi/dietpi/dietpi-drive_manager: line 324: .dietpi-drive_manager_blkid_tmp: No such file or directory
/DietPi/dietpi/dietpi-drive_manager: line 117: .dietpi-drive_manager_blkid_tmp: No such file or directory
rm: cannot remove '.dietpi-drive_manager_blkid_tmp': No such file or directory
/DietPi/dietpi/dietpi-drive_manager: line 381: .dietpi-drive_manager_lsblk_tmp: No such file or directory
/DietPi/dietpi/dietpi-drive_manager: line 117: .dietpi-drive_manager_lsblk_tmp: No such file or directory
rm: cannot remove '.dietpi-drive_manager_lsblk_tmp': No such file or directory
cp: cannot stat '.fstab': No such file or directory
rm: cannot remove '.fstab': No such file or directory
/DietPi/dietpi/dietpi-drive_manager: line 1070: ((: !  : syntax error: operand expected (error token is "!  ")
/DietPi/dietpi/dietpi-drive_manager: line 1210: ((: !  && !  : syntax error: operand expected (error token is "&& !  ")

• Opening a second SSH sessions reveals, that /tmp/dietpi-drive_manager is not available anymore:

2018-08-07 18:13:02 root@VM-Stretch:~# ls -al /tmp
total 4
drwxrwxrwt  7 root root  140 Aug  7 18:13 .
drwxr-xr-x 21 root root 4096 Jul 29 12:33 ..
drwxrwxrwt  2 root root   40 Aug  7 17:54 .font-unix
drwxrwxrwt  2 root root   40 Aug  7 17:54 .ICE-unix
drwxrwxrwt  2 root root   40 Aug  7 17:54 .Test-unix
drwxrwxrwt  2 root root   40 Aug  7 17:54 .X11-unix
drwxrwxrwt  2 root root   40 Aug  7 17:54 .XIM-unix

• More related errors on navigation, even shell errors on quit:

shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory

• Perhaps somehow the "trap" on quitting the script, has access to the parent script variable $G_WORK_DIR and removes it as cleanup. This should actually not be the case, but I have no other idea. Investigation required!

[MichaIng]

Recheck:

2018-08-07 18:47:26 root@VM-Stretch:/tmp# cat script1
#!/bin/bash
FP_WORK_DIR=/tmp/script1_dir
trap 'rm -R $FP_WORK_DIR && echo script1 removed $FP_WORK_DIR' EXIT

echo "script1 is in $PWD"
mkdir -p $FP_WORK_DIR
cd $FP_WORK_DIR
echo "script1 moved to $PWD"

/tmp/script2

echo "script1 is in $PWD"
2018-08-07 18:47:36 root@VM-Stretch:/tmp# cat script2
#!/bin/bash
FP_WORK_DIR=/tmp/script2_dir
trap 'rm -R $FP_WORK_DIR && echo script2 removed $FP_WORK_DIR' EXIT

echo "script2 is in $PWD"
mkdir -p $FP_WORK_DIR
cd $FP_WORK_DIR
echo "script2 moved to $PWD"

/tmp/script3

echo "script2 is in $PWD"
2018-08-07 18:47:38 root@VM-Stretch:/tmp# cat script3
#!/bin/bash
trap '[[ -d $FP_WORK_DIR ]] && rm -R $FP_WORK_DIR && echo script3 removed $FP_WORK_DIR' EXIT

echo "script3 is in $PWD"
cd $HOME
echo "script3 moved to $PWD"
2018-08-07 18:47:40 root@VM-Stretch:/tmp# ./script1
script1 is in /tmp
script1 moved to /tmp/script1_dir
script2 is in /tmp/script1_dir
script2 moved to /tmp/script2_dir
script3 is in /tmp/script2_dir
script3 moved to /root
script2 is in /tmp/script2_dir
script2 removed /tmp/script2_dir
script1 is in /tmp/script1_dir
script1 removed /tmp/script1_dir

• Actually everything works as expected. $FP_WORK_DIR is not known in subscripts (until assigned), as well not in "trap". Subscripts start within the working dir of parent script. If they cd somewhere else, parent script remains within it's dir => cd does not apply for parent scripts/shells, just subscripts/shells, similar to exported variables.

Still not sure what happens here...

[MichaIng]

Debug via second SSH session:
Main:

...
'/mnt/other_userdata/Video' -> '/mnt/dietpi_userdata/Video'
[  OK  ] DietPi-Drive_Manager | RootFS R/W access verified.

[  OK  ] DietPi-Software | Initialized database
[  OK  ] DietPi-Software | Reading database completed
[  OK  ] DietPi-Software | Set permissions completed
[ SUB2 ] DietPi-Services > start
[  OK  ] DietPi-Services | start : mysql
[  OK  ] DietPi-Services | start : php7.0-fpm
[  OK  ] DietPi-Services | start : lighttpd
[  OK  ] DietPi-Services | start : cron
[ SUB3 ] DietPi-Process_tool > Apply
[  OK  ] DietPi-Process_tool | Completed
[ INFO ] DietPi-Set_userdata | /mnt/dietpi_userdata is already symlinked to target directory.

[  OK  ]  Completed
cp: cannot create regular file '.fstab': No such file or directory
...

• dietpi_userdata successfully moved.
• dietpi-software setpermissions call
• 🈺 dietpi-services call before dietpi-userdata script finishes. This is actually wrong here, as for dietpi-drive_manager we expect services to be stopped.
-

In parallel second session observes content of /tmp:

2018-08-07 19:53:30 root@VM-Stretch:/tmp# while :; do ls; sleep 0.2; done
...
dietpi-drive_manager
dietpi-drive_manager
dietpi-drive_manager  dietpi-software
dietpi-drive_manager  dietpi-software
dietpi-process.pid  dietpi-software
dietpi-process.pid  dietpi-software
dietpi-process.pid  dietpi-software
dietpi-process.pid  dietpi-software
dietpi-process.pid  dietpi-software
dietpi-process.pid  dietpi-software
dietpi-process.pid  dietpi-software
dietpi-software
dietpi-software
dietpi-software
dietpi-software
...

• /tmp/dietpi-drive_manager is removed shortly after dietpi-software init. It must happen ....

AHHH, found the issue:

• DietPi-Drive_Manager calls DietPi-Set_Userdata calls DietPi-Software calls again DietPi-Drive_Manager for rootfs r/w check, which removes it's own work dir after exit.

Solution:

• Move rootfs r/w check to globals. Loading while drive_manager for this trivial check is anyway much overhead, and we need this quite often.
• Also dietpi-set_userdata should not call dietpi-software to set permissions. Deny moving userdata to drive without permissions support, then rely on cp -a to keep permissions.
• This is an exceptional case, but we need to take care about dietpi-* script loops, generally try to minimize script calls from within scripts. func/dietpi-set_* of course are there for this, but they should not call the main scripts again at best. Then better move the related features/functions to globals.
• Again thinking about a parallel script execution handling. Check for same script already runs (reliable way via ps/pgrep? Otherwise .flag or simply check for existing $FP_WORK_DIR?), then either wait... for other script call (e.g. cron as well and stuff) to finish, or better pop whiptail menu to inform user, that second call needs to finished, e.g. on second terminal actively in case, then go on + recheck with [ Ok ] button. More difficult how to handle when script is called non-interactively. Wait for a certain time? Directly exit? But exit could lead to possible parent script to fail unexpectedly, if no good child script exit code handling is done.
  
  
  
  • Alternatively let scripts on exit trap check for second execution and in case not touch the $FP_WORK_DIR?
  
  

[MichaIng]

Fixed within: https://github.com/Fourdee/DietPi/pull/1994

[Fourdee]

@MichaIng

Move rootfs r/w check to globals. Loading while drive_manager for this trivial check is anyway much overhead, and we need this quite often.

Yep, great work, makes sense to global this 👍

There are a few rare cases where DietPi scripts can re-run multiple instances of itself. Ideally, we should add something like this to the non-func scripts (eg: dietpi-software/config etc):

if (( $G_PROGRAM_NAME pgrep active )); then

already running
exit

Should be easier to debug and resolve all chances of multiple duplicate script instance loops?

---

It may also make sense to simply remove the contents of rm -R /tmp/dietpi-software/* instead of the parent directory itself?

[MichaIng]

@Fourdee
Jep, thats what I thought about. Could be placed into G_INIT().

[MichaIng]

@Fourdee
Did a start: https://github.com/Fourdee/DietPi/pull/1998

This prompts whiptail with info and choice to user. Concurrent executions should be impossible with this, first the second instance need to be closed. Otherwise cancel to revert to parent script.
But this needs some testing, especially how our scripts handle errors on child scripts.
Cron/SystemD executions will cancel automatically (NO default choice).

Let's play around where loops occur.

[Fourdee]

@MichaIng

Tests:

• 🈺 PREP install of v6.13 existing DietPi install
  -- Hung at install_stage -1. Had to manually login and run boot script to continue.

[MichaIng]

@Fourdee

pgrep -ci 'dietpi-drive'

Try pgrep -cf 'dietpi-drive_manager. -i is not supported on Jessie, but should also not be needed, -f searches the whole process name including path and arguments. With this, bash /DietPi/dietpi/dietpi-drive_manager is recognized, even that the base command was just bash.

Does not yet explain, why in your case $FP_WORK_DIR=/tmp/dietpi-drive_manager got removed, it seems during APT execution?

We really need to add debug output code on all this steps 🤔.

---

But yeah the check does not yet work reliably. Also found the following:

$0 does only include the path as how the script is started:
./script => $0=./script
bash script => $0=script
bash /path/to/script => $0=/path/to/script
bash alias => $0=/path/to/script

But also the process is listed (and checked with) just as the $0 above.
If the script is started via bash dietpi-software or ./dietpi-software from within cd /DietPi/dietpi, then pgrep -f /DietPi/dietpi/dietpi-software will not match 🤔.

Only solution I can currently thing of, is to just check for ${0##*/}, which will always be just the script name without path. Only problem is then with our "boot" and "login" scripts, where many other processes could match those names, by name, path or arguments.
Or we rely on users always start the scripts via alias (and our scripts themselves via full path).

---

It makes sense we track those issues down, but to be failure safe for now:
[[ $FP_WORK_DIR && -d $FP_WORK_DIR ]] && (( $(pgrep -cf "$0") <= 2 )) && rm $FP_WORK_DIR
So the work dir will stay, in case of concurrent executions.

Alternative:

• Keep $FP_WORK_DIR in place, just remove content?
• On concurrent execution even clearing content can lead to scripts breaking each other, but at least the above problem do not occur any more.

[Fourdee]

@MichaIng

Had an idea:

• under G_INIT, use the G_PROGRAM_NAME (lowercase?) to create /tmp/$G_PROGRAM_NAME and cd into it by default for all scripts.
• Then in the scripts, the temp fp and current directory of script is /tmp/$G_PROGRAM_NAME
• We can also check in G_INIT if the /tmp/$G_PROGRAM_NAME (and processes as a failsafe?) already exists, in which case, we can assume the program is already running and show the whiptail error for that.
• And G_EXIT will remove the /tmp/$G_PROGRAM_NAME dir on exit.

🈯️ Seems solid, don't believe we even need the process check in G_INIT.

root@DietPi:~# dietpi-backup 1
[  OK  ] RootFS R/W access verified.

[ INFO ] DietPi-Backup | Entered assigned working directory: /tmp/DietPi-Backup
[  OK  ] DietPi-Backup | Checking for pre-req APT packages: rsync
[ INFO ] DietPi-Backup | Pre-req APT packages are installed

 DietPi-Backup
─────────────────────────────────────────────────────
 Mode: Backup

[  OK  ] DietPi-Backup | DietPi-Userdata validation: /mnt/dietpi_userdata
[ INFO ] DietPi-Services | Entered assigned working directory: /tmp/DietPi-Services
[ SUB1 ] DietPi-Services > stop
[  OK  ] DietPi-Services | stop : cron
[ INFO ] DietPi-Services | Entered assigned working directory: /tmp/DietPi-Services
[ SUB1 ] DietPi-Services > stop
[  OK  ] DietPi-Services | stop : cron
[ INFO ] DietPi-Backup | Backing up to: /mnt/dietpi-backup
[  OK  ] DietPi-Backup | Free space check: path=/mnt/dietpi-backup/data | available=12459 MB | required=14 MB
[  OK  ] DietPi-Backup | rsync -aH --delete --delete-excluded --exclude-from=/tmp/.dietpi-backup_filter_inc_exc -v --log-file=/var/log/dietpi-backup.log / /mnt/dietpi-backup/data/
[ INFO ] DietPi-Services | Entered assigned working directory: /tmp/DietPi-Services
[ SUB1 ] DietPi-Services > start
[  OK  ] DietPi-Services | start : cron
[ INFO ] DietPi-Process_tool | Entered assigned working directory: /tmp/DietPi-Process_tool
[ SUB2 ] DietPi-Process_tool > Apply
[  OK  ] DietPi-Process_tool | Completed
root@DietPi:~# ls -lha /tmp
total 4.0K
drwxrwxrwt  7 root root  140 Aug 10 08:44 .
drwxr-xr-x 22 root root 4.0K Aug  9 12:18 ..
drwxrwxrwt  2 root root   40 Aug  9 14:23 .font-unix
drwxrwxrwt  2 root root   40 Aug  9 14:23 .ICE-unix
drwxrwxrwt  2 root root   40 Aug  9 14:23 .Test-unix
drwxrwxrwt  2 root root   40 Aug  9 14:23 .X11-unix
drwxrwxrwt  2 root root   40 Aug  9 14:23 .XIM-unix

• 🈯️ DietPi-Software install
• 🈯️ Dupe test of DietPi-Software detected

---

🈯️ VM v6.7 > v6.13

root@DietPi:~# ls -lha /etc/cron.daily/dietpi
-rwxr-xr-x 1 root root 1.2K Aug 10 10:25 /etc/cron.daily/dietpi

~Needs testing/checking, unsure if this would need to change to /tmp/DietPi-Update~

    if (( $EMR_INDEX == 0 )); then

        #Pre-6.9 preparations: Manually copy rootfs files in place: https://github.com/Fourdee/DietPi/pull/1802
        #   NB: dietpi-update takes over, post v6.9 update completion.
        GITBRANCH="$(grep -m1 '^[[:blank:]]*DEV_GITBRANCH=' /DietPi/dietpi.txt | sed 's/^[^=]*=//')"
        if (( $SUBVERSION_CURRENT < 9 )) && [[ -d /tmp/dietpi-update/DietPi-$GITBRANCH/rootfs ]]; then

            l_message='Copy rootfs files in place' G_RUN_CMD cp -Rf /tmp/dietpi-update/DietPi-"$GITBRANCH"/rootfs/. /
            l_message='Set execute permissions for DietPi scripts' G_RUN_CMD chmod -R +x /etc/cron.*/dietpi /var/lib/dietpi/services
            systemctl daemon-reload
            l_message='Clean download location' G_RUN_CMD rm -R /tmp/dietpi-update/DietPi-"$GITBRANCH"/rootfs

        fi

[Fourdee]

Spoke too soon. 1 sec

🈺 Issue again once we select "install" during 1st run in dietpi-software

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
[ INFO ] DietPi-Obtain_network_details | Entered assigned working directory: /tmp/DietPi-Obtain_network_details
[ INFO ] DietPi-Banner | Entered assigned working directory: /tmp/DietPi-Banner
 ───────────────────────────────────────
 DietPi     | 10:55 | Fri 10/08/18
 ───────────────────────────────────────
 v6.12      | Virtual Machine (x86_64)
 ───────────────────────────────────────
 eth0       | 192.168.0.31
 ───────────────────────────────────────
[ INFO ] DietPi-Software | Entered assigned working directory: /tmp/DietPi-Software
[  OK  ] DietPi-Software | Root access verified.
[  OK  ] DietPi-Software | RootFS R/W access verified.

[  OK  ] DietPi-Software | Initialized database
[  OK  ] DietPi-Software | Reading database completed
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
[ INFO ] DietPi-Set_software | Entered assigned working directory: /tmp/DietPi-Set_software
[ SUB1 ] DietPi-Set_software > passwords ()
[  OK  ] passwords  | Completed
[  OK  ] DietPi-Software | Connection test: https://deb.debian.org/debian/
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory

Either, current directory is not in dietpi-software and it no longer exists, or, dietpi-software no longer exists.

Appears dietpi-software dir is removed/not created.

root@DietPi:~# ls -lha /tmp
total 4.0K
drwxrwxrwt  7 root root  140 Aug 10 10:55 .
drwxr-xr-x 22 root root 4.0K Aug 10 10:55 ..
drwxrwxrwt  2 root root   40 Aug 10 10:55 .ICE-unix
drwxrwxrwt  2 root root   40 Aug 10 10:55 .Test-unix
drwxrwxrwt  2 root root   40 Aug 10 10:55 .X11-unix
drwxrwxrwt  2 root root   40 Aug 10 10:55 .XIM-unix
drwxrwxrwt  2 root root   40 Aug 10 10:55 .font-unix

Is created, but removed straight afterwards

[ .... ] DietPi-Software | Reading database, please wait...DELETED
EXIT HAS RUN

Cause:

G_DIETPI-NOTIFY -2 'Reading database, please wait...'
G_DIETPI-NOTIFY 0 'This triggers the G_EXIT/delete?)

G_RUN_CMD is fine 🈯️

disown the issue causing G_EXIT to trigger?

• Below code triggers issue

                set -o noclobber
                if { > /tmp/dietpi-process.pid; } &> /dev/null; then

                    set +o noclobber
                    { Print_Process_Animation & echo $! > /tmp/dietpi-process.pid; disown; } 2> /dev/null

                else

                    set +o noclobber
                    echo 1

                fi

[Fourdee]

@MichaIng

Unsure how to fix this currently, so for now i've reverted the code instead of the below (which causes the issue):

set -o noclobber
                if { > /tmp/dietpi-process.pid; } &> /dev/null; then

                    set +o noclobber
                    { Print_Process_Animation & echo $! > /tmp/dietpi-process.pid; disown; } 2> /dev/null

                else

                    set +o noclobber
                    echo 1

                fi

Any ideas why this is causing G_EXIT to be triggered ?

[MichaIng]

@Fourdee
Hmm, jep in case stay with old code. I will test a bid, if disown or the failure on set -o noclobber is the problem. Interesting that this just occurs on first run setup? Are there some of scripts running in background &, so that /tmp/dietpi-process.pid might exist from one script, while another one tries to create it (and fails due to set -o noclobber)? Then this failure could be reason for EXIT trap trigger? Doesn't make sense to me, but who knows how this works.

As disown is done on every other processing message, would be even more strange, why this only on first run setup triggers EXIT trap.

[Fourdee]

@MichaIng

Interesting that this just occurs on first run setup?

I managed to get it to occur during dietpi-backup 1 1st run, after G_AGI rsync was run, the issue occured.

[MichaIng]

@Fourdee
I remember already another case where I first thought about APT run to be the issue, perhaps internal systemctl daemon-reload other other system core services/daemon reload/restart, that this somehow shortly exits and "re-joins" the script execution...
Another testing schedule 😄.

[MichaIng]

2018-08-10 15:58:31 root@VM-Stretch:/tmp# cat test
#!/bin/bash
trap 'echo EXIT triggered by $0; rm testfile' EXIT

cd /tmp
> testfile

set -o noclobber
> testfile
set +o noclobber

sleep 1

func(){

        echo 'func start'
        sleep 1
        echo 'func exit'
        exit

}

{ func & disown; }

sleep 1

DEBIAN_FRONTEND=noninteractive apt-get -y install net-tools
DEBIAN_FRONTEND=noninteractive apt-get -y purge net-tools

sleep 1

[[ -f testfile ]] && echo 'FINE: testfile still there' || echo 'BAD: testfile already removed'
2018-08-10 15:58:33 root@VM-Stretch:/tmp# ./test
./test: line 8: testfile: cannot overwrite existing file
func start
func exit
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  net-tools
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/248 kB of archives.
After this operation, 963 kB of additional disk space will be used.
Selecting previously unselected package net-tools.
(Reading database ... 33322 files and directories currently installed.)
Preparing to unpack .../net-tools_1.60+git20161116.90da8a0-1_amd64.deb ...
Unpacking net-tools (1.60+git20161116.90da8a0-1) ...
Setting up net-tools (1.60+git20161116.90da8a0-1) ...
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
  net-tools*
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
After this operation, 963 kB disk space will be freed.
(Reading database ... 33381 files and directories currently installed.)
Removing net-tools (1.60+git20161116.90da8a0-1) ...
FINE: testfile still there
EXIT triggered by ./test

🈯️ noclobber error and APT install does not trigger EXIT trap. It is also not used by background processes, with and without disown. Just main script exit triggers the trap.
🈯️ Also killing the background process from main script does not trigger EXIT trap.

[MichaIng]

Other not necessarily related fix: https://github.com/Fourdee/DietPi/commit/ddda020ee1f39fd72fbd05ad54c897fb86b292e2
As /tmp/dietpi-drive_manager_selmnt is read by parent scripts, e.g. dietpi-software userdata mount list selection.

[MichaIng]

@Fourdee
First run setup:

• I believe first run update is the issue: https://github.com/Fourdee/DietPi/blob/testing/dietpi/dietpi-software#L14592
• It runs dietpi-software for reinstall, nested, after exit removes /tmp/DietPi-Software.
• Related errors then show up the first time, when next subscript (dietpi-set_software passwords) is executed, as for subscript/-shell bash want's to submit cwd, which does not exist any more.

---

So what we need to resolve, if we strictly want to avoid concurrent executions:

• dietpi-set_userdata should not execute dietpi-software
  
  
  
  • PR up, which also cleans and enhances code: https://github.com/Fourdee/DietPi/pull/2000
  
  
• dietpi-software > dietpi-update > dietpi-software
  
  
  
  • Not sure how to resolve this best. I already thought about moving most of first run setup outside of dietpi-software into separate dietpi-automation. But this can only include the steps that do not include software installs. Only marking titles for install/uninstall would be possible by editing /.installed and starting an automated dietpi-software run, e.g. dietpi-software 1, which then just applies 1/-1 entries.
  
  
  • But puuuh, this looks like another larger project. Simpler solution: Just mkdir /tmp/DietPi-Software and maybe needed cd /tmp/DietPi-Software after first run update? Not too nice, but dietpi-software does not create tmp files that need to be read at a later stage. Thus no problem to just recreate the dir if removed.
  
  

---

Minor: https://github.com/Fourdee/DietPi/commit/a29aaeba5f814dd1e0c7a0720f91c1e9e0864752

• Prior version did not assure we are either in work dir or home

[Fourdee]

@MichaIng

But puuuh, this looks like another larger project. Simpler solution: Just mkdir /tmp/DietPi-Software and maybe needed cd /tmp/DietPi-Software after first run update?

Yep, big job.

Ok, I think for now (as we have other tickets that need addressing, eg: Pi-hole etc). We should simply disable the rm /tmp/$G_PROGRAM_NAME from G_EXIT. Until such a time as we can resolve/prevent cross running of DietPi scripts (which is currently how it was designed to run, where functions related to the script and contained within it, can be run from another script)?

note to self:

• dietpi-software setpermissions > Move to dietpi-set_software ?

[MichaIng]

@Fourdee
Agree, so for release disable the concurrency check.

We do not need to completely remove cross running scripts. It should just go in one direction (e.g. dietpi-* => funcs/*) and loops should be avoided. The last urgent case that I found is dietpi-software (first run) => dietpi-update => dietpi-software. We could add a check or special variable/flag export or simply check for the existing install state. If patch_file recognizes it is during first run setup, it could just mark packages for reinstall via .installed file instead of directly calling dietpi-software? Hmm actually since the file should not yet exist, reinstall anyway should not be needed?
... great if the mind sorts while writing ... So lets go though it. Until v6.9 dietpi-software reinstall is just called with [[ -f /DietPi/dietpi/.installed ]], which is not true on first run setup?
But v6.10/11/12 all have calls without the check. I add the check and then retry if first run does not hang anymore... Hehe, Fourdee being faster 👍 😄

Alternatively we would need to carefully check for possible problems of concurrent runs to allow them. But actually they always lead to doubled code executions and at least performance wise it would be better to avoid loops completely.

[Fourdee]

Hmm actually since the file should not yet exist, reinstall anyway should not be needed?

Yep 👍

This can be used, same effect as file check on .installed (does not exist when install stage = 0, 1st run setup)

if (( $G_DIETPI_INSTALL_STAGE == 1 )); then
dietpi-software reinstall xxx
fi

[Fourdee]

so for release disable the concurrency check.

Hmm unsure, this really does need to be resolved now, else I feel it will never get resolved.

Apologies for being indecisive, not enough coffee in my system yet.

Ok, i'am going to run some tests over next few days (release will have to wait), and try to fix the situations where we have dupe scripts running.

From what I can see, dietpi-software > dietpi-update > dietpi-software should now be fixed.

Cron jobs look fine.

• 🈯️ PREP test v6.13

[MichaIng]

@Fourdee
Jep, setpermissions move to set_software also makes sense to be more safe here 👍.

So finally dietpi/dietpi-* are frontend scripts with user interface and alias (+boot/postboot/login, which can never be called from other scripts),
dietpi/func/dietpi-* are backend scripts without UI.

Usually frontend scripts call backend scripts, but not the other way round. Frontend scripts may call each other (update => software, config => drive_manager, software => config, launcher => all, ...), but we take care that in case of a cross call it cannot occur again in the other direction.

This should increase overall performance and stability besides fixing the bugs with working dir 😃.

Following scripts might be moved to func/ to be consistent?

• dietpi-banner
• dietpi-logclear? On the other hand we maybe want to keep this available for end user. Or hmm, frontend is dietpi-cleaner and we could add it to dietpi-explorer, if user has navigated to log dir? 🙂
• dietpi-ramlog
• dietpi-ramdisk: Those anyway might give wrong impression, that ramlog/disk can be started/stopped with this, which is actually done via fstab only on reboot.

dietpi-set_cpu_fan can be moved to parent folder. Initially I though it will be a backend, but then added UI 😆. Could be as well renamed to dietpi-fan_control. But for my impression we should anyway move this to v6.14 or later, when we find some time to add support for more devices.

[Fourdee]

Ok, as it stands this is fully working and no issues in my testing.

I'll mark this as completed.

I have not reverted to https://github.com/Fourdee/DietPi/issues/1997#issuecomment-412091475. As that will require additional re-testing which I lack time for currently.