DietPi-Drive Manager | Loss of file/folder permissions during move of user data

[ghost]

Upon following instructions when setting the file path to "/mnt/dietpi_userdata/urbackup" or "/mnt/usb_1/urbackup".

Error code suggests UrBackup application does not have the proper permissions to write to urbackup folders.

Error as detailed in web interface.
The directory where UrBackup will save backups is inaccessible. Please fix that by modifying this folder in 'Settings' or by giving UrBackup rights to access this directory. (err_cannot_create_subdir). Permission denied (code: 13)

Verified UrBackup exist as a user

cat /etc/passwd
urbackup:x/:110:113::/usr/local/var/urbackup:/bin/false

Verified the UrBackup file paths are owned by root

root@DietPi:~# ls -l /mnt/dietpi_userdata/
drwxr-xr-x 3 root root 4096 Sep 21 01:23 urbackup

Verified that the process for the urbackup application is owned by urbackup user

root@DietPi:~# ps aux | less
root@DietPi:~# ps u 10792
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
urbackup 10792  0.0  1.0 1725996 17676 ?       Ssl  00:45   0:01 /usr/local/bin/urbackupsrv run --config /etc/default/urbackupsrv --no-consoletime

Possible Solutions

• Change permissions to 'urbackup' folder ownership from root to urbackup. - Preferred
• Execute urbackup process as root instead of urbackup. - Security risk

Verified Solution -

• [x] Change permissions to 'urbackup' folder ownership from root to urbackup.

find /mnt/dietpi_userdata/urbackup -type d -exec chown urbackup {} \;
find /mnt/usb_1/urbackup -type d -exec chown urbackup {} \;

Verification as defined by...
[x] The permission denied (code: 13) no longer shows in the web UI
[] Writing a large amount data to the urbackup server folder via urbackup client - In progress

Side Note
The instructions recommend when setting the UrBackup file path restarting dietpi-services. An alternative is clicking on the UrBackup logo. It forces the user to re-logon which in turn forces UrBackup to reload the its configuration.

[Fourdee]

@Zone22

Great report 👍

We actually set permissions for the data store directory during installation:

chown -R urbackup:urbackup "$FP_DIETPI_USERDATA_DIRECTORY"/urbackup

Most likely, we need to apply to the physical location /mnt/usb_1 aswell as the symlink /mnt/dietpi_userdata. I'll run a test to check.

[Fourdee]

@Zone22

Tested on a fresh installation, Odroid C2. Permissions for me, are set correctly during installation:

root@DietPi:~# ls -lha /mnt/dietpi_userdata/
total 28K
drwxr-xr-x  4 root     root     4.0K Sep 21 16:21 .
drwxr-xr-x 11 root     root     4.0K Sep 21 16:08 ..
drwx------  2 root     root      16K Sep 21 13:31 lost+found
drwxr-xr-x  3 urbackup urbackup 4.0K Sep 21 16:21 urbackup
root@DietPi:~# ls -lha /mnt/usb_1/
total 28K
drwxr-xr-x  4 root     root     4.0K Sep 21 16:21 .
drwxr-xr-x 11 root     root     4.0K Sep 21 16:08 ..
drwx------  2 root     root      16K Sep 21 13:31 lost+found
drwxr-xr-x  3 urbackup urbackup 4.0K Sep 21 16:21 urbackup
root@DietPi:~# ls -lha /mnt/dietpi_userdata
lrwxrwxrwx 1 root root 10 Sep 21 16:08 /mnt/dietpi_userdata -> /mnt/usb_1

Which:

• device?
• format type on usb drive?

[ghost]

dietpi-drive_manager was used.

• Odroid C2
• btrfs

[ghost]

@Fourdee
I'm experiencing the same issues with owncloud as well with file permissions.
The use of 'dietpi-drive_manager' resets permissions.

[Fourdee]

@Zone22

Ok, did the following test on fresh installation:

• Format drive to btrfs (userdata automatically moved during 1st run to /mnt/usb_1)
• Install Owncloud and UrBackup

Permissions set fine for me:

root@DietPi:~# ls -lha /mnt/dietpi_userdata/
total 20K
drwxr-xr-x  1 root     root       62 Sep 22 14:17 .
drwxr-xr-x 11 root     root     4.0K Sep 22 14:02 ..
drwx------  1 root     root        0 Sep 22 14:02 lost+found
drwxrwx---  1 www-data www-data    8 Sep 22 14:17 owncloud_data
drwxr-xr-x  1 urbackup urbackup   36 Sep 22 14:02 urbackup

@Zone22
With owncloud, did you use /mnt/dietpi_userdata/owncloud_data during the wizard, as per http://dietpi.com/phpbb/viewtopic.php?f=8&t=5#p47?

Again urbackup looks fine.

Is there anything else you may have done (eg: move user data from drives etc, using testing branch etc), that you can remember? Cant seem to replicate this issue.

[ghost]

@Fourdee
I apologize for this being so irksome.

I've also tested this repeatedly with a fresh install. I'm at a bit of a loss.

Reading your above post this stuck out to me in bold.
'''Format drive to btrfs (userdata automatically moved during 1st run to /mnt/usb_1)'''

I didn't realize there was an option for the first run experience to format btrfs/GPT and move user data automatically. I thought the initial configuration for the external medium in the initial set up was always default settings. Therefore dietpi-drive_manager was needed to utilize alternative filesystems.

For experimentation sake I've tried the below processes

1. First boot
2. Update Dietpi
3. Run through start up wizard
4. Install Owncloud and UrBackup
5. dietpi-drive_manager to GPT and btrfs
6. move userdata via Dietpi on hard drive - because dietpi-drive_manager requests you to do so if
   or
7. First boot
8. Update Dietpi
9. Run through start up wizard - setting up hard drive default settings
10. Install Owncloud and UrBackup
11. Move userdata via Dietpi folders off
12. Dietpi-drive_manager to GPT and btrfs
13. Move userdata via Dietpi on external hard drive

So permissions must be being reset through "Moving userdata via Dietpi"..??
If we still can't replicate the issue on based on what's above I'll create a video capturing my procedures.

[Fourdee]

@Zone22

I didn't realize there was an option for the first run experience to format btrfs/GPT and move user data automatically.

We have a var in dietpi.txt: dietpi_userdata_basedirectory=Autohttps://github.com/Fourdee/DietPi/blob/master/dietpi.txt#L171-L176

In DietPi-Software:
If the var is Auto during 1st run setup, and, a USB drive is attached and mounted (regardless of filesystem type), DietPi will automatically setup your user data for /mnt/usb_1. If not, it stays on SD.

So permissions must be being reset through "Moving userdata via Dietpi"..??

If the drive source is FAT, then there are no permissions to copy over. So, for example, transferring from a FAT to NTFS/EXT4 may cause the issue. Not sure about HFS+.

If we still can't replicate the issue on based on what's above I'll create a video capturing my procedures.

I'll try both your methods listed, see if I can get it to break 👍

[Fourdee]

Ok, found it:

• Permissions are not preserved during moving of user data, across multiple filesystem types.
• If the drive is FAT or HFS+. Urbackup throws the permission error. Cant set permissions on FAT or HFS+ drives.

Options:

• 🈴 disable FAT/HFS+ formatting and support. Nope.
• 🈺 find a way to mount FAT/HFS+ drives that ignores permissions and allows all access.
• 🈴 cp and preserve permissions when transfering. Not all filesystems support permissions.
• 🈺 grant access to all users on all USB drives in /etc/fstab by default. New fstab entry will be

/dev/sda1       /mnt/usb_1      auto     defaults,noatime,nofail,x-systemd.automount,user,umask=0000  0       0

#We can use sed pattern match with source of:
[[:space:]]defaults,noatime,nofail,x-systemd.automount[[:space:]]

#Target of:
defaults,noatime,nofail,x-systemd.automount,user,umask=0000

• 🈺 Add a function into dietpi-software that reapplies all known permissions.

Also need to consider:

• USB <> SD transfers of userdata

When moving user data from a FAT/HFS+ drive:

• Set permissions to nobody:nogroup (as they will all be reset to root:root)? No effect on urbackup. Permission denied.

[Fourdee]

To cover all filesystems:

• USB drives will be mounted with no permissions in /etc/fstab by default.

To cover moving of user data back to SD:

• After the data transfer is completed, dietpi-software setpermissions will be run which re-applies permissions for known software.

I've tested with Owncloud + FAT/BTRFS/EXT4 formats and moving user data across all of them and back to SD. All working fine.

Marking as closed.

@Zone22
You can test the changes with the testing branch: https://github.com/Fourdee/DietPi/blob/master/TESTING-BRANCH.md
Please reopen if required.

[Fourdee]

EXT4 mounts fail if umask=0000 is set. Further testing required.

[Fourdee]

EXT4 source, copy existing user/chmod permissions:

• 🈺 ntfs. No user permissions copied or supported by default mount options. However, defaults to 777 (all access)
• 🈺 btrfs. Yes, but requires cp -p
• 🈴 vfat.
• 🈺 hfs+. Yes, but requires cp -p

[Fourdee]

Reworked moving userdata:

• DietPi will attempt to preserve existing file permissions, if supported by target filesystem.
• At the end of the transfer, DietPi-Software will apply user file/folder permissions for known software in the database.

To Do:

• Work out a way of mounting FAT32 and with no permissions, without breaking auto mount system (eg: ext4 fails to mount)
  Not much can be done about this. If we enable user,umask=0000 in fstab entries, it will break ext4 mounting on that slot.
  So for now, warning when trying to move user data to FAT32 filesystem: