Dietpi: Odroid N2 - F2FS Formatted Micro SD Card Does Not Mount

[BillyCorgan1]

Creating a bug report/issue

Required Information

• DietPi version | v6.33.3
• Distro version | Buster
• Kernel version | Linux OdroidN2 4.9.236+ #1 SMP PREEMPT Tue Oct 13 17:03:36 CEST 2020 aarch64 GNU/Linux
• SBC model | Odroid N2 (aarch64)
• Power supply used | 2A Odroid Official Power Supply
• SDcard used | Samsung Pro Endurance 64GB/ 128GB Sandisk Ultra

Additional Information (if applicable)

• Software title | dietpi-drive_manager
• Was the software title installed freshly or updated/migrated? Fresh Install and Updated
• Can this issue be replicated on a fresh installation of DietPi? Yes
  
• Bug report ID | echo $G_HW_UUID Not Applicable

Steps to reproduce

1. Insert Micro SD Card into Odroid N2 Micro SD Slot.
2. Open dietpi-drive_manager.
3. Select the Micro SD Card in the menu.
4. Select the "FORMAT" item and enter OK.
5. Format Mode either PARTITION or DRIVE (GPT or MBR).
6. FILE SYSTEM TYPE: F2FS.
7. Once the format has completed, go back.
8. Select Mount and enter the desired Mount Point.

Expected behaviour

• The Micro SD Card should be mounted and the mount point displayed. The card will be usable.

Actual behaviour

• The card will not mount with F2FS as the file system on the Micro SD Card. I have tried formatting the cards as a Partition. I have tried formatting the card as a drive both as a GPT drive and an MBR drive. I cannot get the card to mount at all with it formatted as F2FS. F2FS tools is installed and was triggered on the first attempt to format the card.

Extra details

• I have successfully formatted the cards as FAT32/EXT4/exFAT and mounted it successfully. I have tried rebooting the Odroid after installation and after formatting the card. I just can't get F2FS to mount at all.

I hope you are able to understand the trouble I am having. Thanks for Dietpi and for your time.

[MichaIng]

Many thanks for your report.

What is the error message you get, or if none, how do you recognise that the mount was not successful?

[BillyCorgan1]

Hello,

Thanks for your reply. I don't get any error message. The format is always successful and does not fail. I have even formatted a card in Parted Magic with F2FS on a seperate computer and that does not mount either. The way I recognise that the mount is not successful is in drive manager (dietpi-drive_manager). I am using an emmc module that holds my root folder and boot folder. Underneath those in Drive Manager it gives the mount point and the dev address, then it states F2FS and NOT MOUNTED. I then
select the drive and hit OK. At the top the status says "Drive is not mounted and can be unplugged". I then try to mount it (highlight mount and select OK). It gives the mount location prompt and I hit OK. The status never changes to "Drive is online and ready for use". It just stays on "Drive is not mounted and can be unplugged". I have tried refreshing on the Drive Manager and nothing changes it states "Drive is not mounted and can be unplugged". I have tried exiting the Drive Manager and going back in but that does not work. I have even tried rebooting the Odroid after mounting and I still get the same message. I am sorry I do not have an error code for you to look at. I will try and take down the Odroid tomorrow and see if I can get it narrowed down. At the moment it is running fine with the Micro SD Card mount as EXT4 and it is readable and usable after reboots. Thanks again for your time.

[MichaIng]

It might be a problem is DietPi-Drive_Manager or one of the low-level tools it uses. Please check the following commands, if your F2FS mount is listed there:

df -T
mount
findmnt

The above three commands should either all or none show your mount.
The following shows if it has been added to fstab file for auto-mount on boot:

cat /etc/fstab

[BillyCorgan1]

This is the output I get after exiting from the Drive Manager in MobaXterm:

[ INFO ] DietPi-Drive_Manager | Detecting drives, please wait...
[ INFO ] DietPi-Drive_Manager |  - Detected mounted drive: /dev/mmcblk0p2 > /
[ INFO ] DietPi-Drive_Manager |  - Detected mounted drive: /dev/mmcblk0p1 > /boot
[ INFO ] DietPi-Drive_Manager |  - Detected unmounted drive: /dev/mmcblk1p1
[.     ] DietPi-Drive_Manager | mkdir -p /mnt/Testmount
[  OK  ] DietPi-Drive_Manager | mkdir -p /mnt/Testmount
[FAILED] DietPi-Drive_Manager | mount /dev/mmcblk1p1 /mnt/Testmount
[ INFO ] DietPi-Drive_Manager | Detecting drives, please wait...
[ INFO ] DietPi-Drive_Manager |  - Detected mounted drive: /dev/mmcblk0p2 > /
[ INFO ] DietPi-Drive_Manager |  - Detected mounted drive: /dev/mmcblk0p1 > /boot
[ INFO ] DietPi-Drive_Manager |  - Detected unmounted drive: /dev/mmcblk1p1

This is after I have formatted the card with F2FS using GPT and DRIVE options.

df -T: Command does not show the mount.
mount: Command does not show the mount.
findmnt: Command does not show the mount.

cat /etc/fstab Output below:

# Please use "dietpi-drive_manager" to setup mounts
#----------------------------------------------------------------
# NETWORK
#----------------------------------------------------------------


#----------------------------------------------------------------
# TMPFS
#----------------------------------------------------------------
tmpfs /tmp tmpfs size=1856M,noatime,lazytime,nodev,nosuid,mode=1777
tmpfs /var/log tmpfs size=50M,noatime,lazytime,nodev,nosuid,mode=1777

#----------------------------------------------------------------
# MISC: ecryptfs, vboxsf (VirtualBox shared folder), gluster, bind mounts
#----------------------------------------------------------------


#----------------------------------------------------------------
# SWAPFILE
#----------------------------------------------------------------


#----------------------------------------------------------------
# PHYSICAL DRIVES
#----------------------------------------------------------------
UUID=e139ce78-9841-40fe-8823-96a304a09859 / ext4 noatime,lazytime,rw 0 1
UUID=EC51-067C /boot vfat noatime,lazytime,rw 0 2
**_#UUID=3263075c-faeb-428e-9264-cd59c32c64e7 /mnt/3263075c-faeb-428e-9264-cd59c32c64e7 f2fs noatime,lazytime,rw,nofail,noauto,x-systemd.automount_**

If you look at the last line (I have made it bold and italicised). This is the card I am trying to mount. I have double checked it in Drive Manager. The beginning has a hash symbol which I take to mean that it is set to ignore that line. I have just edited the file to take that symbol out: sudo nano /etc/fstab. This makes no difference, I still cannot get the drive to mount. I have tried to access the Mount Point in dietpi-explorer but it causes an error and states the drive cannot be found.

I am wondering if there is an error with how F2FS is dealing with Mount Points. My thinking is that when I try to change the mount point after formatting from the default given to /mnt/Backup it does not stick and stays as the default. This could be caused by the error of the drive not mounting and once solved it might change to the desired one.

I hope this is helpful, thanks again for your time.

[MichaIng]

[FAILED] DietPi-Drive_Manager | mount /dev/mmcblk1p1 /mnt/Testmount

Okay so indeed the mount command failed but the drive is detected. It is then added commented (the leading hash # character) to /etc/fstab.

Let's see if we can get some more info by mounting manually:

mount /dev/mmcblk1p1 /mnt/Testmount

[BillyCorgan1]

I have run the command and got the following output:

mount: /mnt/Testmount: wrong fs type, bad option, bad superblock on /dev/mmcblk1p1, missing codepage or helper program, or other error.

I have done some research into F2FS and how to repair the filesystem. I used the following command:

fsck.f2fs -f /dev/mmcblk1p1

I got the following output:

Info: Force to fix corruption
Info: Segments per section = 1
Info: Sections per zone = 1
Info: sector size = 512
Info: total sectors = 125038592 (61054 MB)
Info: MKFS version
  "Linux version 4.9.236+ (root@odroid-stretch64) (gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1) ) #1 SMP PREEMPT Tue Oct 13 17:03:36 CEST 2020"
Info: FSCK version
  from "Linux version 4.9.236+ (root@odroid-stretch64) (gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1) ) #1 SMP PREEMPT Tue Oct 13 17:03:36 CEST 2020"
    to "Linux version 4.9.236+ (root@odroid-stretch64) (gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1) ) #1 SMP PREEMPT Tue Oct 13 17:03:36 CEST 2020"
Info: superblock features = 0 :
Info: superblock encrypt level = 0, salt = 00000000000000000000000000000000
Info: total FS sectors = 125038592 (61054 MB)
Info: CKPT version = 6949fc87
Info: Corrupted valid nat_bits in checkpoint
Info: Write valid nat_bits in checkpoint
Info: checkpoint state = 185 :  trimmed nat_bits compacted_summary unmount

[FSCK] Unreachable nat entries                        [Ok..] [0x0]
[FSCK] SIT valid block bitmap checking                [Ok..]
[FSCK] Hard link checking for regular file            [Ok..] [0x0]
[FSCK] valid_block_count matching with CP             [Ok..] [0x2]
[FSCK] valid_node_count matcing with CP (de lookup)   [Ok..] [0x1]
[FSCK] valid_node_count matcing with CP (nat lookup)  [Ok..] [0x1]
[FSCK] valid_inode_count matched with CP              [Ok..] [0x1]
[FSCK] free segment_count matched with CP             [Ok..] [0x7682]
[FSCK] next block offset is free                      [Ok..]
[FSCK] fixing SIT types
[FSCK] other corrupted bugs                           [Fail]
Info: Write valid nat_bits in checkpoint
Info: Write valid nat_bits in checkpoint

Done.

Running the command again I get:

mount /dev/mmcblk1p1 /mnt/Testmount mount: /mnt/Testmount: wrong fs type, bad option, bad superblock on /dev/mmcblk1p1, missing codepage or helper program, or other error.

I have also found another user who had the same type of problem on the Odroid Forum:

https://forum.odroid.com/viewtopic.php?t=34978

The title is F2FS: can't mount FS: "Wrong bitmap size". To summarise the user has problems getting F2FS to work properly and gives the commands that he has used to try to make it work. Disappointingly, there is no resolution in the thread or any steps to try.

In another post on the Odroid forum:

https://forum.odroid.com/viewtopic.php?t=5939

The title is Can't mount f2fs USB stick on Odroid. THIS POST IS SIX YEARS OLD. I thought the exchange was quite interesting:

Re: Can't mount f2fs USB stick on Odroid

Post by mdrjr » Sun Aug 03, 2014 12:05 am

There isn't f2fs support on our kernel. When I tested it, it was dead unstable.

Re: Can't mount f2fs USB stick on Odroid

Post by sh*t's'cool » Sun Aug 03, 2014 12:24 am

Thank you for the information!

Gonna use ext4.

How does it come that it's not stable? Is it because the kernel is quite old?

Works on Raspberry Pi with the latest kernel...

Re: Can't mount f2fs USB stick on Odroid

Post by mdrjr » Sun Aug 03, 2014 2:53 am

rPi is on 3.12 (as they official images)

I tested f2fs on 3.13 Ubuntu X86 desktop and it was still unstable.. I've got several kernel panics..

So I didn't even bothered to backport it to our kernel.

I am wondering if there is a bug in the code for Odroid N2 F2FS support. I think if this implementation is faulty then that would be causing the drive to not mount. The code you (DietPi) are using to format the drive is not working properly. You have implemented it correctly, it's just that the code from Odroid is not working. From the first URL I posted I get the feeling that if my assumptions are correct, then they are not too bothered about it not working correctly.

This is all conjecture on my part and I would be interested in your thoughts. I am sorry it is a wall of text and I am grateful for your help.

[MichaIng]

Okay, so generally it is Linux 4.9, that should be new enough to serve a stable F2FS. At least it should mount, so looks like a general issue with the official Hardkernel Odroid kernel. @meveric is this a known issue?

[meveric]

Not necessarily known, but I could recreate it, on both the N2 as well as the C4 which share the same Kernel.
I could however mount it without issue on the C2 which is also arm64 so that is not the reason why it's not working.
Also XU4 with Kernel 5.4 was working fine.
So my guess is that's a problem within the Kernel of the N2/C4.
I'll see if I can find something out.

[MichaIng]

Also XU4 with Kernel 5.4 was working fine.

Oh awesome, I didn't know that Linux 5.4 is available for Odroid XU4 now. Will upgrade our images 👍.

[meveric]

careful with Kernel 5.4 I haven't done all the testings yet, and need to update the boot.ini to incorporate device tree overlays.
I did this today for the ODROID N2, still have to do this for C4 and XU4 later.

[MichaIng]

Ah yes makes sense. We also need to update our config scripts and add I2C/SPI switches via dtoverlays.
Btw /usr/lib/linux-image-* currently contains all device trees for a large number of other models and the overlays again. Since all required ones are located at /boot already, what do you think about excluding /usr/lib from being added to the package?

[meveric]

/usr/lib/linux-image-*/overlays is added automatically.
/boot/overlays is what I do after the image was already created to make sure only the ones required are in /boot/ (easier to find).

Some already moved from bootfs being an fat partition to either having it ext4 or using only one partition that has boot in it.
That way you could actually symlink/hardlink the overlays folder instead, but I'm not feeling comfortable to move away from fat just yet.
If someone breaks something in the boot.ini or with the Kernel, you can still fix it using Windows or any other OS as long as it's fat.

[MichaIng]

If someone breaks something in the boot.ini or with the Kernel, you can still fix it using Windows or any other OS as long as it's fat.

Also one can pre-configure boot.ini and in our case dietpi.txt easily from Windows right after flashing the SD card. As long as Windows has not ext4 support, sadly for user convenience it's good to stick with FAT /boot partition.

/usr/lib/linux-image-*/overlays is added automatically.

Understood, but since it's doubled/not required, the package could be cleaned up a bid by removing them 🙂. But since file sizes are small, not much benefit aside of making perfectionists like me happy 😅.

[meveric]

I'm hunting down the f2fs issue.
I found a github: https://github.com/jaegeuk/f2fs-stable/
with the code for f2fs, basically backported to the most commonly used Kernel versions, including Kernel 4.9.
I currently try to merge the changes for f2fs and it's included files into the Kernel from HK to see if it will work or not.
But it takes forever to piece together the different files that are affected by f2fs.
If it works I may have a new Kernel tomorrow, but I can't promise this yet.

[meveric]

So just a status update:
I tried to use the code from git repo above and sync it with the code from hardkernel.. but it was just too many differences and after about 12hrs of trying to get it to work I gave up.
I also tried to find a different Kernel just to try a different version that is not 4.9 but no success either.
I haven't stopped trying yet, I'll compare with code from the XU4 which had a version 4.9 as well long time in the past as well as Kernel 4.14 and will try if I can merge that code. But it's definitely taking much longer than I hoped for.

[meveric]

success!

I was able to merge the code from an old XU4 branch (Kernel 4.9) with the code from HKs 4.9 Kernel for C4/N2 and with that I was able to fix f2fs_fs support.
I'm currently building C4 Kernel as well (only tested on N2 until now) and will upload the new Kernel tomorrow.

Although I encountered a different issue unrelated to this, which probably came with a recent update as well, as I currently have no sound output (pulseaudio), not sure if anyone else has this issue I'm still investigating.

[MichaIng]

Awesome, maybe you can commit this to Hardkernel "upstream" kernel.

Although I encountered a different issue unrelated to this, which probably came with a recent update as well, as I currently have no sound output (pulseaudio), not sure if anyone else has this issue I'm still investigating.

We use/ship ALSA by default, didn't remember a report about failing audio but will keep eyes open.

[meveric]

Just tested ALSA.. doesn't work either.. but then again, not sure what is causing this, as a couple of days ago it was working fine.
So I'll have to check what might cause this.
Maybe I broke my image by so much testing :D

[MichaIng]

Hopefully it is not related to the F2FS fix then. Many thanks for your work on this. I am experimenting with F2FS root partition on RPi, probably the same can then be done for Odroid as well, to allow a bid reduced SD card writes and probably better performance 😃.

[meveric]

ALSA was broken for me even before the fix so that's not the problem :)

On the matter of F2FS being better, performance wise it was tested to be slower than ext4.
My opinion, using ex4 with options like discard and noatime, already improves the lifetime of SD cards a lot.

[MichaIng]

On the matter of F2FS being better, performance wise it was tested to be slower than ext4.

Interesting, I read the opposite. But I guess it highly depends on the testing scenario and also F2FS seems to have gone through a larger amount of enhancements with Linux 5.0, although I would need to again search for the source I got this from. In the Linux changelogs one sees changes for all file systems with nearly every update, so hard to derive which one has a significant performance or lifetime effect 😄.

noatime for sure but this can be applied to both file systems while its not default on both (?), so when comparing one just needs to assure that it is either set or atime explicitly on both test mounts.

But yeah as said, for now I just aim to add general support for root on F2FS throughout DietPi scripts and then start to do some tests.

[meveric]

just to point out the issue I had with sound was not related to anything I did, but in fact an issue from HardKernel:
https://forum.odroid.com/viewtopic.php?f=177&t=40741

I'll make sure the update is available soon (when mdrjr fixes his site again) and upload a fixed Kernel for the sound issue ;)