Dietpi: touch /forcefsck doesn't fix the file system

Created on 16 Feb 2017 · 26Comments · Source: MichaIng/DietPi

Typically I would use touch /forcefsck on a desktop system to make the OS fix the file system for any errors. However, I found that this does not work or even if it does it does not help with any file system corruption issues.

I would say that they're a problem with the card itself if it weren't for the fact that I can take the same card and do an off-line fsck on it.

I noted it in https://github.com/Fourdee/DietPi/issues/229

I think what may be missing is having some fsck tools in the boot partition to provide an offline self repair on bootup.

Bug Question

Source

trajano

👍1

All 26 comments

@Fourdee how do you think this would be fixed? Just curious.

trajano on 17 Feb 2017

@trajano
No idea yet sorry, not had a chance to look into it. Once i've finished a few things up i'll take a look, hopefully next day or two.

Fourdee on 17 Feb 2017

👍1

NP I thought you had and idea since you put a milestone target already

trajano on 17 Feb 2017

👍1

@trajano

NP I thought you had and idea since you put a milestone target already

Ah yes.
Milestone basically means I'll take a look during work on v145+, and, ensures I do not forget about it 😉

Fourdee on 17 Feb 2017

@trajano

Apologies for delay, which device was this on?

Fourdee on 2 Mar 2017

Raspberry Pi 3

trajano on 2 Mar 2017

👍1

But it should be common across should it not?

trajano on 2 Mar 2017

@trajano

But it should be common across should it not?

Yes, but i like to replicate same tests, on same device.

Fourdee on 2 Mar 2017

👍1

Do you have a branch that shows how you did it so I can look at the specific commits that fix it? (i.e. a pull request)

trajano on 2 Mar 2017

Do you have a branch that shows how you did it so I can look at the specific commits that fix it?

Not sure what you mean? Although, I've not made any commits yet (or previously) that would effect this ticket (AFAIK).

Fourdee on 2 Mar 2017

You haven't but I kind of want to know what approach you took or are planning to take just to learn from it.

trajano on 2 Mar 2017

@trajano

Just to clarify, touch /forcefsck has no effect on starting a filesystem repair at next boot?

You haven't but I kind of want to know what approach you took or are planning to take just to learn from it.

No idea yet, i'll do some testing/investigating and keep you in the loop.

Fourdee on 2 Mar 2017

It does not fix anything from what I can tell because the file system is still corrupted. However, if I take the SD card out and do an fsck from another machine it will repair it.

trajano on 2 Mar 2017

👍1

Notes:

Purposefully corrupt a filesystem: http://superuser.com/a/1019495

Fourdee on 2 Mar 2017

😄1

Just do something simple like do a git clone and unplug would work too.

trajano on 2 Mar 2017

😄1

@trajano

Appears the RPi default is to disable fsck repair on startup:

cat /etc/default/rcS | grep 'FSCKFIX'

Enable:

sed -i '/FSCKFIX=/c\FSCKFIX=yes' /etc/default/rcS

Test:

touch /forcefsck

Seems is touch /forcefsck deprecated:

root@DietPi:~# cat /var/log/syslog | grep fsck
Mar  3 18:49:25 DietPi systemd-fsck[100]: Please pass 'fsck.mode=force' on the kernel command line rather than creating /forcefsck on the root file system.
Mar  3 18:49:25 DietPi systemd-fsck[100]: e2fsck 1.42.12 (29-Aug-2014)
Mar  3 18:49:25 DietPi systemd-fsck[100]: Pass 1: Checking inodes, blocks, and sizes
Mar  3 18:49:25 DietPi kernel: [    0.000000] Kernel command line: 8250.nr_uarts=0 dma.dmachans=0x7f35 bcm2708_fb.fbwidth=1280 bcm2708_fb.fbheight=720 bcm2709.boardrev=0xa02082 bcm2709.serial=0x88e535b smsc95xx.macaddr=B8:27:EB:8E:53:5B bcm2708_fb.fbdepth=16 bcm2708_fb.fbswap=1 bcm2709.uart_clock=48000000 vc_mem.mem_base=0x3ea00000 vc_mem.mem_size=0x3f000000  dwc_otg.lpm_enable=0 console=ttyS0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait
Mar  3 18:49:25 DietPi systemd-fsck[100]: Pass 2: Checking directory structure
Mar  3 18:49:25 DietPi systemd-fsck[100]: Pass 3: Checking directory connectivity
Mar  3 18:49:25 DietPi systemd-fsck[100]: Pass 4: Checking reference counts
Mar  3 18:49:25 DietPi systemd-fsck[100]: Pass 5: Checking group summary information
Mar  3 18:49:25 DietPi systemd-fsck[100]: rootfs: 42968/952000 files (0.3% non-contiguous), 361389/3872384 blocks
Mar  3 18:49:25 DietPi systemd-fsck[244]: Please pass 'fsck.mode=force' on the kernel command line rather than creating /forcefsck on the root file system.
Mar  3 18:49:25 DietPi systemd-fsck[244]: fsck.fat 3.0.27 (2014-11-12)
Mar  3 18:49:25 DietPi systemd-fsck[244]: /dev/mmcblk0p1: 219 files, 10896/32183 clusters

/boot/cmdline.txt add fsck.mode=force.

Fourdee on 3 Mar 2017

So just run the sed ... and add the fsck.mode=force (which I think is already there) then everything should be fine then?

trajano on 3 Mar 2017

@trajano

From what its telling me, i believe simply adding fsck.mode=force to /boot/cmdline.txt will force a fsck and repair on each boot. Not sure if the sed is needed, would need testing.

I did try the DD method, it upgraded my 16GB SD to 53ZB lol, completely broken filesystem beyond any testing:

root@DietPi:~# dd if=/dev/urandom of=/dev/mmcblk0p2 count=1 bs=1M
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.22528 s, 4.7 MB/s
root@DietPi:~# touch /forcefsck
touch: cannot touch ‘/forcefsck’: Read-only file system

root@DietPi:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        53Z   53Z     0 100% /

Fourdee on 3 Mar 2017

But I think that's what we already have is it not? (Sorry dont' have the device on me at the moment) but SSHing to it I can see

dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.mode=force fsck.repair=yes rootwait

However, I still get the errors like this on dmesg ...

[259822.643398] EXT4-fs (mmcblk0p2): error count since last fsck: 8 [259822.643439] EXT4-fs (mmcblk0p2): initial error at time 1487728499: ext4_iget:4223: inode 1836929 [259822.643461] EXT4-fs (mmcblk0p2): last error at time 1487728859: ext4_iget:4223: inode 1836929 [346330.163686] EXT4-fs (mmcblk0p2): error count since last fsck: 8 [346330.163726] EXT4-fs (mmcblk0p2): initial error at time 1487728499: ext4_iget:4223: inode 1836929 [346330.163748] EXT4-fs (mmcblk0p2): last error at time 1487728859: ext4_iget:4223: inode 1836929 [432837.683997] EXT4-fs (mmcblk0p2): error count since last fsck: 8 [432837.684032] EXT4-fs (mmcblk0p2): initial error at time 1487728499: ext4_iget:4223: inode 1836929 [432837.684055] EXT4-fs (mmcblk0p2): last error at time 1487728859: ext4_iget:4223: inode 1836929 [519345.204282] EXT4-fs (mmcblk0p2): error count since last fsck: 8 [519345.204324] EXT4-fs (mmcblk0p2): initial error at time 1487728499: ext4_iget:4223: inode 1836929 [519345.204346] EXT4-fs (mmcblk0p2): last error at time 1487728859: ext4_iget:4223: inode 1836929

trajano on 3 Mar 2017

@trajano

Apologies for the delay.

Ok, we should probably check the syslog fsck logs for any info.

apt-get install rsyslog

Set /boot/cmdline.txt:

dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.mode=force fsck.repair=yes rootwait

Check logs

cat /var/log/syslog | grep fsck

Fourdee on 13 Mar 2017

Sorry I missed this issue here's the results.

trajano@wp:~ $ cat /var/log/syslog | grep fsck Jun 14 00:16:00 wp kernel: [ 0.000000] Kernel command line: 8250.nr_uarts=0 bcm2708_fb.fbwidth=1280 bcm2708_fb.fbheight=720 bcm2708_fb.fbdepth=16 bcm2708_fb.fbswap=1 vc_mem.mem_base=0x3ea00000 vc_mem.mem_size=0x3f000000 dwc_otg.lpm_enable=0 console=ttyS0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.mode=force fsck.repair=yes rootwait Jun 14 00:16:00 wp systemd-fsck[260]: fsck.fat 4.1 (2017-01-24) Jun 14 00:16:00 wp systemd-fsck[260]: /dev/mmcblk0p1: 249 files, 2989/8057 clusters Jun 14 00:16:00 wp kernel: [ 3.301372] EXT4-fs (mmcblk0p2): warning: mounting fs with errors, running e2fsck is recommended

trajano on 14 Jun 2017

based on that last line it appears that it is recommending but not running e2fsck

trajano on 14 Jun 2017

Maybe this needs to be incorporated https://github.com/AlexGhiti/rpi_initramfs_e2fsck

trajano on 14 Jun 2017

Closing due to no updates/outdated.

Fourdee on 30 Dec 2017

@Fourdee
Just found that the way we add drives to fstab, fsck on reboot get's ignored:

> /forcefsck
cmdline.txt > fsck.mode=force
tune2fs -c 1 /dev/mmcblk0p2

All did not help, until I found:
https://linuxconfig.org/how-to-force-fsck-to-check-filesystem-after-system-reboot-on-linux
https://wiki.ubuntuusers.de/fstab/#Aufbau

We add pass value of 0 and disable fsck on boot by this (?). Not sure if this also counts for kernel cmd line, but at least now that I changed it to 1 for root and 2 the others, something is happening. But seems to take a while, will report back later.