Mbed-os: LittleFS failure to format Blockdevice - FAT works fine

[drahnr]

Description

• Type: Bug
• Priority: Major

---

Bug

Target
NRF52_DK / CUSTOM

Toolchain:
GCC_ARM

Toolchain version:

mbed-cli version:
1.3.0

mbed-os sha:
4.7.4 + 2 patches

Expected behavior
Same succesful test as with FATFileSystem

Actual behavior
Failure -52

Steps to reproduce

Code taken from one of the mbed fat fs examples

#define LITTLE_IS_BETTER_THAN_FAT true
const static char LOG_TAG[] = "Test";

int fstest(BlockDevice& bd) {
  int error = 0;
  Log::d(LOG_TAG, "***Start***");
    #if LITTLE_IS_BETTER_THAN_FAT
/// 4096 is the native block/sector size
    lfs_size_t read_size = 4096;
    lfs_size_t prog_size = 4096;
    lfs_size_t bock_size = 256;
    lfs_size_t lookahead = 128;
    error = LittleFileSystem::format(&bd, read_size, prog_size, bock_size, lookahead);
    #else
  error = FATFileSystem::format(&bd);
  #endif
  return_error(error);

  Log::d(LOG_TAG, "Mounting the filesystem on \"/woot\". ");
  error = fs.mount(&bd);
  return_error(error);

  Log::d(LOG_TAG, "Opening a new file, numbers.txt.");
  FILE* fd = fopen("/woot/numbers.txt", "w");
  errno_error(fd);

  for (int i = 0; i < 20; i++){
    Log::d(LOG_TAG, "Writing decimal numbers to a file (%d/20)\r", i);
    fprintf(fd, "%d", i);
  }
  Log::d(LOG_TAG, "Writing decimal numbers to a file (20/20) done.");

  Log::d(LOG_TAG, "Closing file.");
  fclose(fd);
  Log::d(LOG_TAG, " done.");

  Log::d(LOG_TAG, "Re-opening file read-only.");
  fd = fopen("/woot/numbers.txt", "r");
  errno_error(fd);

  Log::d(LOG_TAG, "Dumping file to screen.");
  char buff[16] = {0};
  while (!feof(fd)){
    int size = fread(&buff[0], 1, 15, fd);
    fwrite(&buff[0], 1, size, stdout);
  }
  Log::d(LOG_TAG, "EOF.");

  Log::d(LOG_TAG, "Closing file.");
  fclose(fd);
  Log::d(LOG_TAG, " done.");

  Log::d(LOG_TAG, "Opening root directory.");
  DIR* dir = opendir("/woot/");
  errno_error(fd);

  struct dirent* de;
  Log::d(LOG_TAG, "Printing all filenames:");
  while((de = readdir(dir)) != NULL){
    Log::d(LOG_TAG, "  %s", &(de->d_name)[0]);
  }

  Log::d(LOG_TAG, "Closeing root directory. ");
  error = closedir(dir);
  return_error(error);
  Log::d(LOG_TAG, "Filesystem Demo complete.");

  while (true) {}
}

Output:

Test  Mounting the filesystem on "/woot".
Test  Failure. -52

---

Question

Anything wrong with the parameters above

[geky]

Thanks for raising an issue. Which block device is this on?

[drahnr]

Custom, 4096 sector/blocksize

[geky]

Looking at the parameters, why is block_size < prog_size? It should be the other way around. It sounds like your block_size should be 4096.

Though those parameters are the "minimum sizes" and get rounded up based on the geometry reported by the underlying block device.

What's the read_size, program_size, and erase_size of the block device?

Is the behaviour different when you don't pass any parameters? (the defaults should determined from the underlying block device).

[drahnr]

The program/erase/read sizes are all 4096 bytes and are implemented as such in the BlockDevice API. Every write is verified by a read and the first 32 bytes are compared and assured to match or report failure and retry.

I did the following test sequence:

1. Little::format(4096,4096,256,128) -> fails to mount -52
2. Little::format() -> fails to mount -52
3. Little::format(4096,4096,4096,256) -> fails to mount -52
4. Little::format() -> fails to mount -52
5. FAT::format() -> mounts successfully and all tests pass
6. Little::format() -> mounts successfully and all tests pass

So this is interesting, not sure what this implies regarding Little::format() - from the above test it seems that there are remainders of the previous FS still on the blockdevice? Maybe someone with more in depth knowledge can shed some light here.

Is it expected to erase the whole device before calling format?

[geky]

That _is_ strange. You really shouldn't need to erase the block device before format.

However, it is possible that the parts of the previous filesystem is being picked up during mount. My best guess is is it possible your block device driver ends up with overlapping blocks being written to?

Are you passing the same configuration to the LittleFileSystem constructor? These are used during mount:
https://github.com/ARMmbed/mbed-os/blob/master/features/filesystem/littlefs/LittleFileSystem.h#L54-L58

One thing you can try is enabling all of the littlefs logging. This might show more info about what's going wrong. Try adding these to an mbed_app.json file in the root of your program:

{                                       
    "target_overrides": {               
        "*": {                          
            "littlefs.enable_info": true,
            "littlefs.enable_debug": true,
            "littlefs.enable_warn": true,
            "littlefs.enable_error": true
        }                               
    }                                   
}                                       

[drahnr]

I can run some additional tests this week, anything beyond the above flags that could help?

[geky]

If you're willing to, adding log statements into your block device to see which addresses are being erased, read, and written too, would help a lot to see what's happening.

One nice thing about storage is that you have no limit to how much you can log, timing has no consequence here : )

[drahnr]

@geky

I tried to reproduce this issue, but so far no success, I am going to investigate some more but if nothing pops up I think this can be closed.

0 I 0x200059a8 WincFlash  Total flash size of 1048576 bytes detected, skipping 524288 bytes
0 D 0x200059a8 WincFlash  Attempting to program 12288 4096
0 D 0x200059a8 WincFlash  Attempting to program 4096 4096
0 D 0x200059a8 WincFlash  Attempting to program 0 4096
lfs info: format -> 0
0 D 0x200059a8 LittleVsFAT  fs_lil [format] -> 0
lfs info: mount(0x20005158)
lfs info: mount -> 0
0 D 0x200059a8 LittleVsFAT  fs_lil [mount] -> 0
lfs info: unmount()
lfs info: unmount -> 0
lfs info: format(0x20005158, 4096, 4096, 4096, 128)
0 D 0x200059a8 WincFlash  Attempting to program 12288 4096
0 D 0x200059a8 WincFlash  Attempting to program 4096 4096
0 D 0x200059a8 WincFlash  Attempting to program 0 4096
lfs info: format -> 0
0 D 0x200059a8 LittleVsFAT  fs_lil [format] -> 0
lfs info: mount(0x20005158)
lfs info: mount -> 0
0 D 0x200059a8 LittleVsFAT  fs_lil [mount] -> 0
lfs info: unmount()
lfs info: unmount -> 0
lfs info: format(0x20005158, 4096, 4096, 4096, 256)
0 D 0x200059a8 WincFlash  Attempting to program 12288 4096
0 D 0x200059a8 WincFlash  Attempting to program 4096 4096
0 D 0x200059a8 WincFlash  Attempting to program 0 4096
lfs info: format -> 0
0 D 0x200059a8 LittleVsFAT  fs_lil [format] -> 0
lfs info: mount(0x20005158)
lfs info: mount -> 0
0 D 0x200059a8 LittleVsFAT  fs_lil [mount] -> 0
lfs info: unmount()
lfs info: unmount -> 0
0 D 0x200059a8 LittleVsFAT  fs_fat [format] -> -9
lfs info: format(0x20005158, 64, 64, 512, 512)
0 D 0x200059a8 WincFlash  Attempting to program 12288 256
0 D 0x200059a8 WincFlash  Attempting to program 4096 256
0 D 0x200059a8 WincFlash  Attempting to program 0 256
lfs info: format -> 0
0 D 0x200059a8 LittleVsFAT  fs_lil [format] -> 0
lfs info: mount(0x20005158)
lfs info: mount -> 0
0 D 0x200059a8 LittleVsFAT  fs_lil [mount] -> 0
lfs info: unmount()
lfs info: unmount -> 0
0 D 0x200059a8 Winc  Deinitializing
0 D 0x200059a8 Winc  Deinitialized

---

void try_mount(BlockDevice* flash, FileSystem& fs, const int r) {
    Log::d(LOG_TAG, "%s [format] -> %i", fs.getName(), r);
    if (r == 0) { // should be mounted afterwards
        const int q = fs.mount (flash);
        Log::d(LOG_TAG, "%s [mount] -> %i", fs.getName(), q);
        if (q == 0) {
            fs.unmount ();
        }
    }
}


#include <LittleFileSystem.h>

#include <FATFileSystem.h>

FATFileSystem fs_fat ("fs_fat");
LittleFileSystem fs_lil ("fs_lil");

BlockDevice flash; // my impl

int main() {
    try_mount(&flash, fs_lil, LittleFileSystem::format(&flash, 4096, 4096, 256, 128));
    try_mount(&flash, fs_lil, LittleFileSystem::format(&flash, 4096, 4096, 4096, 128));
    try_mount(&flash, fs_lil, LittleFileSystem::format(&flash, 4096, 4096, 4096, 256));

    try_mount(&flash, fs_fat, FATFileSystem::format(&flash));
    try_mount(&flash, fs_lil, LittleFileSystem::format(&flash));
}

[ciarmcom]

ARM Internal Ref: MBOTRIAGE-262

[yossi2le]

Hi @drahnr,
I see this issue is open for a long time now. Is this issue still relevant or we can close it?
Kind Regards

[drahnr]

I think it is save to close it, I can not reproduce this since at least 5.7

[yossi2le]

Thanks

[bsturgess]

Is it possible to use the inbuilt flash of the NRF52 as a blockdevice for tinyfs?

Ive tried using this... FlashIAPBlockDevice bd(301854,8192);

But I get fails on program function and when trying to format, i get error: EDOM - function argument out of range (-1)

Any ideas?