Mbed-os: fopen() fails on PC-formatted FAT32 SD cards, even after formatting them in mbed-os

[amq]

Description

• Type: Bug
• Priority: Major

---

Bug

Target
Tested on EFM32PG12

Toolchain:
GCC_ARM

Toolchain version:
6-2017-q2-update

mbed-cli version:
1.3.0

mbed-os sha:
caeaa49d68c67ee00275cece10cd88e0ed0f6ed3

sd-driver version:
f4ab55d

Expected behavior

mount() and fopen() should work after a successful format()

Actual behavior

1. Inserted an 8GB Kingston SDHC card into the module and ran the code, everything fine
2. Inserted it into PC, appeared as non-formatted, I formatted it with FAT32, 4096 byte block size
3. Ran the code again, mount() and fopen() started to constantly fail
4. The card gets formatted just fine on PC, including writing files

Tested with 100 KHz, 1 MHz, 10 MHz and 25 MHz.

Steps to reproduce

const string SD_MOUNT_PATH = "sd";
const string SD_FILE_PATH = "/" + SD_MOUNT_PATH + "/file.tmp";

int main() {
    static SDBlockDevice sd(SDMOSI, SDMISO, SDSCK, SDCS);
    static FATFileSystem fs(SD_MOUNT_PATH.c_str());

    sd.init(); // returns 0
    fs.format(&sd); // returns 0
    fs.mount(&sd); // returns -2

    FILE *file = fopen(SD_FILE_PATH.c_str(), "w+"); // errno is 2
    if (file != NULL) {
        fclose(file);
    }

    fs.unmount();
    sd.deinit();
}

[0xc0170]

@geky Please review
[Mirrored to Jira]

[amq]

Debug output:

disk_initialize on pdrv [0]
disk_ioctl(3)
disk_ioctl(2)
disk_ioctl(1)
disk_ioctl(4)
disk_write(sector 63, count 1) on pdrv [0]
disk_write(sector 69, count 1) on pdrv [0]
disk_write(sector 70, count 1) on pdrv [0]
disk_write(sector 64, count 1) on pdrv [0]
disk_write(sector 95, count 64) on pdrv [0]
disk_write(sector 159, count 64) on pdrv [0]
disk_write(sector 223, count 64) on pdrv [0]
disk_write(sector 287, count 64) on pdrv [0]
disk_write(sector 351, count 64) on pdrv [0]
disk_write(sector 415, count 64) on pdrv [0]
disk_write(sector 479, count 64) on pdrv [0]
disk_write(sector 543, count 64) on pdrv [0]
disk_write(sector 607, count 64) on pdrv [0]
disk_write(sector 671, count 64) on pdrv [0]
disk_write(sector 735, count 64) on pdrv [0]
disk_write(sector 799, count 64) on pdrv [0]
disk_write(sector 863, count 64) on pdrv [0]
disk_write(sector 927, count 64) on pdrv [0]
disk_write(sector 991, count 64) on pdrv [0]
disk_write(sector 1055, count 64) on pdrv [0]
disk_write(sector 1119, count 64) on pdrv [0]
disk_write(sector 1183, count 64) on pdrv [0]
disk_write(sector 1247, count 64) on pdrv [0]
disk_write(sector 1311, count 64) on pdrv [0]
disk_write(sector 1375, count 64) on pdrv [0]
disk_write(sector 1439, count 64) on pdrv [0]
disk_write(sector 1503, count 64) on pdrv [0]
disk_write(sector 1567, count 64) on pdrv [0]
disk_write(sector 1631, count 64) on pdrv [0]
disk_write(sector 1695, count 64) on pdrv [0]
disk_write(sector 1759, count 64) on pdrv [0]
disk_write(sector 1823, count 64) on pdrv [0]
disk_write(sector 1887, count 64) on pdrv [0]
disk_write(sector 1951, count 9) on pdrv [0]
disk_write(sector 1960, count 64) on pdrv [0]
disk_write(sector 0, count 1) on pdrv [0]
disk_ioctl(0)
Mounting [sd] on ffs drive [0:]
disk_initialize on pdrv [0]
disk_ioctl(2)
disk_read(sector 0, count 1) on pdrv [0]
disk_read(sector 63, count 1) on pdrv [0]
open(file.tmp) on filesystem [sd], drv [(null)]
disk_initialize on pdrv [0]
disk_ioctl(2)
disk_read(sector 0, count 1) on pdrv [0]
disk_read(sector 63, count 1) on pdrv [0]
f_open('w') failed: 13

[Mirrored to Jira]

[geky]

Thanks for raising an issue. This will need some investigation.

cc @deepikabhavnani
Sorry but it will take me a bit before I can look into this guy.
[Mirrored to Jira]

[amq]

Possibly related: https://os.mbed.com/forum/mbed/topic/30213/
[Mirrored to Jira]

[amq]

#define FF_USE_TRIM 0 in ffconf.h fixes the problem for me.

Tried multiple times, with multiple SD cards.

I wish ffconf.h was configurable with mbed_lib.json. Would it make sense if I'd create a PR which covers all settings in that file?
[Mirrored to Jira]

[deepikabhavnani]

@amq Thanks for finding the root cause.
#define FF_USE_TRIM 1 is to enable erase operation on SD cards during format. Its like fast erase vs full erase. We will have to dig more to understand and add proper solution.
[Mirrored to Jira]

[0xc0170]

@amq @deepikabhavnani Can this be closed or is there anything else that needs to be fixed?
[Mirrored to Jira]

[0xc0170]

cc @ARMmbed/mbed-os-storage
[Mirrored to Jira]

[geky]

I believe this depends on https://github.com/ARMmbed/mbed-os/issues/6253

@deepikabhavnani is there another FAT issue here that needs to be tracked?

I wish ffconf.h was configurable with mbed_lib.json. Would it make sense if I'd create a PR which covers all settings in that file?

@amq, Sorry I didn't see this earlier! If you're interested in creating that pr we would love to take it in.
[Mirrored to Jira]

[deepikabhavnani]

@geky - I am not sure of another FAT issue, haven't got chance to reproduce or analyze it :-( .

@amq - Config option is good idea.
[Mirrored to Jira]

[yossi2le]

@amq @geky
Its looks to me, that if most of the users has a solution and only online compiler users need to use a workaround to overcome this specific issue then we can close the issue and stay only with #6253 enhancement request.
What do you think?
[Mirrored to Jira]

[dHutchings]

@yossi2le I am one such online compiler user (for now... looks like this is extra persuasion to switch).

This sounds like the problem I'm running into. I'm using a computer formated SD card. I hang forever on fopen. Is there a way to edit ffconf.h in the online compiler?

Side note: I've recently come back to mbed after 2-3 years, before mbed5 was a thing. At that time, the online compiler seemed to be preferred over offline gcc-arm. Is this no longer the case?

[Mirrored to Jira]

[dHutchings]

@amq @geky
Hate to be pushy here, I am one such online compiler user. I'm running into what (I think) is this exact problem.

ffconf.h doesn't seem to be editable from the online compiler. Is there anyway to edit or override the #define FF_USE_TRIM 1 on the online compiler, or am I forced to switch to offline compilation?

Thanks!
[Mirrored to Jira]

[deepikabhavnani]

Shall we set ff_use_trim as zero, till we have config options ?
[Mirrored to Jira]

[dannybenor]

@deepikabhavnani I need some explanation of the issue. We can format the card in Windows and use it on mbedos with sd-driver without any issue. I what conditions FF_USE_TRIM 1 is causing problems?
[Mirrored to Jira]

[amq]

@dannybenor try 10-20 SD cards, even the same model and manufacturer and I'm sure you'll also encounter this issue
[Mirrored to Jira]

[dannybenor]

@amq This is very interesting. I am looking to understand the real issue. Maybe changing FF_USE_TRIM 0 will solve for some SD and cause a new issue for other? Need to deep into to find the real cause, but currently this is low priority.
[Mirrored to Jira]

[amq]

@dannybenor all of the many SD cards which I tested had no issues with FF_USE_TRIM 0. Also note that 0 is the default setting, but it got changed in the course of commits
[Mirrored to Jira]

[deepikabhavnani]

I need some explanation of the issue. We can format the card in Windows and use it on mbedos with sd-driver without any issue. I what conditions FF_USE_TRIM 1 is causing problems?

Yes @dannybenor I tried to reproduce this issue with multiple SD cards and was not able to. USE_TRIM functionality was added to assist in erase operations. Without trim operation also the card will work, if pre-formatted using some utility.

@amq @dHutchings - Can you try formatting the SD card with SD Memory Card Formatter utility provided by SD association instead windows. Utility is designed to take care of correct AU size, https://www.sdcard.org/downloads/formatter_4/
[Mirrored to Jira]

[amq]

@deepikabhavnani SD Memory Card Formatter doesn't help at all. I tried both, it and the Windows formatter, the result has been the same.
[Mirrored to Jira]

[deepikabhavnani]

@amq - I have few questions, sorry if I am repeating few things from above.

1. Application to reproduce the error follows below steps in sequence?

    sd.init(); // returns 0
    fs.format(&sd); // returns 0
    fs.mount(&sd); // returns -2

1. If yes then please note format does init of block device internally (which should not be an issue) but also does deinit (which can be problem here). Can you change the sequence and see if error exists

    fs.format(&sd); // returns 0
    sd.init(); // returns 0
    fs.mount(&sd); // returns -2

1. Are you able to reproduce issue on master branch of mbed-os and sd-driver.

[Mirrored to Jira]

[deepikabhavnani]

@dHutchings - Please share your example / steps to reproduce issue with online compiler
[Mirrored to Jira]

[adbridge]

Internal Jira reference: https://jira.arm.com/browse/IOTSTOR-468

[deepikabhavnani]

https://github.com/ARMmbed/mbed-os/issues/6374#issuecomment-414741890

Seems like sequence issue in application. If issue still exists after correcting sequence as in above comment, please re-open issue. Closing for now