Marlin: [BUG] SDCARD_RATHERRECENTFIRST issue on 32 bits board

Created on 31 Mar 2019 · 30Comments · Source: MarlinFirmware/Marlin

Description

When I access to the list of the gcode files on my SD card, they are sorted by name with my 32bits board instead of "rather recent first."

Steps to Reproduce

Put an sd card with several gcode files in the sdcard holder.
Go to files list

In 8bits motherboard compiled with Arduino, the files are stored by rather recent first as expected

In 32bits motherboard (skr 1.1 and 1.3), compiled by Atom, the files are stored by name, like if SDCARD_RATHERRECENTFIRST was not concidered

I expect to have the same order in list of file on 8bits and 32bits motherboard

Source

spilz87

Most helpful comment

@ShaggyDog18 you might want to check those pins again. Looking at the standard SKR V1.3 pins file

#define LCD_PINS_ENABLE P1_18
#define MOSI_PIN         P0_18

Those pins are on different ports, so none of them are the same.

gloomyandy on 17 May 2019

😄1 👍1

All 30 comments

I think you need to post your configuration files so that someone can try and reproduce the problem you are having.

Are you using the same SD card on both systems?

gloomyandy on 31 Mar 2019

Same configuration_adv.h file
Same configuration.h file except #define MOTHERBOARD
Same LCD with card reader
Same SD card (so same files and folders)

Only differences :
Motherboard
Arduino / Atom

spilz87 on 31 Mar 2019

Post the files here, that way anyone testing can be sure they are using the same options as you. Also How recent a version of Marlin are you using (I assume you are using 2.0.x on both)?

gloomyandy on 31 Mar 2019

You have an SKR board which uses the LPC1768 and I would guess you are also using a REPRAP_DISCOUNT_FULL_GRAPHIC_SMART_CONTROLLER, it would seem that for that particular combination sorting is deliberately turned on (mainly to enable caching which helps with display glitching).

// Force SDCARD_SORT_ALPHA to be enabled for Graphical LCD on LPC1768
// because of a bug in the shared SPI implementation. (See #8122)
#if defined(TARGET_LPC1768) && ENABLED(REPRAP_DISCOUNT_FULL_GRAPHIC_SMART_CONTROLLER)
  #define SDCARD_SORT_ALPHA         // Keeps one directory level in RAM. Changing
                                    // directory levels still glitches the screen,
                                    // but the following LCD update cleans it up.

You can read all about the thoughts that went into this here: https://github.com/MarlinFirmware/Marlin/pull/8122

Personally I'm not sure they have this right. I would have either made the sort the default for all 32bit systems or made this specific only to boards with a shared SPI bus between the SD card and LCD. This is not the case for some LPC1768 boards (like Sbase and SKR V1.3, it is the case for SKR V1.1).

@p3p @thinkyhead @Bob-the-Kuhn any thoughts on if the current setup is still valid?

gloomyandy on 31 Mar 2019

Well done, it looks like you found the origine of the issue :)
But I don’t understand in which file they finally force it. Do you know it ?

spilz87 on 31 Mar 2019

It is here:
https://github.com/MarlinFirmware/Marlin/blob/bugfix-2.0.x/Marlin/src/inc/Conditionals_post.h#L1635

gloomyandy on 31 Mar 2019

The solution would be to have SDCARD_SORT_ALPHA + SDCARD_RATHERRECENTFIRST == sort, but keep the most recently printed one at the top. Unfortunately it would be near the bottom of my todo list, even only regarding improvements on/for my printer.

7eggert on 1 Apr 2019

I comment the part and it solved the issue
More than that, it solved an other issue : I couldn’t turn on the printer with a SD card in the slot, now it works well.

spilz87 on 1 Apr 2019

👍1

Good news! I will apply whatever patch seems to work best.

thinkyhead on 1 Apr 2019

As far as I know the SPI issue discussed in #8122 is still existing. For now we'll say, maybe filename scrolling can work without sorting enabled on some boards. YMMV. Keep us posted if you see this issue on your own LPC176x-based boards.

thinkyhead on 3 Apr 2019

The SPI issue is on the SKR V1.1 but for me it solved by the pin file where the sd card on lcd is desable and the onboard sd card is enable. Maybe forcing alpha sorting is not necessary

spilz87 on 3 Apr 2019

this issue is back after updating to 3272d0db01ffb4c34904d31b1c67d2fc0033ac51
to be exact, all files added on the SD card after the update are at the end of the list

spilz87 on 2 May 2019

there is no file date saved with our board, does yours set one (on the sdcard) ?

tpruvot on 2 May 2019

Files are saved as usual, they have saved date. The only thing I did was the marlin update.
Folder are upper than files. Before update it wasn’t.
After update there is an additional item menu like “update SD card” (mine is in French, so it’s maybe not exactly this translation) below “print sd card” which wasn’t before update.

spilz87 on 2 May 2019

This item has been closed so it will not get any attention. You should create a new issue (but please reference this one) and please post your configuration files and the details of any changes you have made to any other file. But before you do that are you sure that when you updated your files you did not overwrite the modifications you made to the Condidionals_post.h file? That change has not been made to standard Marlin.

gloomyandy on 2 May 2019

Same issue here: SKR v1.3 (LPC1768) + REPRAP_DISCOUNT_FULL_GRAPHIC_SMART_CONTROLLER

To resolve it I've commented the entire #if...#endif section in the Condidionals_post.h file starting:

if defined(TARGET_LPC1768) && ENABLED(REPRAP_DISCOUNT_FULL_GRAPHIC_SMART_CONTROLLER)

....

endif

Works fine, sorts by date... No issue so far.

ShaggyDog18 on 15 May 2019

Actually I solved it by format my SD card.
I absolutely don’t know why it solved it.
I just noticed that new files where stored in the good way but always after some old files.

spilz87 on 15 May 2019

it was mentioned in one of the earlier post that _"This is not the case for some LPC1768 boards (like Sbase and SKR V1.3, it is the case for SKR V1.1)."_ Then should the #if line look like dedicated to SKR v1.1 only. Essentially, limiting that to a troublesome SKR v1.1. only:

#if MB(BIQU_SKR_V1_1) && ENABLED(REPRAP_DISCOUNT_FULL_GRAPHIC_SMART_CONTROLLER)

That would resolve the issue for other LPC1768 boards than SKR v1.1
What do you think?

ShaggyDog18 on 17 May 2019

Actually I solved it by format my SD card.
I absolutely don’t know why it solved it.
I just noticed that new files where stored in the good way but always after some old files.

Strange... it should not work, because whatever sort method was chosen in config.h, it is overridden by Alphabetical by the following lines in the Condidionals_post.h file:

#if defined(TARGET_LPC1768) && ENABLED(REPRAP_DISCOUNT_FULL_GRAPHIC_SMART_CONTROLLER)
#define SDCARD_SORT_ALPHA
....

ShaggyDog18 on 17 May 2019

Specifying a particular board is almost certainly a bad idea as it would mean an update any time a new board is added and someone would need to identify every existing board that may have this problem. A better solution would be to identify the particular characteristics that cause the problem (in this case an enabled SD with a shared SPI bus). You may be able to do this by comparing the following pairs (SCK_PIN, LCD_PINS_D4 ), (MOSI_PIN, LCD_PINS_ENABLE) if both pairs are equal and SDSUPPORT is enabled then you probably need to add the sort etc.

Be warned though this area is pretty complicated in Marlin, there are several places that will set defaults if nothing else is set for pins like MOSI_PIN and SCK_PIN and there may be multiple ways to define the SPI pins being used for a display, so this test may not catch all cases.

But you may want to take a step back here and consider what it is you want. I'd question if SDCARD_RATHERRECENTFIRST is very desirable as it really just exposes the order of files in the directory on disk. For a newly formatted disk this is indeed recent first, but as time goes on and you delete and create files it becomes in effect random. You might also want to think about what fix is really needed here. If you do disable the sort options I find that scrolling through an SD card is a little jerky/slow, I much prefer to have some sort of caching. So perhaps a better solution would be to find a way to enable the caching (which is what helps reduce the screen glitches), but keep the sort order you prefer.

gloomyandy on 17 May 2019

👍1

gloomyandy, Thank you for extended answer!
Just checked for the SKR v1.3 => just one pair coincides: LCD_PINS_ENABLE and MOSI_PIN... while for the SKR v1.1 both pairs of pins are equal ...
You are right, it is matter of what is preferable. I use Toshiba Wi-Fi FlashAir SDcard and sorting by date is extremely useful; it saves time and improve usability.
Thus, I will continue testing the abode solution (disabling entire #if-#endif section in Conditionals_post.h file) and observing display behavior.
Looks good so far with NO jerky scrolling of screen delays...

ShaggyDog18 on 17 May 2019

Actually I solved it by format my SD card.
I absolutely don’t know why it solved it.
I just noticed that new files where stored in the good way but always after some old files.

Strange... it should not work, because whatever sort method was chosen in config.h, it is overridden by Alphabetical by the following lines in the Condidionals_post.h file:

#if defined(TARGET_LPC1768) && ENABLED(REPRAP_DISCOUNT_FULL_GRAPHIC_SMART_CONTROLLER) #define SDCARD_SORT_ALPHA ....

I deleted the entire #if as described before and it worked well

but I got a strange issue after a Marlin update :

All files stored before the update were on the top of the list
All news files added on sd card after the update were on the bottom of the list, but well ordered.
I solved it by format my sad card and copy back all my files, I don’t really understand

spilz87 on 17 May 2019

All news files added on sd card after the update were on the bottom of the list, but well ordered.
I solved it by format my sad card and copy back all my files, I don’t really understand

right, because files are not sorted by time... Take a closer look, they should be sorted alphabetically... or not sorted at all...

but I got a strange issue after a Marlin update :

Might it be a case that your update (if you downloaded a new Marlin release) had simply overridden a previous fix you did in the Conditionals_post.h file?

ShaggyDog18 on 17 May 2019

@spilz87 When you enable sorting there is a limit on the number of files that will be sorted. So if this is set to say 64 the first 64 files will be sorted alphabetically, but after 64 files you will just get whatever random order they are in the directory table. So that may explain what you are seeing, you just got "lucky" in that you had used up all of the sorted slots when you did the update, just a coincidence.

Just to be clear SDCARD_RATHERRECENTFIRST does not sort the files based on creation time (or anything else), you are just seeing the order that they have been added to the disk. With a newly formatted disks this just looks like they are sorted in creation order. But if you continue to use the disk and in particular if you delete files then that "creation order" will not continue.

gloomyandy on 17 May 2019

👍1

@ShaggyDog18 you might want to check those pins again. Looking at the standard SKR V1.3 pins file

#define LCD_PINS_ENABLE P1_18
#define MOSI_PIN         P0_18

Those pins are on different ports, so none of them are the same.

gloomyandy on 17 May 2019

😄1 👍1

All news files added on sd card after the update were on the bottom of the list, but well ordered.
I solved it by format my sad card and copy back all my files, I don’t really understand

right, because files are not sorted by time... Take a closer look, they should be sorted alphabetically... or not sorted at all...

but I got a strange issue after a Marlin update :

Might it be a case that your update (if you downloaded a new Marlin release) had simply overridden a previous fix you did in the Conditionals_post.h file?

No, I’m using git, it wasn’t overridden, I checked it

@gloomyandy : that can explain it, thanks

spilz87 on 17 May 2019

👍1

since SDCARD_RATHERRECENTFIRST works for SKR v1.3 I would offer the following replacement for #if statement to limit workaround just to SKR v1.3 boards:

if !MB(BIGTREE_SKR_V1_3) && defined(TARGET_LPC1768) && ENABLED(REPRAP_DISCOUNT_FULL_GRAPHIC_SMART_CONTROLLER)

What do you think?

ShaggyDog18 on 17 May 2019

See my comments above. It would be better to test for the actual condition that causes the problem. If you tried hard you probably could redefine the pins on the SKR V1.3 board so that it does suffer from this issue! So I'd go for something like...

#if defined(TARGET_LPC1768) && 
     ENABLED(REPRAP_DISCOUNT_FULL_GRAPHIC_SMART_CONTROLLER &&
     (LCD_PINS_ENABLE == MOSI_PIN) && (LCD_PINS_D4 ==  SCK_PIN,))

or something similar. Why not submit a PR and see what Thinkyhead makes of it?

gloomyandy on 17 May 2019

👍1

@gloomyandy, looks very reasonable facing the root cause of the problem, SPI pins definition!
I will not redefine SKR v1.3 pins, will just deploy the above #if statement; sure it will work for my SKR v1.3 since pairs of checked pins are both different...
I've read entire thread #8122, looks like Thinkyheads get through lots of iterations coming up with existing solution... They would not be happy getting back to it again :-)

ShaggyDog18 on 17 May 2019

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.