Hardware: WEMOS D1 mini Pro - 16M
This is not an issue and this is not the place to advertise your affiliate-links.
Hi, I've got 2 of these now and would like to understand how to update the boards.txt to make use of the 16M memory?
Not as simple as I'd hoped. I'm guessing that these are a copy of the D1 Mini but with the larger FLASH chips attached.
@TotallyInformation It'd be best to make a proper variant out of it. I can take a stab at it in a few hours.
Fab, thanks WereCatf.
@TotallyInformation Okay, copy https://github.com/WereCatf/Arduino/blob/WemosD1MiniPro/boards.txt over your existing one and that should be it. I do not have one of these boards, so I can't actually verify that it works, so it would be nice if you'd chime back here if it works as it should.
@WereCatf I have some of the Pro's and did some manual adjustments to boards.txt. Am I right in thinking we still can't access the memory from 4M to 16M?
@pieman64 I don't know, I ain't got any 16MB-boards. If you've got the pros then you could just try.
why WEMOS never answer the forum questions ?? launch a product, but don't have support !
HOW TO USE 16MB ???
@wfraga18 Ask them, not us. We don't work for Wemos. As for using it: how about you actually read the replies here, instead of raging? I provided a new boards.txt above you can try.
Thanks @WereCatf, I'm at a conference for a few days, I'll try after that. Good work.
@WereCatf, I'm try flash one of examples to Wemos D1 mini Pro with new boards.txt without any issues (spiffs not tested)
I've received my Mini Pro today. The sketch is running fine with 16MB settings, but SPIFFS and OTA becomes broken. It is not broken if I switch back to 4MB settings.
So, for now, I think the additional free Flash is only usable using direct calls to Esp.flashWrite() and Esp.flashRead(), although I didn't try it yet ...
EDIT : I think using Esp.flashWrite() and Esp.flashRead() as is won't work either, but doable if hacked with the following method :
http://www.packom.org/esp8266/16mb/flash/eeprom/2016/10/14/esp8266-16mbyte-flash_handling.html
Using the hack details here http://www.packom.org/esp8266/16mb/flash/eeprom/2016/10/14/esp8266-16mbyte-flash_handling.html, I was able to get SPIFFS working on 16MB, but there still issues : SDK still try to access WiFi configs like 4MB, so I had to move SPIFFS_START after the first 4MB, with 12MB remaining. Also, I still face issue with OTA, when e-boot is call, I get "need boot 1.4+", and hangs forever until doing a erase_flash with esptool.py.
Any update on over 4 mB support?
I read that should be included in next Arduino Code release 2.4!
It is already supported by the GitHub version?
Take a look at https://github.com/esp8266/Arduino/pull/2351#issuecomment-290830716
Of course, you need to tweak Arduino code to get those hacks merged manually.
Thanks!
I understand that your mod and packing.org one are not merged yet.
Since I have some Esp8266 100 with 8Mb loosing 3 out of 8 isn't the best solution.
I was thinking to add a second Spiffs with SpiffsStart after 4Mb.Do you think it is feasible?
Their a lot of work to be done to have 2 SPIFFS at the same time ...
But you can still use the unused part of the Flash as a raw flash storage, or you can reduce the 4MB part to 2MB and have a 6MB SPIFFS.
This may be the best solution ....Some other value set to reduce 4Mb to 2?
Yes, the SPIFFS location is defined in LD scripts, and you choose which one from Arduino Menu.
@martinayotte I can't make SPIFFS from FS.h to work with the 12Mb of flash.
In my setup, I moved the start above the 4M boundary, because ROM still access WiFi configs at the end of this first 4M.
PROVIDE ( _SPIFFS_start = 0x40600000 );
@capedra , did you modified the EspClass in Esp.cpp with the hack mentioned in https://github.com/platformio/platform-espressif8266/issues/8#issuecomment-293421839 ?
@martinayotte Yes, but these changes do not work with the functions from "FS.h" file.
I have read all of the posts on this issue. I think most of what is here is beyond the "usual" Arduino user. So, at the risk of collecting some developer ire, I would gently ask if added D1 mini Pro support is a goal and on what time-frame that might happen?
BTW I replaced the boards.txt with the one that has D1 mini Pro definitions. I still get Arduino IDE messages about the maximum size is 1MB.
Not sure if relevant, but SDK 2.1 is out, changelog says it now supports up
to 16M.
On May 5, 2017 4:33 PM, "Jon H Peterson" notifications@github.com wrote:
I have read all of the posts on this issue. I think most of what is here
is beyond the "usual" Arduino user. So, at the risk of collecting some
developer ire, I would gently ask if added D1 mini Pro support is a goal
and on what time-frame that might happen?
BTW I replaced the boards.txt with the one that has D1 mini Pro
definitions. I still get Arduino IDE messages about the maximum size is 1MB.—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/esp8266/Arduino/issues/2581#issuecomment-299556052,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AQC6Bkv32d83ScRPnOmVBRGI7xWT3v3cks5r23mbgaJpZM4KPw47
.
SDK update to 2.1 will be tracked here: https://github.com/esp8266/Arduino/issues/3215
@igrr any progress?
You can try the branch in #3278. As far as having SPIFFS on a 16 MB flash goes, it works.
@igrr so 16MB SPIFFS support is still not available in the master branch, right?
Any idea when it might be?
@pieman64
@igrr
@projectgus
@Spritetm
Ivan, I've been using SDK 2.4.0-RC1 because you said that the new fixes were more important than this 16 megabytes upgrade. I don't even know how to install the branch #3278 on Windows. Also, does this branch include the new changes of 2.4.0-RC1 ?
When will all the checkpoints become checked there on that branch? Are you going to finish that or is your main focus on ESP-32 from now on?
Maybe SDK 2.4.1 would sound better than 2.4.0-RC2?
Best Regards.
You can follow instructions for "Using git version" given in project's README.md to install git version of ESP8266 Arduino core. Then do git checkout pull/3278/head to switch to the PR branch. Make sure you uninstall any versions of the core installed using boards manager, before you attempt installing git version.
Regarding next steps, the plan is to fix pending issues with 2.4.0 release first. I can work on an update to the new non-OS SDK after that. Out of the checklist, all points except for "esptool-ck: support uploading to 8/16 MB flash chips" are fairly straightforward. I can't give any ETA unfortunately, as i have very little spare time right now.
@igrr
Hello, I can't mount the flash file system using the function "SPIFFS.begin();" declared on "FS.h" file. It's always returning "false".
Could you please fix it?
Thank you very much!
What debug output do you get if you set "Debug level: Core" and add Serial.setDebugOutput(true); after Serial.begin?
@igrr
SPIFFSImpl: allocating 512+180+1400=2092 bytes
SPIFFSImpl: mounting fs @100000, size=efb000, block=2000, page=100
SPIFFSImpl: mount rc=-1
_spif_erase addr=400000 size=4096 i=0
_spif_erase addr=401000 size=4096 i=0
_spif_write(118) addr=4000fe size=2 ab=400100 ae=400100
SPIFFS_format: rc=-1, err=-10027
COULD NOT MOUNT SUCCESSFULLY !!
@igrr Ivan, what could possibly be wrong?
Looks like flash size is still set to 4 MB in the binary image header.
Can you try replacing your esptool-ck with a version from this branch?
https://github.com/igrr/esptool-ck/tree/feature/8m_16m
@igrr Hello again, Ivan. Thank you a lot for helping me solve that. How can I generate the esptool.exe from this branch? I tried importing the files to a new project on Visual Studio but it didn't work at all. Please help newbies like me. :)
Best Regards!
Normally it is build in MinGW, you can find the steps in appveyor.yml file in esptool-ck repository root.
There is also an automatically built version available here: https://ci.appveyor.com/project/igrr/esptool-ck/build/0.0.113/artifacts
@igrr All I get now is the following output from the SPIFFS sketch:
SPIFFSImpl: allocating 512+180+1400=2092 bytes
SPIFFSImpl: mounting fs @100000, size=efb000, block=2000, page=100
SPIFFSImpl: mount rc=-1
I guess "SPIFFSImpl: mount rc" should return 0 (SPIFFS_OK) instead of -1?
WoW! After about six minutes, I got the following output:
SPIFFSImpl: mounting fs @100000, size=efb000, block=2000, page=100
SPIFFSImpl: mount rc=0
MOUNTED SUCCESSFULLY !
Start formatting (millis): 343277
SPIFFSImpl: mounting fs @100000, size=efb000, block=2000, page=100
SPIFFSImpl: mount rc=0
So, the whole output from my Sketch called "formatFS" is the following:
nr⸮⸮⸮nb⸮lr⸮⸮nb⸮l⸮⸮⸮⸮⸮⸮⸮⸮prl⸮⸮n⸮59
59
PLEASE WAIT 4 SECONDS !4060
4060
SPIFFSImpl: allocating 512+180+1400=2092 bytes
SPIFFSImpl: mounting fs @100000, size=efb000, block=2000, page=100
SPIFFSImpl: mount rc=-1
SPIFFSImpl: mounting fs @100000, size=efb000, block=2000, page=100
SPIFFSImpl: mount rc=0
MOUNTED SUCCESSFULLY !
Start formatting (millis): 343277
SPIFFSImpl: mounting fs @100000, size=efb000, block=2000, page=100
SPIFFSImpl: mount rc=0
SPIFFSImpl: mounting fs @100000, size=efb000, block=2000, page=100
SPIFFSImpl: mount rc=0
Formatted successfully !
Done formatting (millis): 682198
682198
682198
So, the Wemos D1 mini Pro took about 11 minutes and a half (682 seconds) to mount and format the flash file system.
In case anyone wants the code for the sketch, here it goes:
#include "FS.h"
void setup() {
Serial.begin(115200);
Serial.setDebugOutput(true);
Serial.println(millis());
Serial.println(millis());
Serial.println("PLEASE WAIT 3 SECONDS !");
delay(3000);
Serial.println(millis());
Serial.println(millis());
if (mountFlash()) {
Serial.println("MOUNTED SUCCESSFULLY !");
Serial.print("Start formatting (millis): ");
Serial.println(millis());
if (formatFlash()) {
Serial.println("Formatted successfully !");
Serial.print("Done formatting (millis): ");
Serial.println(millis());
}
else {
Serial.println("Error while Formatting !");
}
} else {
Serial.println("COULD NOT MOUNT SUCCESSFULLY !!");
}
Serial.println(millis());
Serial.println(millis());
}
void loop() {
}
bool mountFlash() {
return SPIFFS.begin();
}
bool formatFlash() {
return SPIFFS.format();
}
@capedra so did you just replace the esptool.exe with the one provided by @igrr and then run your "12 minute" sketch to get 16MB SPIFFS (or more precisely 1MB + 15MB SPIFFS)?
Are you using the master branch, 2.4.0-RC1 or the old 2.3.0?
@pieman64
Exactly, I replaced the esptool.exe that the Arduino IDE was using with that one included on the ZIP file of Ivan.
It's important to note that it took 12 minutes because it was my first run after moving from 4 MB.
So, it seems to me that the mount part became also the format part after the -1 response.
About the version that I'm using, it's still 2.4.0-RC1. I did want to keep that version because of the corrections but I had to manually implement the changes of the branch 3278 on RC1.
Good luck for everyone. I know it may sound off topic but now I can't understand why ESP-32 has only 4 MB of flash, @igrr. I heard some rumors that it will support 64 MB of flash soon? Great job!
know it may sound off topic but now I can't understand why ESP-32 has only 4 MB of flash
Just like the ESP8266, ESP32 does not have flash by itself. Board/module designers can use any flash size between 1MB and 64MB. Up to 16MB of flash can be memory-mapped. ESP-WROOM-32 modules can be ordered with flash sizes other than 4MB, subject to MOQ.
@igrr
I look for the day that ESP will run Linux and has enough storage and RAM memory to smoothly work with LAMP.
Off topic now by a long way but if you want an ESP with enough grunt to run Linux, you really should be looking at a Raspberry Pi Zero or one of the similar devices. The ESP devices are microprocessors not general purpose computers.
@capedra thanks for the details. I'm using the master branch as I wanted all the latest fixes. Could you please provide details of how you "patched" your 2.4.0-RC1 with the 3278 branch on your Windows system?
@igrr you wrote _"Then do git checkout pull/3278/head to switch to the PR branch. Make sure you uninstall any versions of the core installed using boards manager, before you attempt installing git version."_ Could you please expand on the uninstall process?
@TotallyInformation
Besides the fact that microprocessors don't work well with sensors, there's also the problem that customers can buy only one Raspberry Pi Zero from each store, so, if I need ten Raspberry Pi Zero units, I will have to pay about USD 25-35 per unit on China (not UK because of the problem that I mentioned up above). So, I think that we still need a board that we can call "the killer" in terms of dimensions and also logistics.
@pieman64
Assuming that you or someone else already installed RC1, you just need to implement Ivan's 43 latest files' changes of the branch 3278 directly on the RC1 files. If you're using Windows, I suggest you to use a software called Everything, from Voidtools. Besides the fact that it's an adventure to change each file hand by hand, it's also an interesting way to learn how his outstanding LOC (Lines of code) work very nicely.
About the uninstall that he was talking about, I uninstalled 2.3.0 using the Arduino IDE board manager. It helps you to uninstall the SDK version that you're currently using and then you can choose another version on that same combobox.
There are other similar units that are available now & the limitation on PiZero numbers should hopefully be temporary.
The point I was making is that there is a difference between an SBC and a microprocessor. As you say they are good for different things. Linux is an OS for computers not for microprocessors.
The ESP units are microprocessors. Microprocessors are good at realtime processing and need an OS that supports that, Linux is not designed for realtime systems.
Pi Zero in quantities greater than one has been promised ever since it
first appeared on the cover of MagPi magazine - it seems highly unlikely
that it will ever change as there is little profit margin on the boards
themselves but quite a bit on the shipping.
Back to the original topic...
On 17 Jul 2017 11:01, "Julian Knight" notifications@github.com wrote:
There are other similar units that are available now & the limitation on
PiZero numbers should hopefully be temporary.The point I was making is that there is a difference between an SBC and a
microprocessor. As you say they are good for different things. Linux is an
OS for computers not for microprocessors.The ESP units are microprocessors. Microprocessors are good at realtime
processing and need an OS that supports that, Linux is not designed for
realtime systems.—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/esp8266/Arduino/issues/2581#issuecomment-315701191,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALP8xabcLNKDcsCU-x1qvv_rlqDhAfYFks5sOyL3gaJpZM4KPw47
.
@igrr
Ivan, does the fact that I changed Flash speed to 80 MHz instead of 40 MHz and also Flash mode to QIO cause flash data corruption? Because I was repeatedly overwriting two files at the flash these days when it suddenly stopped rewriting anymore. Also, whenever I deleted the files, nothing happened. The function "SPIFFS.remove" had no effect toward the files. According to WinSCP, it couldn't create/open file to rewrite to disk.
Unfortunately, I mistakenly formatted the SPIFFS and everything got back to normal. Sorry for that.
Right now, should I test it with Flash speed of 40 MHz and/or DIO Flash mode? I think I'm going to set it to DIO and 80 MHz of Flash speed. What do you think? What was the most likely cause of corruption?
Best Regards!
@igrr Could you please help?
Sorry, i don't have any boards which would have >4MB of QIO flash to test this. If you are seeing data corruption at 80MHz, try dropping the frequency to 40MHz and see if that resolves the problem.
@igrr Ivan, this is the third time that I face problems with the flash of Wemos D1 mini Pro during my durability tests.
I will post a spreadsheet on here soon with the results of my tests.
I'm currently testing.
@igrr
I'm sorry but 16 MB (actually 14900866 bytes, according to FSInfo) hasn't been working well. I've been getting corrupted files on flash, the fragmentation is increasing all the time ( Richard also proved that on http://www.esp8266.com/viewtopic.php?f=32&t=12052 ). At the beginning I was thinking that it was the library that I was using (https://github.com/nailbuster/esp8266FTPServer), but now, I'm sure that the problem is really on these flash modules. Reading time decreases a lot after uploading new files, Writing time also decreases a lot after writing new files, as you can see at the spreadsheet on the end of this post.
This is the final test that I did in case anyone wants to know:
I formatted SPIFFS after not being able to delete/write new files on it;
after that, I uploaded a 10 MB MP3 file to the flash module. Works very well when I listened to it on the Web Server that I put on Wemos. After that, I uploaded a new song, and the system considered the transfer successful. So, when I listened to it on the Web Server, the music was corrupted, because there were so many glitches on it that the ESP even stopped sending the file. Finally, to prove that it's really a flash memory corruption on Wemos D1 Mini Pro, I downloaded this same file and proved that I can't trust SPI NOR Flash memories anymore because the same problem happened.
This way, I can't commercialize my products with these flash modules because my clients aren't going to complain that their devices stopped upgrading someday! These flash modules aren't reliable at all. Also, I can't add an SD card to my project because I don't have 4 digital pins available and I won't change my one year and a half project.
Oh, and FSInfo details below just in case anyone get interested:
Total Bytes:14900866
Used Bytes:14174974
Block Size:8192
Page Size:256
Max Open Files:5
Max Path Length:32
I have been testing night and day (even on weekends) the SPIFFS. I'm sorry but it's the truth about it. I'm going to test on 4MB flash modules now but it seems to me that Richard already tested that. I even made a spreadsheet detailing everything that happens on Wemos. You can take a look below:
https://docs.google.com/spreadsheets/d/1PzlkPWUjimDEFwIp7aisl51l3x82thJiUp7Gj_lKtLo/edit?usp=sharing
Apparently, the same thing has been going on with 4 MB modules:
https://docs.google.com/spreadsheets/d/1EZMl0QQpJEAXiOtZD6HPva6eVbaHfxS47wsQSCsu3EU/edit?usp=sharing
Can you load a flash up to 14.1/14.9 and have reliable results?On Jul 30, 2017 3:01 AM, capedra notifications@github.com wrote:@igrr
I'm sorry but 16 MB (actually 14900866 bytes, according to FSInfo) hasn't been working well. I've been getting corrupted files on flash, the fragmentation is increasing all the time ( Richard also proved that on http://www.esp8266.com/viewtopic.php?f=32&t=12052 ). At the beginning I was thinking that it was the library that I was using (https://github.com/nailbuster/esp8266FTPServer), but now, I'm sure that the problem is really on these flash modules. Reading time decreases a lot after uploading new files, Writing time also decreases a lot after writing new files, as you can see at the spreadsheet on the end of this post.
This is the final test that I did in case anyone wants to know:
I formatted SPIFFS after not being able to delete/write new files on it;
after that, I uploaded a 10 MB MP3 file to the flash module. Works very well when I listened to it on the Web Server that I put on Wemos. After that, I uploaded a new song, and the system considered the transfer successful. So, when I listened to it on the Web Server, the music was corrupted, because there were so many glitches on it that the ESP even stopped sending the file. Finally, to prove that it's really a flash memory corruption on Wemos D1 Mini Pro, I downloaded this same file and proved that I can't trust SPI NOR Flash memories anymore because the same problem happened.
This way, I can't commercialize my products with these flash modules because my clients aren't going to complain that their devices stopped upgrading someday! These flash modules aren't reliable at all. Also, I can't add an SD card to my project because I don't have 4 digital pins available and I won't change my one year and a half project.
Oh, and FSInfo details below just in case anyone get interested:
Total Bytes:14900866
Used Bytes:14174974
Block Size:8192
Page Size:256
Max Open Files:5
Max Path Length:32
I have been testing night and day (even on weekends) the SPIFFS. I'm sorry but it's the truth about it. I'm going to test on 4MB flash modules now but it seems to me that Richard already tested that. I even made a spreadsheet detailing everything that happens on Wemos. You can take a look below:
https://docs.google.com/spreadsheets/d/1PzlkPWUjimDEFwIp7aisl51l3x82thJiUp7Gj_lKtLo/edit?usp=sharing
—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or mute the thread.
Then, it is probably a SPIFFS issue ...
Did you try to write your music files as plain Raw data and keep track of the begin sectors and lengths where files are located ?
If no corruption there, it will prove that it is a software issue.
Spiffs is not without bugs. I believe the max theoretical file size is
16MB, but I haven't seen anyone even approach such a large file.
I think the suggestion of trying the raw flash api as a test has merit.
However, don't consider it as a final solution, because you wouldn't be
doing wear leveling.
If Spiffs does have an issue, there are debug messages that can be turned
on, and an issue can be opened on pellepl's repo.
On Jul 30, 2017 8:39 AM, "martinayotte" notifications@github.com wrote:
Then, it is probably a SPIFFS issue ...
Did you try to write your music files as plain Raw data and keep track of
the begin sectors and lengths where files are located ?
If no corruption there, it will prove that it is a software issue.—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/esp8266/Arduino/issues/2581#issuecomment-318899120,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AQC6Bg_jfmi9SnxaXyOxayqZkwDSmICuks5sTHlygaJpZM4KPw47
.
@jonhp Yes, I guess so. The 10 MB of the MP3 file worked fine just after I formatted the SPIFFS.
@martinayotte I didn't try to do that but I really think that it's a software issue because I tested it on a new Wemos D1 Mini Pro from their official store. I don't think that I already achieved the well-known 10000 limit of flash. Maybe you could try to do that because SPIFFS isn't reliable anyway.
@devyte I've seen lots of bugs on SPIFFS lately but then again, we have a very primitive file system for us to consider on our professional projects, and we can see that it's not evolving over time, it's stuck, so, I'm moving on and I'll be using SD cards instead of SPIFFS from now on.
Thank you everyone for your support.
@capedra you provided some details a few weeks ago about the 3278 branch. As it happens I did a fresh install of 3278 to a portable install of Arduino over the last few days. It was to test the new and improved deepSleep(). But to get into 16MB flashing I believe I need to "add" igrr's version of esptool-ck.
Could you please provide a few pointers as to how I add esptool-ck and hopefully I can then add to the discussion on trashing 16MB MCU's :-)
Ok I found out that esptool-ck can exist as a standalone exe and I have checked out the flashing to various ESP's. Also noticed that with Generic ESP8266 selected in the IDE that 8M (7M SPIFFS) and 16M (15M SPIFFS) are available. Do I need to switch the original exe with the "improved" exe before attempting to flash at 8M or 16M?
If I use the new exe for the IDE I get:
error: Invalid argument or value after -cf (argument #11)
Which indicates a filename problem. I would have thought the IDE would handle all that.
Anyone care to explain how we finally get 16M running?
Hello, I am trying to solve this problem with using more memory of pro version, does anyone have this [https://github.com/WereCatf/Arduino/blob/WemosD1MiniPro/boards.txt] file available ? or does it even help to solve this problem ? thank you
I tried it and discarded it. (sorry). It did not help.
@WereCatf Hi,just want to know if you still have a copy of the wimos boards text file for the support off 16mb flash? thanks
After I found it to be unhelpful I deleted it. Sorry.
Bill
dunk8888 , This may help you. It appears to let the top 15M be used as SPIFFS. Better than nothing I suppose.
It does not sadly since there is no option to choose 15M SPIFFS or anything having to do with 16M flash even after updating to 2.4.0-rc2.
My selected board was "WeMos D1 R2 & mini".
If i recall correctly, i have only added these new options to the "Generic
ESP8266" board.
On Thu, Nov 16, 2017 at 6:19 PM, sztanpet notifications@github.com wrote:
It does not sadly since there is no option to choose 15M SPIFFS or
anything having to do with 16M flash even after updating to 2.4.0-rc2.
My selected board was "WeMos D1 R2 & mini".—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/esp8266/Arduino/issues/2581#issuecomment-344878677,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEJcek1Aj1DM5EVqCeN7vTEQa1KOdMn9ks5s3AwkgaJpZM4KPw47
.
Could the D1 mini pro be considered like the D1 mini/R2 with up to 16M (1M or 15M spiffs) instead of 4M as the only difference ?
In that case, I could commit in the boards.txt generator #3722 and propose here how to test it locally.
I think so since it works perfectly otherwise with that board selected. Not an authoritative source though. Will gladly test it for you.
(edit) try boards.txt in #3722's OP, and please report back.
There is no ldscript for anything else than 16M/15Mspiffs so I included the 4M config to allow faster flashing during dev.
Okay, so there is no option in the menu for 15M spiffs or anything, but the blink example uploaded fine and works.
Oh, there is an entirely separate board for it, sorry, now I get the following errors:
Arduino: 1.8.4 (Windows 10), Board: "WeMos D1 mini Pro, 80 MHz, 16M (15M SPIFFS), v2, Disabled, None, 115200"
C:\Users\sztanpet\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-rc2\cores\esp8266\spiffs\spiffs_hydrogen.c: In function 'SPIFFS_buffer_bytes_for_filedescs':
C:\Users\sztanpet\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-rc2\cores\esp8266\spiffs\spiffs_hydrogen.c:16:49: warning: unused parameter 'fs' [-Wunused-parameter]
u32_t SPIFFS_buffer_bytes_for_filedescs(spiffs *fs, u32_t num_descs) {
^
C:\Users\sztanpet\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-rc2\cores\esp8266\Esp.cpp: In member function 'bool EspClass::eraseConfig()':
C:\Users\sztanpet\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-rc2\cores\esp8266\Esp.cpp:399:10: warning: unused variable 'ret' [-Wunused-variable]
bool ret = true;
^
c:/users/sztanpet/appdata/local/arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld.exe: cannot find -llwip2
collect2.exe: error: ld returned 1 exit status
exit status 1
Error compiling for board WeMos D1 mini Pro.
This report would have more information with
"Show verbose output during compilation"
option enabled in File -> Preferences.
after switching the lwip variant to espressif xcc , it works fine seemingly
oops, managed to copy the boards.txt to the wrong place, updated my previous comment now
Does the mkspiffs tool work for 15M from the Arduino IDE menu?
On Nov 16, 2017 8:36 AM, "sztanpet" notifications@github.com wrote:
oops, managed to copy the boards.txt to the wrong place, updated my
previous comment now—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/esp8266/Arduino/issues/2581#issuecomment-344897438,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AQC6Bn_eI8Oon-aESzv5vcsZJ8lFHMOiks5s3B47gaJpZM4KPw47
.
no sadly
[SPIFFS] data : Z:\thermo-esp\data
[SPIFFS] size : 23532
[SPIFFS] page : 256
[SPIFFS] block : 8192
/captiveportal.html
[SPIFFS] upload : C:\Users\sztanpet\AppData\Local\Temp\arduino_build_605311/thermo-esp.spiffs.bin
[SPIFFS] address: 0x100000
[SPIFFS] reset : nodemcu
[SPIFFS] port : COM5
[SPIFFS] speed : 115200
Uploading 24096768 bytes from C:\Users\sztanpet\AppData\Local\Temp\arduino_build_605311/thermo-esp.spiffs.bin to flash at 0x00100000
warning: Invalid size/address, too large: 16fb000 100000
warning: espcomm_send_command(FLASH_DOWNLOAD_BEGIN) failed
error: espcomm_upload_mem failed
SPIFFS Upload failed!
Maybe not or perhaps apparently not. Have you tried uploading using FTP? The FTP library works really well and uses a completely different method. Its wireless and faster too.
I am just now trying the same D1 Pro board. I updated the boards.txt from the board generator since the link in the last few posts had expired. I get the D1 Pro and Size options in the IDE but when I run the CheckConfiguration example it shows the IDE is reporting a flash size of 0 and the board reporting the correct size of 16m. I don't know if that is cosmetic or not. Has there been any movement yet? Does the developer still need a D1 Pro because I have three.
14 months on and presumably you still have to jump through hoops to use the extra 12MB provided by WeMos.
@jamesarm97 Link is now in #3722's OP.
@devyte Since you have received your d1 mini pro can you please check #3940 with it ?
@d-a-v I haven't gotten to testing my new boards yet. I'll report here once I do.
I was able to do some testing. Everything loads well but if you run the sample code to show the flash size / reported size, it always reports the IDE as thinking there is 0 bytes of flash. I don't know if it is related or not but I was trying to use a program that accessed EEPROM using the EEPROM.read / write and it was failing. Does anyone know what the available EEPROM for storage (arduino EEPROM functions) is? The code was using 1024 bytes so I don't know if that was the problem. I had to move to a different board since it needed to access the 1024 bytes to store configuration.
@jamesarm97 EEPROM on esp8266/arduino is an emulation that needs to be initialized and committed.
https://arduino-esp8266.readthedocs.io/en/latest/libraries.html#eeprom
I have read this entire thread and some others. My question, just an end-user, is will this change add support for uploading programs larger than 1MB into the WeMos D1 Mini Pro (2.4.0 does not appear to)?
@johnp no the extra 15MB is for storing data not program code.
Right, you are still limited to the 1M code space which could be a downer, but for what I am seeing the 15M is good for web applications where you can store the js, css, html and data files locally and not depend on the internet if you are doing an AP application.
@jamesarm97 thanks for the details. Are there any guides that you know of on how to reduce the program footprint (other than the static web files that you mentioned?
@johnp do you have embedded html strings or similar? If so, move them out too spiffs.
Not the actual code size, but the skeleton I am using all web / html code is on SPIFFS and it has a nice gulp script that compresses and minifies everything into one index.html.gz file that you can upload through the interface.
@jamesarm97
Could you share the link to that gulp script? I am dealing with issues of speed not so much size. Compressing the files manually is a pain.
This is the script. There is a dependency install file on the same project. You can see the directory structure in the gulpfile.js www www/images www/js www/css which all get minimized and optionally packed into one .gz file that can be served. Even if you don't package them, it is nice to be able to compress and have in a dist folder that can easily be updated with one command.
https://github.com/luc-github/ESP3D-WEBUI/blob/master/gulpfile.js
Boy, I am confused. You run gulpfile.js in your index.html on the esp8266 or on the PC and upload the compressed files to the esp8266? I am not sure how to use it without the rest of the project of which it is a part.
This script uses gulp which can be installed on a PC / Mac via NodeJS. The resulting file(s) is what would be used on the ESP in the data directory. This is a project I am referencing but didn't write. If you don't package the files into one index.html.gz file then you can still use the minified versions and put the resulting files in the data directory that gets loaded with SPIFFS tools and server the individual files like normal.
An update from my (slowly) ongoing testing of the Pro 16MB boards:
Hi all. I know this is is slightly off-topic but @devyte wrote above "SPIFFS mount fails and triggers a format (as expected)", but I find that strange. @weswitt also ranted against a similar behaviour for mount() - https://github.com/esp8266/Arduino/issues/4111#issuecomment-355855993 - and suggests that a failure should just return an error to let the caller deal with it, and I concur.
On ESP32, it seems the decision was made that begin() does NOT format on mount failure unless it is called with "true" as first argument (formatOnFail) - see https://github.com/espressif/arduino-esp32/issues/411#issuecomment-329017296 .
I searched for one hour but could not find a reason for that different (and dangerous IMHO) behaviour... Should I open a separate issue to have this addressed (and hopefully unified), or has it been discussed somewhere already ?
@vdeconinck that is off-topic, but to explain briefly: the decision made here predates that of the esp32. Our decision was made to cover the case where you update, and the update could break the mounting, hence you _have_ to format.
They chose to take a different default, based on different logic. We could discuss whether it makes sense to change the default in this repo to match, but I think that would be a breaking change, as it silently changes the behavior of who knows how many sketches out there.
In my case, I ordered three WeMos D1 Mini Pros, across two separate orders. The first order was for two 32MB boards. I thought I was ordering a third 32MB board but I made a mistake and wound up with an 8MB board.
I have had no trouble with the 8MB board. I can upload a "data" directory and/or let SPIFFS.begin() create the partition automatically.
Conversely, I had a lot of trouble with the 32MB boards. Choosing "Tools > ESP8266 Sketch Data Upload" seemed to work but the sketch would not progress beyond SPIFFS.begin(). With debugging enabled, the code coming back was -10025 (SPIFFS_ERR_NOT_A_FS). What was happening was that SPIFFS.begin() was not seeing the partition created by "Sketch Data Upload" and was attempting to auto-format a fresh partition but was getting it wrong. Part of "getting it wrong" appeared to involve dribbling over the top of the sketch.
There is a fair bit of advice on using esptool to work around this kind of problem but none of the examples I found made any difference. Always a -10025 error as above.
The solution I stumbled across and which may be useful for others was to tell the IDE that the board was a "WeMos D1 R2 & Mini" rather than a "WeMos D1 mini Pro". When told that the 32MB board is a Mini rather than a Mini Pro, both uploading a "data" directory or allowing SPIFFS.begin() to auto-format the partition works.
While I have not checked whether this is actually the case, it seems likely that the board will be configured as if it were 4MB so, yes, a lot of flash is sitting there doing nothing. But, right now, I don't have the need for a huge amount of flash so this is an effective workaround.
I had precisely the same experience. I don't know if a solution to this problem is in work or not. In any case, extra SPIFFS will not do much for me. What I need is variable storage space. I am reading in large JSON strings (46k per string). I can get the code to run on ESP32 but not ESP8266.
What I wrote above has one factual error. I said that the third mini pro board had 8MB but it actually has 16MB.
I have also done some more experimenting:
Just to confirm that I have exactly the same issue with a Wemos D1 mini Pro 16MB. After uploading sketch data, calling Spiffs.begin() will reformat, after raising error -10025.
But, I've had a few occasions (1 in 20), where it actually did work after uploading...
In my humble opinion, I rather stay with TTGO T1 (cheaper than Wemos Pro: only US$ 5.00) or LOLIN D32 PRO V2.0.0 instead of getting my files corrupted in a very slow flash SPI memory.
You guys could tell me that it will be more expensive this way because you also need to buy a SD card, but it's the price that you pay for a decent reliability.
I can confirm this issue and that it can be mitigated by calling a "Wemos Mini Pro" a "WeMos D1 R2 & Mini" (with resulting 1MB SPIFFS limitation). Please advise.
I'll just add a bit to this. I've got a "pro" working with 14 MB SPIFFS. The only problem I have with it is that uploading is inconsistent - with no other changes, SPIFFS will be OK after some uploads, but not others. It seems to be better when the upload speed is slower (I used 115200 on the last successful upload, and waited forever), but I haven't done it enough to be sure. But once I get a "good" SPIFFS upload, things seem to be fine. I'm doing 99% reading, very little writing.
I suppose the SPIFFS image could be being built differently each time, but that's doubtful because I'm making no changes to the "data" folder between attempts.
Is that with latest git or with 2.5.0?
Sorry for not being more complete.
ESP Core version 2_5_0, SDK version 3.0.0-dev(c0f7b44), Arduino IDE 1.8.8. Flash Chip ID 0x1840c8 (Winbond 25Q128, I think), on a CCCC (Cheap Chinese Crap Clone) of a v1.0.0 WEMOS D1 mini pro. If you would like any more detail, just ask and tell me how to get it.
It seems to act the same with both 0.3.0 and the very recent 0.4.0 of the uploader (arduino-esp8266fs-plugin). Whatever the root cause, if I upload enough times, eventually it works, which makes me suspicious of the upload process. Even when there's a good upload, SPIFFS.begin() takes a while on the first reboot after uploading (15-30 seconds?, less than if it's a bad upload). The equivalent of a *nix fsck, perhaps? On further reboots, it passes SPIFFS.begin() quickly, so there may be a minor issue with building the SPIFFS image.
I'd think there would be good error checking on the upload, but perhaps not.
YMMV. I thought I was wrong once, but I was mistaken. :)
Try installing latest git and retry. There was a PR migrating from esptool-ck to latest esptool.py after 2.5.0 was released. Please report back here.
Don't forget to uninstall 2.5.0 before installing latest git.
Uhm, OK. I'm a rookie (Google is my friend). I'll work on figuring that out, but I will note that the "Latest version of ESP8266 Arduino [0.4.0, arduino-esp8266fs-plugin] uses esptool.py for uploads.", and I've installed that independently.
I don't think installing esptool.py by itself will do anything, unless you're manually using it to flash the bin from command line. The core installation installs its own esptool.py and sets up to use it from the IDE.
Several comments on this.
First, I've found that using:
Tools > Board > [ESP8266 Boards] LOLIN (WEMOS) D1 R2 & mini
is the only way to get anywhere. This is irrespective of the actual board type and actual SPIFFS size. I acknowledge that this wastes a lot of flash on "pro" boards but I prefer reliability.
Second, I've found that all flavours of WeMos D1 are much better behaved by having a jumper between D0 and RST, and then including this code in setup()
// were we just reset by the IDE?
if (ESP.getResetInfoPtr()->reason == REASON_EXT_SYS_RST) {
// yes! perform a 1-second timed sleep to fake a reset
ESP.deepSleep(1E6);
}
The kinds of misbehaviour I'm talking about definitely include odd SPIFFS problems such as have been mentioned in the recent posts.
As a by-product of this setup, any code elsewhere in a sketch that runs into difficulties can trigger a restart - which simplifies a lot of design decisions.
Third, several times after using a different type of board and then coming back to a WeMos board I've fallen into the trap of opening the sketch and forgetting to check the Tools > Flash Size setting. If I upload any SPIFFS-dependent sketch while Tools > Flash Size is set to the default of "No SPIFFS" then everything turns to custard. It does not matter whether the upload is by wire or OTA. Once this happens, it usually requires a USB connection, a re-upload of the "data" directory (sometimes several attempts), and then the sketch, before normal service is resumed.
You can protect against compiling for the wrong board using something like:
#if (!ARDUINO_ESP8266_WEMOS_D1MINI)
#error "Board may not be LOLIN (WEMOS) D1 R2 & mini (check SPIFFS)"
#endif
but I have yet to find any way of chucking a compile-time error if SPIFFS is not set correctly (if anyone reading this has solved that problem, I'd be grateful if you'd share the knowledge).
Phil,
That was a very interesting post.
When using your code for stability, do you trigger the reset in code having difficulties using simply: esp_restart(); or is something else required?
Bill – AB6OR
From: Phill
Sent: Tuesday, March 19, 2019 9:38 PM
To: esp8266/Arduino
Cc: bill-orange ; Comment
Subject: Re: [esp8266/Arduino] WEMOS D1 mini Pro - 16M supporting (#2581)
Several comments on this.
First, I've found that using:
Tools > Board > [ESP8266 Boards] LOLIN (WEMOS) D1 R2 & mini
is the only way to get anywhere. This is irrespective of the actual board type and actual SPIFFS size. I acknowledge that this wastes a lot of flash on "pro" boards but I prefer reliability.
Second, I've found that all flavours of WeMos D1 are much better behaved by having a jumper between D0 and RST, and then including this code in setup()
// were we just reset by the IDE?
if (ESP.getResetInfoPtr()->reason == REASON_EXT_SYS_RST) {
// yes! perform a 1-second timed sleep to fake a reset
ESP.deepSleep(1E6);
}
The kinds of misbehaviour I'm talking about definitely include odd SPIFFS problems such as have been mentioned in the recent posts.
As a by-product of this setup, any code elsewhere in a sketch that runs into difficulties can trigger a restart - which simplifies a lot of design decisions.
Third, several times after using a different type of board and then coming back to a WeMos board I've fallen into the trap of opening the sketch and forgetting to check the Tools > Flash Size setting. If I upload any SPIFFS-dependent sketch while Tools > Flash Size is set to the default of "No SPIFFS" then everything turns to custard. It does not matter whether the upload is by wire or OTA. Once this happens, it usually requires a USB connection, a re-upload of the "data" directory (sometimes several attempts), and then the sketch, before normal service is resumed.
You can protect against compiling for the wrong board using something like:
#if (!ARDUINO_ESP8266_WEMOS_D1MINI)
#error "Board may not be LOLIN (WEMOS) D1 R2 & mini (check SPIFFS)"
#endif
but I have yet to find any way of chucking a compile-time error if SPIFFS is not set correctly (if anyone reading this has solved that problem, I'd be grateful if you'd share the knowledge).
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.
Try installing latest git and retry. There was a PR migrating from esptool-ck to latest esptool.py after 2.5.0 was released. Please report back here.
Don't forget to uninstall 2.5.0 before installing latest git.
@devyte, that seems to work swell. I uploaded 14MB SPIFFS four times, with two different boards, all without error. Much faster, too, with the compression.
Wrote 14659584 bytes (940371 compressed) at 0x00200000 in 65.0 seconds (effective 1803.6 kbit/s)...
Hash of data verified.
Hi bill-orange
You are quite right. I should have explained what I meant when I wrote "any code elsewhere in a sketch that runs into difficulties can trigger a restart". Sorry about that.
One of the problems I hit with the ESP8266 is that ESP.restart() does not work properly if the most-recent reboot was from the IDE. The pattern is that the restart fails, the watchdog timer fires, and faff-all happens after that.
It is easy to work around this problem by manually hitting the reset button after an upload but you have to remember to do that every time. Once a project becomes a production system, it's easy to forget to reset it by hand after a code change.
The code in my earlier post checking for REASON_EXT_SYS_RST mimics pressing the reset button (because of the way D0 pulls RST LOW at the end of the sleep).
Further, once you accept that D0+RST should be tied together to solve the original problem, there's no actual reason to use ESP.restart() anywhere.
So, the code I actually use is:
````
void reboot () {
// blink the LED rapidly
for (int i = 0; i < 20; i++) {
digitalWrite(LED_BUILTIN, !digitalRead(LED_BUILTIN));
delay(50);
}
// go into deep sleep for a little while (one second in this case)
ESP.deepSleep(1E6);
}
````
and in setup() my code actually looks like this:
````
// were we just reset by the IDE?
if (ESP.getResetInfoPtr()->reason == REASON_EXT_SYS_RST) {
// yes! perform a timed sleep/wake to cure this problem
reboot();
}
````
Elsewhere, any condition that might benefit from a prophylactic restart could go something like this (using WiFi as an example):
````
// 10-second timeout assuming 1/2-second delays
int timeout = 20;
// start WiFi
WiFi.begin(ssid,password);
// wait for connection
while (WiFi.status() != WL_CONNECTED) {
delay(500);
digitalWrite(LED_BUILTIN, !digitalRead(LED_BUILTIN));
timeout--;
if (timeout < 1) reboot();
}
digitalWrite(LED_BUILTIN, LOW);
````
Best case is that the reboot will cure whatever ails WiFi. Worst case is that the ESP8266 will keep restarting but the LED pattern (10 slow blinks followed by 10 rapid blinks) will be a giant clue that something is wrong.
And, to tie this back to the SPIFFS issue, if I forget to set the flash size menu item to 1M SPIFFS and an upload then corrupts SPIFFS (or makes it inaccessible), because my WiFi credentials are stored in SPIFFS the result is that the above code will exhibit this continual slow/fast blink reboot pattern. Cue sagged shoulders, slap of the forehead, and resolving to do better next time.
That’s brilliant. I am rewriting some code to remove Wundergound calls and substitute OpenWeatherMap calls. I will definitely incorporate your idea.
P. S. The reason for the rewrite is that IBM bought Wundergound and is no longer issuing Wunderground APIs.
Bill - AB6OR
Sent from my iPhone
On Mar 20, 2019, at 3:10 PM, Phill notifications@github.com wrote:
Hi bill-orange
You are quite right. I should have explained what I meant when I wrote "any code elsewhere in a sketch that runs into difficulties can trigger a restart". Sorry about that.
One of the problems I hit with the ESP8266 is that ESP.restart() does not work properly if the most-recent reboot was from the IDE. The pattern is that the restart fails, the watchdog timer fires, and faff-all happens after that.
It is easy to work around this problem by manually hitting the reset button after an upload but you have to remember to do that every time. Once a project becomes a production system, it's easy to forget to reset it by hand after a code change.
The code in my earlier post checking for REASON_EXT_SYS_RST mimics pressing the reset button (because of the way D0 pulls RST LOW at the end of the sleep).
Further, once you accept that D0+RST should be tied together to solve the original problem, there's no actual reason to use ESP.restart() anywhere.
So, the code I actually use is:
void reboot () {
// blink the LED rapidly for (int i = 0; i < 20; i++) { digitalWrite(LED_BUILTIN, !digitalRead(LED_BUILTIN)); delay(50); } // go into deep sleep for a little while (one second in this case) ESP.deepSleep(1E6);}
and in setup() my code actually looks like this:// were we just reset by the IDE? if (ESP.getResetInfoPtr()->reason == REASON_EXT_SYS_RST) { // yes! perform a timed sleep/wake to cure this problem reboot(); }Elsewhere, any condition that might benefit from a prophylactic restart could go something like this (using WiFi as an example):
// 10-second timeout assuming 1/2-second delays int timeout = 20; // start WiFi WiFi.begin(ssid,password); // wait for connection while (WiFi.status() != WL_CONNECTED) { delay(500); digitalWrite(LED_BUILTIN, !digitalRead(LED_BUILTIN)); timeout--; if (timeout < 1) reboot(); } digitalWrite(LED_BUILTIN, LOW);Best case is that the reboot will cure whatever ails WiFi. Worst case is that the ESP8266 will keep restarting but the LED pattern (10 slow blinks followed by 10 rapid blinks) will be a giant clue that something is wrong.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.
@mike-s123 great news, thanks for reporting back!
Have you recently tried to use again the full 16MB chip with LittleFS or esptool.py ?
I have not.
Neither have I. Treating all Mini Pros as D1/R2 with 1MB SPIFFS is ample. Not needing more storage means no need to keep trying to find a better solution.
Truth to tell, I have no idea what LittleFS is so I can't even make a sensible guess as to whether I have been using it or not - assuming that it would be plausible for it to be buried within the IDE and for me to be using it unawares.
As to esptool.py, implicitly via the IDE, yes, but subject to the caveat of D1/R2 + 1MB SPIFFS. Via command line, no.
LittleFS is a better FS replacement in place of SPIFFS (faster in many cases - slowness/wdt was one of the issues with SPIFFS and 16MB flash chips - and sub-directory support).
esptool.py allows to flash the FS independently from the sketch.
Closing as 16MB flash chips are supposed to be working on every boards now.
Please open a new issue if something is wrong with git-master code.
Most helpful comment
This is not an issue and this is not the place to advertise your affiliate-links.