Esp-idf: Statically allocated DRAM is limited to 160KB (IDFGH-1187)

[dsyleixa]

code compiles and works for M4 plus TFT HX8357,
but fails for ESP32;
Arduino IDE 1.8.8
Board: Adafruit Feather ESP32
core: ESP32 espressive 1.0.2

compile errors (when linking) for my code

• what am I missing?
  Perhaps too much code and too little RAM at ESP32?

Arduino: 1.8.8 (Windows 7), Board: "Adafruit ESP32 Feather, 80MHz, 921600, None"

D:\arduino\arduino-builder -dump-prefs -logger=machine -hardware D:\arduino\hardware -hardware D:\arduino\portable\packages -tools D:\arduino\tools-builder -tools D:\arduino\hardware\tools\avr -tools D:\arduino\portable\packages -built-in-libraries D:\arduino\libraries -libraries D:\arduino\portable\sketchbook\libraries -fqbn=esp32:esp32:featheresp32:FlashFreq=80,UploadSpeed=921600,DebugLevel=none -ide-version=10808 -build-path C:\Users\hwAppData\Local\Temp\arduino_build_226627 -warnings=none -prefs=build.warn_data_percentage=75 -prefs=runtime.tools.esptool_py.path=D:\arduino\portable\packages\esp32\tools\esptool_py\2.6.1 -prefs=runtime.tools.esptool_py-2.6.1.path=D:\arduino\portable\packages\esp32\tools\esptool_py\2.6.1 -prefs=runtime.tools.mkspiffs.path=D:\arduino\portable\packages\esp32\tools\mkspiffs0.2.3 -prefs=runtime.tools.mkspiffs-0.2.3.path=D:\arduino\portable\packages\esp32\tools\mkspiffs0.2.3 -prefs=runtime.tools.xtensa-esp32-elf-gcc.path=D:\arduino\portable\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0 -prefs=runtime.tools.xtensa-esp32-elf-gcc-1.22.0-80-g6c4433a-5.2.0.path=D:\arduino\portable\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0 -verbose D:\Akten\Programmierung\ArduinoProgs\ConwaysGameOfLife\ConwayGoL_M4_Logic_NOR\ConwayGoL_M4_Logic_NOR.ino
D:\arduino\arduino-builder -compile -logger=machine -hardware D:\arduino\hardware -hardware D:\arduino\portable\packages -tools D:\arduino\tools-builder -tools D:\arduino\hardware\tools\avr -tools D:\arduino\portable\packages -built-in-libraries D:\arduino\libraries -libraries D:\arduino\portable\sketchbook\libraries -fqbn=esp32:esp32:featheresp32:FlashFreq=80,UploadSpeed=921600,DebugLevel=none -ide-version=10808 -build-path C:\Users\hwAppData\Local\Temp\arduino_build_226627 -warnings=none -prefs=build.warn_data_percentage=75 -prefs=runtime.tools.esptool_py.path=D:\arduino\portable\packages\esp32\tools\esptool_py\2.6.1 -prefs=runtime.tools.esptool_py-2.6.1.path=D:\arduino\portable\packages\esp32\tools\esptool_py\2.6.1 -prefs=runtime.tools.mkspiffs.path=D:\arduino\portable\packages\esp32\tools\mkspiffs0.2.3 -prefs=runtime.tools.mkspiffs-0.2.3.path=D:\arduino\portable\packages\esp32\tools\mkspiffs0.2.3 -prefs=runtime.tools.xtensa-esp32-elf-gcc.path=D:\arduino\portable\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0 -prefs=runtime.tools.xtensa-esp32-elf-gcc-1.22.0-80-g6c4433a-5.2.0.path=D:\arduino\portable\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0 -verbose D:\Akten\Programmierung\ArduinoProgs\ConwaysGameOfLife\ConwayGoL_M4_Logic_NOR\ConwayGoL_M4_Logic_NOR.ino
Using board 'featheresp32' from platform in folder: D:\arduino\portable\packages\esp32\hardware\esp32\1.0.2
Using core 'esp32' from platform in folder: D:\arduino\portable\packages\esp32\hardware\esp32\1.0.2

...

Linking everything together...
"D:\arduino\portable\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0/bin/xtensa-esp32-elf-gcc" -nostdlib "-LD:\arduino\portable\packages\esp32\hardware\esp32\1.0.2/tools/sdk/lib" "-LD:\arduino\portable\packages\esp32\hardware\esp32\1.0.2/tools/sdk/ld" -T esp32_out.ld -T esp32.common.ld -T esp32.rom.ld -T esp32.peripherals.ld -T esp32.rom.spiram_incompatible_fns.ld -u ld_include_panic_highint_hdl -u call_user_start_cpu0 -Wl,--gc-sections -Wl,-static -Wl,--undefined=uxTopUsedPriority -u __cxa_guard_dummy -u __cxx_fatal_exception -Wl,--start-group "C:\Users\hw\AppData\Local\Temp\arduino_build_226627\sketch\ConwayGoL_M4_Logic_NOR.ino.cpp.o" "C:\Users\hw\AppData\Local\Temp\arduino_build_226627\libraries\Wire\Wire.cpp.o" "C:\Users\hw\AppData\Local\Temp\arduino_build_226627\libraries\SPI\SPI.cpp.o" "C:\Users\hw\AppData\Local\Temp\arduino_build_226627\libraries\Adafruit-GFX-Library-master\Adafruit_GFX.cpp.o" "C:\Users\hw\AppData\Local\Temp\arduino_build_226627\libraries\Adafruit-GFX-Library-master\Adafruit_SPITFT.cpp.o" "C:\Users\hw\AppData\Local\Temp\arduino_build_226627\libraries\Adafruit-GFX-Library-master\glcdfont.c.o" "C:\Users\hw\AppData\Local\Temp\arduino_build_226627\libraries\Adafruit_HX8357_Library-master\Adafruit_HX8357.cpp.o" "C:\Users\hw\AppData\Local\Temp\arduino_build_226627\libraries\Adafruit_STMPE610-master\Adafruit_STMPE610.cpp.o" "C:\Users\hw\AppData\Local\Temp\arduino_build_226627\core\core.a" -lgcc -lopenssl -lbtdm_app -lfatfs -lwps -lcoexist -lwear_levelling -lesp_http_client -lprotobuf-c -lhal -lnewlib -ldriver -lbootloader_support -lpp -lfreemodbus -lmesh -lsmartconfig -ljsmn -lwpa -lethernet -lphy -lfrmn -lapp_trace -lfr_coefficients -lconsole -lulp -lwpa_supplicant -lfreertos -lbt -lmicro-ecc -lesp32-camera -lcxx -lxtensa-debug-module -ltcp_transport -lmdns -lvfs -lmtmn -lesp_ringbuf -lsoc -lcore -lfb_gfx -lsdmmc -llibsodium -lcoap -ltcpip_adapter -lprotocomm -lesp_event -limage_util -lc_nano -lesp-tls -lasio -lrtc -lspi_flash -lwpa2 -lwifi_provisioning -lesp32 -lface_recognition -lapp_update -lnghttp -lspiffs -lface_detection -lespnow -lnvs_flash -lesp_adc_cal -llog -ldl_lib -lsmartconfig_ack -lexpat -lfd_coefficients -lm -lmqtt -lc -lheap -lmbedtls -llwip -lnet80211 -lesp_http_server -lpthread -ljson -lesp_https_ota -lstdc++ -Wl,--end-group -Wl,-EL -o "C:\Users\hw\AppData\Local\Temp\arduino_build_226627/ConwayGoL_M4_Logic_NOR.ino.elf"
d:/arduino/portable/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/bin/ld.exe: C:\Users\hwAppData\Local\Temp\arduino_build_226627/ConwayGoL_M4_Logic_NOR.ino.elf section .dram0.bss' will not fit in regiondram0_0_seg'

d:/arduino/portable/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/bin/ld.exe: DRAM segment data does not fit.

d:/arduino/portable/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/bin/ld.exe: region `dram0_0_seg' overflowed by 74776 bytes

collect2.exe: error: ld returned 1 exit status

Bibliothek Wire in Version 1.0.1 im Ordner: D:\arduino\portable\packages\esp32\hardware\esp32\1.0.2\libraries\Wire wird verwendet
Bibliothek SPI in Version 1.0 im Ordner: D:\arduino\portable\packages\esp32\hardware\esp32\1.0.2\libraries\SPI wird verwendet
Bibliothek Adafruit-GFX-Library-master in Version 1.4.10 im Ordner: D:\arduino\portable\sketchbook\libraries\Adafruit-GFX-Library-master wird verwendet
Bibliothek Adafruit_HX8357_Library-master in Version 1.1.4 im Ordner: D:\arduino\portable\sketchbook\libraries\Adafruit_HX8357_Library-master wird verwendet
Bibliothek Adafruit_STMPE610-master in Version 1.0.1 im Ordner: D:\arduino\portable\sketchbook\libraries\Adafruit_STMPE610-master wird verwendet
exit status 1
Fehler beim Kompilieren für das Board Adafruit ESP32 Feather.

complete error msg in attachement
compileError.txt

source code also attached
ConwayGoL_M4_Logic_NOR.zip

[negativekelvin]

Yes too much static ram usage. This is the wrong place to post

Read this
https://github.com/espressif/arduino-esp32/issues/1163
Ask for help here
https://esp32.com/viewforum.php?f=19

[dsyleixa]

hmmm - that's weird:

ESP32: 4 MByte flash, 520 KB SRAM
https://learn.adafruit.com/adafruit-huzzah32-esp32-feather/overview

M4: 512 KB flash, 192 KB RAM
https://learn.adafruit.com/adafruit-huzzah32-esp32-feather/overview

if ESP32 RAM is so restricted, then that is big drawback and that issue should be fixed ASAP!

[dsptech]

from https://docs.espressif.com/projects/esp-idf/en/latest/api-reference/system/mem_alloc.html

"Note:
Due to a technical limitation, the maximum statically allocated DRAM usage is 160KB. The remaining 160KB (for a total of 320KB of DRAM) can only be allocated at runtime as heap."

[dsyleixa]

OMG, that should have been made absolutely clear BEFORE I had purchased that thing!
OTOH, Adafruit advertises the ESP by
520 KB SRAM

Can that "technical limitation" be reworked and fixed?

[projectgus]

Hi @dsyleixa,

The total SRAM is 520KB but this is DRAM+IRAM, and only DRAM (320KB) is normally used for data, IRAM is normally used for executable instructions. It is possible to store data in IRAM but there are restrictions.

Regarding how Adafruit advertise the ESP32, you'll have to take this up with them. But they are technically correct.

Working around the static memory limitation should be possible without too much work. If you have some larger buffer like this:

static uint8_t framebuffer[131072];

Then change it to a pointer, and allocate it during setup with malloc() or calloc(). This makes it dynamic memory not static memory:

static uint8_t *framebuffer;

void setup() {
   framebuffer = calloc(1, 131072);
   assert(framebuffer != NULL);

   // do other setup stuff
}

Also, as @negativekelvin mentions, for Arduino issues please post on the Arduino issue tracker or the Arduino forum of esp32.com.

[dsyleixa]

sorry, but I can't handle pointer and allocates, that is far beyond my skills :(

I don't post to Arduino.cc forum, because there are too many outrageous and insulting users.

[projectgus]

There is an Arduino subforum on esp32.com, that's the one I was referring to:
https://esp32.com/viewforum.php?f=19

[dsyleixa]

ok, perhaps in future.
As to now, to me it's an ESP32 hardware issue which should be reworked and fixed, not discussed ;)

[projectgus]

There's no hardware issue. This is a software issue in ESP-IDF that the amount of statically assigned DRAM in an app is limited to 160KB. We don't have a GitHub issue tracking this, so we can start tracking it here.

We don't have an ETA for fixing this issue as it's complex to fix (for architectural reasons) and it's generally easy to work around by assigning large buffers dynamically (see above post with suggestion). But we do plan to fix it at some point.

(There is also the related but different issue #3211 )

[galah92]

I'm also looking for a solution for that, dynamic allocation makes my code much more error-prone.