Arduino-esp32: base64 encode resets esp32 on certain strings

[euquiq]

Hi there!

I think I found a bug: Sometimes my DOIT ESP32 DEVKIT V1 starts resetting itself when encoding a string by using the base64 library. .

When trying to encode certain quantity of characters, my ESP32 will give the following error:

CORRUPT HEAP: Bad tail at 0x3ffc9aac. Expected 0xbaad5678 got 0xbaad5600
 assertion "head != NULL" failed: file "/Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/heap/./multi_heap_poisoning.c", line 199, function: multi_heap_free
 abort() was called at PC 0x400d2af3 on core 1

 Backtrace: 0x400870ac:0x3ffcee50 0x400871ab:0x3ffcee70 0x400d2af3:0x3ffcee90 0x40086dd9:0x3ffceec0 0x40082796:0x3ffceee0 0x40082d19:0x3ffcef00 0x4000bec7:0x3ffcef20 0x400d0905:0x3ffcef40 0x400d0931:0x3ffcef80 0x400d0880:0x3ffcefa0 0x400dc84f:0x3ffceff0

 Rebooting...

I can reproduce the error with the following code:

#include <base64.h>
base64 b;

void setup()
{
    Serial.begin(115200);
    Serial.println("12345678 = " + b.encode("12345678"));
    delay(1000);
    Serial.println("123456 = " + b.encode("123456"));
    delay(1000);
    Serial.println("1234567 = " + b.encode("1234567")); //THIS LINE WILL CRASH ESP32
}

void loop()
{

}

I also tried this simple code in two other ESP32 boards, the geekworm EASY KIT ESP32-01 (revision 0 silicon) and another one with revision 1 silicon, which says ESP32_Core_board-V2.

Same behavior in the three of them.

Regards,

Enrique

[me-no-dev]

how about Serial.println(String("123456 = ") + b.encode("123456"));

[euquiq]

Hi me-no-dev!

It is not that line of code which you suggest to change that breaks the Arduino ESP32: it is the following one. Anyways I added the String() to the one you say, and ALSO the one that really crashes the ESP32, with the same behavior as before.

The line:

Serial.println("1234567 = " + b.encode("1234567")); //THIS LINE WILL CRASH ESP32

is a no-go. And the use of the String function helps not:

Opening port
Port open

12345678 = MTIzNDU2Nzg=
123456 = MTIzNDU2
CORRUPT HEAP: Bad tail at 0x3ffcc894. Expected 0xbaad5678 got 0xbaad5600
assertion "head != NULL" failed: file "/Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/heap/./multi_heap_poisoning.c", line 199, function: multi_heap_free
abort() was called at PC 0x400d2bc7 on core 1

Backtrace: 0x40087120:0x3ffcecb0 0x4008721f:0x3ffcecd0 0x400d2bc7:0x3ffcecf0 0x40086e4d:0x3ffced20 0x40082842:0x3ffced40 0x40082dc5:0x3ffced60 0x4000bec7:0x3ffced80 0x400d09a1:0x3ffceda0 0x400d09cd:0x3ffcede0 0x400d08d7:0x3ffcee00 0x400dc923:0x3ffcee50

Rebooting...

If I delete the third b.encode with that specific 7 letter string to encode, the code works as expected.

The encode function crashes the ESP32 chip when asked to encode a string with 7 chars.

Weird bug!

Regards,

Enrique.

[me-no-dev]

ahhh OK will have a look :)

[anteph]

Hi!

Same problem here with the code below.

#include <base64.h>

void setup() {
  Serial.begin(115200);
  delay(1000);
  Serial.println(base64::encode("test"));
}

void loop() {
}

Then the ESP reboots a couple of times and then stops rebooting. The message I get is the following:

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:812
load:0x40078000,len:0
load:0x40078000,len:10212
entry 0x40078a00
CORRUPT HEAP: Bad tail at 0x3ffcc8d7. Expected 0xbaad5678 got 0xbaad003d
assertion "head != NULL" failed: file "/Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/heap/./multi_heap_poisoning.c", line 199, function: multi_heap_free
abort() was called at PC 0x400d2a97 on core 1

Backtrace: 0x40087120:0x3ffcec20 0x4008721f:0x3ffcec40 0x400d2a97:0x3ffcec60 0x40086e4d:0x3ffcec90 0x40082842:0x3ffcecb0 0x40082dc5:0x3ffcecd0 0x4000bec7:0x3ffcecf0 0x400d0c69:0x3ffced10 0x400d0c95:0x3ffced50 0x400d0838:0x3ffced70 0x400dc7f3:0x3ffcedb0

Rebooting...
ets Jun  8 2016 00:22:57

Best regards,
Nuno Santos

[me-no-dev]

can you please decode the backtraces and post them here?

[anteph]

Sorry I've never done that, is there any guide for how to decode the backtraces?

Thank you

[me-no-dev]

yup: https://github.com/me-no-dev/EspExceptionDecoder

[euquiq]

@me-no-dev Here I paste my backtrace:

Decoding 12 results
0x400d2b7f: __assert_func at /Users/ivan/e/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/stdlib/../../../.././newlib/libc/stdlib/assert.c line 63 (discriminator 8)
0x40087120: invoke_abort at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/esp32/./panic.c line 553
0x4008721f: abort at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/esp32/./panic.c line 553
0x400d2b7f: __assert_func at /Users/ivan/e/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/stdlib/../../../.././newlib/libc/stdlib/assert.c line 63 (discriminator 8)
0x40086e4d: multi_heap_free at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/heap/./multi_heap_poisoning.c line 199 (discriminator 1)
0x40082842: heap_caps_free at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/heap/./heap_caps.c line 275
0x40082dc5: _free_r at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/newlib/./syscalls.c line 42
0x400d0d51: base64::encode(unsigned char*, unsigned int) at C:\Users\Enrique\Documents\Arduino\hardware\espressif\esp32\cores\esp32/base64.cpp line 50
0x400d0d7d: base64::encode(String) at C:\Users\Enrique\Documents\Arduino\hardware\espressif\esp32\cores\esp32/base64.cpp line 63
0x400d08cc: setup() at C:\Users\Enrique\Documents\Arduino\probando_base64/probando_base64.ino line 11
0x400dc8db: loopTask(void*) at C:\Users\Enrique\Documents\Arduino\hardware\espressif\esp32\cores\esp32/main.cpp line 15

It crashes, as before, when reaching the line with the "1234567" string:

#include <base64.h>
base64 b;

void setup()
{
  Serial.begin(115200);
  Serial.println("12345678 = " + b.encode("12345678"));
  delay(1000);
  Serial.println("123456 = " + b.encode("123456"));
  delay(1000);
  Serial.println("1234567 = " + b.encode("1234567")); //THIS LINE WILL CRASH ESP32
}

void loop()
{

}

Hopefully it helps.

Regards,

Enrique

[anteph]

Sorry for the delay in posting the backtrace but I'm facing some issues running the decoder tool. I will post it as soon as I can solve them.

Best regards,
Nuno Santos

[euquiq]

@me-no-dev It would seem that the base64 code in espressif IDF, which in turn arduino calls for encoding in base64 is flawed ? Maybe it has a "runaway heap" due to an uncontrolled memory / array / String space usage or something like that (I infer this from reading the:

0x40082842: heap_caps_free at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/heap/./heap_caps.c line 275

... on my backtrace).

But then, I might be completely wrong. I posted this because I would love this bug not to be stalled into oblivion. If there is something I can do, please, tell me. Maybe I can post this problem as an espressif / IDF issue ?

[me-no-dev]

no worries :) I have been completely overwhelmed with other work, so I had not time to really sit down and dig into this (i imagine it will require some time). Source for base64 is in Arduino itself: https://github.com/espressif/arduino-esp32/tree/master/cores/esp32/libb64

[anteph]

Finally had some time to update my old Arduino IDE version and decode the backtrace :)

0x400d1e47: __assert_func at /Users/ivan/e/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/stdlib/../../../.././newlib/libc/stdlib/assert.c line 63 (discriminator 8)
0x4008720c: invoke_abort at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/esp32/./panic.c line 572
0x4008730b: abort at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/esp32/./panic.c line 572
0x400d1e47: __assert_func at /Users/ivan/e/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/stdlib/../../../.././newlib/libc/stdlib/assert.c line 63 (discriminator 8)
0x40086f39: multi_heap_free at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/heap/./multi_heap_poisoning.c line 199 (discriminator 1)
0x400829a2: heap_caps_free at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/heap/./heap_caps.c line 275
0x40082ef9: _free_r at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/newlib/./syscalls.c line 42
0x400d0ca8: base64::encode(unsigned char*, unsigned int) at C:\Users\N\Documents\Arduino\hardware\espressif\esp32\cores\esp32/base64.cpp line 50
0x400d0cd5: base64::encode(String) at C:\Users\N\Documents\Arduino\hardware\espressif\esp32\cores\esp32/base64.cpp line 63
0x400d0859: setup() at C:\Users\N\Desktop\Blog\Posts\212 - ESP32 Arduino base64 encode\Base64_encode/Base64_encode.ino line 8
0x400de22f: loopTask(void*) at C:\Users\N\Documents\Arduino\hardware\espressif\esp32\cores\esp32/main.cpp line 15

Hope it helps with finding the bug.

Meanwhile I've written a small tutorial about using base64 encoding, will leave it here in case someone needs help getting started:
https://techtutorialsx.com/2017/12/09/esp32-arduino-base64-enconding/

Best regards,
Nuno Santos

[sagreen83]

Any updates on this topic?

I'm getting the same error when encoding the string "system1"

Thanks,
Scott...

[euquiq]

No update yet. Which compels me to write the following: Since I found no way of private messaging any of the moderators on espressif/arduino-esp32, maybe I can take the space to ask a couple of questions (and I hope I do not offend anyone):

1) Is espressif in charge of the arduino-esp32 branch development (meaning: are the moderators in here espressif employees or at least paid by them somehow)?

2) Since espressif got now their own IDE (IDE-IDF) ... is this ESP32-Arduino branch being deprecated or at least left to the resources of "free will - time" of enthusiasts like the moderators ? (in case answer to earlier question 1) is that they are guys "like us" and not being compensated in any way ?)

I am trying to understand the Arduino-ESP32 "scene" and decide if it is a pragmatic sound decision to move away from good old Arduino framework, shifting my efforts into espressif IDF framework (which would be more than cumbersome, given the apparent (for me) lack of libraries and third party sensors / peripherals support).

Most probably I got it all wrong, again I am sorry for any impoliteness I may have incurred, as it is not my intention other than grasp the way this espressif/Arduino-ESP32 branch works.

Regards,
Enrique

[me-no-dev]

If you consider the fact that there are a whole bunch of things that i am working on (non-arduino) you will understand. On a more positive note i can say that i am almost done with all other priority tasks and will take care of this soon. Then also this is the same lib not written by us and in use by esp8266 arduino also. You can write whatever you want but that would only bring more negativity if anything.

[g01d10x]

EDIT: : : : sorted this out on my own, I don’t think it was related to this. I had been calling the server class improperly .. thanks anyhow :)

---

Hello folks .. I'm continually astounded at the talent that's contributing to this bunch of tools and capabilities with Arduino and ESP32 ! Thanks so much !!

I seem to be having a similar problem with a Corrupt Heap .. aside from why it's crashing, I have 2 questions:
1 - What does "ficeto" have to do with anything on my local computer? (/goldilocks/ is normal)
2 - I'm far from an expert at this, but would I be accurate in guessing that the exception decoder's showing that the ESPAsyncWebServer.h is doing something with a String that's causing all of this ?

I'm getting a crash:

CORRUPT HEAP: Bad head at 0x3ffdd524. Expected 0xabba1234 got 0x3ffdd578
assertion "head != NULL" failed: file "/Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/heap/./multi_heap_poisoning.c", line 199, function: multi_heap_free
abort() was called at PC 0x400dcb0b on core 1

and here's the backtrace: ((very cool tool by the way))

Decoding 16 results
0x40087ce0: invoke_abort at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/esp32/./panic.c line 572
0x40087ddf: abort at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/esp32/./panic.c line 572
0x400dcb0b: __assert_func at /Users/ivan/e/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/stdlib/../../../.././newlib/libc/stdlib/assert.c line 63 (discriminator 8)
0x40087a09: multi_heap_free at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/heap/./multi_heap_poisoning.c line 199 (discriminator 1)
0x40083b72: heap_caps_free at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/heap/./heap_caps.c line 275
0x400840f5: _free_r at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/newlib/./syscalls.c line 42
0x400d74d6: String::~String() at /Users/goldilocks/Documents/Arduino/hardware/espressif/esp32/cores/esp32/WString.cpp line 840 (discriminator 2)
0x400d4360: AsyncWebHandler::~AsyncWebHandler() at /Users/goldilocks/Documents/Arduino/libraries/ESPAsyncWebServer/src/ESPAsyncWebServer.h line 321
0x400d4376: AsyncWebHandler::~AsyncWebHandler() at /Users/goldilocks/Documents/Arduino/libraries/ESPAsyncWebServer/src/ESPAsyncWebServer.h line 321
0x4012fc7f: std::_Function_handler ::_M_invoke(std::_Any_data const&, AsyncWebHandler* const&) at /Users/goldilocks/Documents/Arduino/libraries/ESPAsyncWebServer/src/WebServer.cpp line 36
:  (inlined by) _M_invoke at /Users/goldilocks/Documents/Arduino/hardware/espressif/esp32/tools/xtensa-esp32-elf/xtensa-esp32-elf/include/c++/5.2.0/functional line 1871
0x400d67af: std::function ::operator()(AsyncWebHandler* const&) const at /Users/goldilocks/Documents/Arduino/libraries/ESPAsyncWebServer/src/WebServer.cpp line 175
0x400d6945: LinkedList ::free() at /Users/goldilocks/Documents/Arduino/libraries/ESPAsyncWebServer/src/StringArray.h line 166
:  (inlined by) AsyncWebServer::reset() at /Users/goldilocks/Documents/Arduino/libraries/ESPAsyncWebServer/src/WebServer.cpp line 179
0x400d69a6: AsyncWebServer::~AsyncWebServer() at /Users/goldilocks/Documents/Arduino/libraries/ESPAsyncWebServer/src/WebServer.cpp line 55
0x400d1eb8: setup() at /Users/goldilocks/Documents/Arduino/2017/201712/MySkyESP32New/MySkyESP32New.ino line 77
0x401190db: loopTask(void*) at /Users/goldilocks/Documents/Arduino/hardware/espressif/esp32/cores/esp32/main.cpp line 15

Much appreciated !
Cheers to all

By the way .. @euquiq .. did you try it with other combinations of shorter/longer strings? Is it all of them with even numbers of digits that work and the odds fail?

[atanisoft]

@g01d10x "ficeto" is who compiled the esp32 core libraries and/or toolchain.

From the backtrace it appears to be an issue of out of heap and not directly related to the base64 code.

I suspect the issue with the base64 code is actually a compile optimization gone wrong. The code is pretty odd in that it uses a switch and while loop embedded in the switch (before the first case statement!). Very likely this can use a rewrite to be cleaner, I can submit a PR with a replacement if it helps. I know @me-no-dev is very busy usually.

[XYZVector]

Guys, I think this is the heap management system. I have the same issue, and the same error. I have plenty of free space. My issue occurs during when my program tries to close a socket. Well it's the me-no-dev AsycWebSocket. However I don't think that the AsycWebSocket is the issue. Here is my stack trace, and it's always pointing to the same thing. What really sucks about this is heap walking is on a level lower than the Arduino ESP 32 port since it uses the ESP libraries. The code is exactly the same I use on a 8266 and it works a treat. This error never pops up, and the fact that you guys are having the same issues in other libraries makes me think that the problem is in the Libraries compiled by ESP32. The error looks like the link list of free blocks or memory blocks in the heap manager gets corrupted. I don't think me-no-dev is gonna be able to fix this guys. This looks to be an issue with the dynamic memory. ESP is going to have to fix this. Or you will have to find work a around to the issue. Remember the ESP32 is a moving target right now.

CORRUPT HEAP: Bad head at 0x3ffe5000. Expected 0xabba1234 got 0x3ffe5104
assertion "head != NULL" failed: file "/Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/heap/./multi_heap_poisoning.c", line 199, function: multi_heap_free
abort() was called at PC 0x400d5f47 on core 1

Decoding 11 results
0x40088030: invoke_abort at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/esp32/./panic.c line 572
0x4008812f: abort at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/esp32/./panic.c line 572
0x400d5f97: __assert_func at /Users/ivan/e/newlib_xtensa-2.2.0-bin/newlib_xtensa-2.2.0/xtensa-esp32-elf/newlib/libc/stdlib/../../../.././newlib/libc/stdlib/assert.c line 63 (discriminator 8)
0x40087d41: multi_heap_free at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/heap/./multi_heap_poisoning.c line 284
0x4008225a: heap_caps_free at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/heap/./heap_caps.c line 136
0x40082f79: _free_r at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/newlib/./syscalls.c line 42
0x40149d8d: tcp_close_shutdown at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/lwip/core/tcp.c line 225
0x40149e4f: tcp_close at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/lwip/core/tcp.c line 305
0x4015a50d: _tcp_close_api(tcpip_api_call*) at e:\CharmasterV9\Firmware/lib\AsyncTCP/AsyncTCP.cpp line 698
0x4014646d: tcpip_thread at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/lwip/api/tcpip.c line 474

[espfreak]

If you change the malloc size in the file base64.cpp it will work.
~/Arduino/hardware/espressif/esp32/cores/esp32/base64.cpp
change line 41
size_t size = ((length * 1.6f) + 1);
to
size_t size = ((length * 1.8f) + 1);

[Orel-A]

Shouldn't it be:
size_t size = 4 * ((length + 2) / 3) + 1;

[espfreak]

This doesn't work with a large string (try 100 characters).

[everslick]

This issue is closed, because it looks as if it is not a bug or problem with the ESP32 Arduino core or its support libraries. For general API usage questions or help on specific coding challenges, please visit the arduino-esp32 Gitter channel. If you feel this issue was closed in error, reopen it and comment, why you think this is a bug in the Arduino-Core.

[euquiq]

@everslick Hi There ! I wonder then if the comments from @espfreak and @RageSmirk about changing malloc size in the file base64.cpp

~/Arduino/hardware/espressif/esp32/cores/esp32/base64.cpp
change line 41 to:

size_t size = 4 * ((length + 2) / 3) + 1;

is not relevant ? It definitely "looks" like an Arduino problem.

Care please to elaborate on it ?

Regards,

Enrique

[everslick]

I'm currently aggressively cleaning the issue tracker, simply due to the big number of open issues, many of them outdated, abandoned or misplaced. It is well possible, that I close issues that should not be closed, but at least people get alerted and will speak up, if they still care about such an issue (like you did).

In this case I assumed (maybe wrongly) that the original bugreport was due to heap corruption caused by a user bug elsewhere. But I'm all for reopening this issue.

[everslick]

how about size_t size = ((4 * length / 3) + 3) & ~3; ?

[toddlesher]

For the String "abc:def", that size is 12. "free()" causes the corruption.
Adding an additional line worked for me:
if (size < 32) size = 32;
I did not try other numbers.

[marcelkottmann]

libbase64 already contains a macro that can be used to calculate the base64 encoded string length.
You also have to increment the calculated size by 1 for the trailing zero character.
Additionally libbase64 introduces a newline character in the output every 72 chars, which is not indended. Therefore I removed it.
I created pull request #2007 which should fix this bug.

[Uzlopak]

"1234567" is not a valid base64-encoded string, because it has to have even number and the length has to be dividable by 4, so it has to add a "=" to the string or atleast the used method must process it as if it was there. "abc:def" is not a valid base64-encoded string, because the ":" character is not a valid character for base64. Only the 26 latin-letters (a-z, A-Z), the 10 numbers(0-9) and + / are allowed characters ( 26 * 2 + 10 + 2 = 64) plus the = for padding.

Probably checking the length and adding padding + checking if the string contains valid characters would prevent the crashing.

[everslick]

The fix has been merged. https://github.com/espressif/arduino-esp32/pull/2007

[euquiq]

@arasabbasi when you say ""1234567" is not a valid base64-encoded string" ... it is not an encoded string, it is the string I want to encode with b.encode("1234567")
I am struggling to understand why this issue has been closed. I have no idea on what goes on behind doors, inside the base64 encoding library, but I would assume that I would be able to throw at it any string ...

Correct me if I am wrong: It can only encode strings with length divisible by 2 ?

[everslick]

I tested with the string "1234567" and it worked as expected. I cannot make it crash whatsoever. @euquiq can you still reproduce the issue with latest master?

[Uzlopak]

Oh wait, you are right. Probably the size of the result was wrong calculated and now it's fixed?