Arduino: "Bad Record Mac" detected on server when establish connection using WiFiClientSecure

[HW-Siew]

I am trying to connect a NodeMcu to a MQTT server using secured connection. The secured connection with server can be successfully established. However, I noticed the frequent disconnection on nodeMcu and i check the log on server end. The log shows that it captured multiple times of network error {tls_alert, "bad record mac"} of client. May i know what could possibly wrong on my NodeMcu? The core i am using is 2.2.0-rc1.

Thanks.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[korg1000]

Same here with mqtt on 2.2.0 with ssl

[igrr]

This issue (and other similar tickets, will link them all later) occurs because axTLS is essentially a half-duplex library, it has one buffer which is used for both RX and TX. This isn't an issue for protocols like HTTP, which are also half-duplex. However with MQTT, what happens is that you may have both read and write operation happening at the same time. If write operation happens after read operation has started, receive buffer is overwritten and bad things happen.

I need to think about this a bit, maybe there is an easy way to work around this issue without having to introduce another buffer into LwIP (that would require considerable amount of work) or switching to other TLS library, like mbedTLS (we are going to use mbedTLS in the ESP32 core, but with RTOS SDK).
Will update this issue next week.

[igrr]

Okay, did some sort of a workaround: now if read or available methods are called, and there is at least _some_ data available from the TCP layer, these methods will block until the whole SSL message is received and decrypted. Then you should read all the data before sending anything — otherwise the data in read buffer will get corrupted. For most applications this shouldn't be a serious limitation.
MQTT is now much more reliable, although mosquitto does disconnect the ESP client occasionally. I don't think this is related to SSL though.

The updated version has been pushed in e8b8a60.

[korg1000]

I already had the 2.2.0. How could i get this patch fix? Remove the esp board and install it again?

[kaeferfreund]

Just use the git version:
https://github.com/esp8266/Arduino#using-git-version

[HW-Siew]

@igrr, Thank you so much for the efforts. The stability improved so much. I experienced occasionally long period (worst case 30+ seconds) blocking. Any idea what could be the possible reason for that?

Thanks for the advise.

[igrr]

I'll run the tests again and see if i can reproduce that.
What is your publish/subscribe pattern? I.e. how often and how much does ESP publish and how often and how much subscribed data does it get?

[igrr]

@korg1000 You can also check latest staging version (2.3.0-rc1), which has these changes.

[HW-Siew]

Hi, @igrr, the esp only subscribe messages and display the message in serial monitor. Each message size < 128 bytes. I sent a message from browser with every click of button, roughly 1 second per message or less. I ran a few hundreds click in a row. Most of the time the esp response promptly. However, the esp ran into blocking for a random amount of time then resumed operation occasionally, worst case encountered almost 30 seconds blocking.

I am using NodeMCU 0.9 and usb power supply. I am wondering would it be the power supply issue. Tho, the esp didnt hang. It resumed operation and display those messages after blocking.

i ran into a soft wdt reset also. the following are the decoded stack dump.

0x402087a0: WiFiClientSecure::available() at C:\Users\HW\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0-rc1\libraries\ESP8266WiFi\src/WiFiClientSecure.cpp line 502 0x4020cb2b: PubSubClient::readByte(unsigned char*) at C:\Users\HW\Documents\Arduino\libraries\PubSubClient\src/PubSubClient.cpp line 641 0x4020cc75: PubSubClient::readPacket(unsigned char*) at C:\Users\HW\Documents\Arduino\libraries\PubSubClient\src/PubSubClient.cpp line 641 0x4020877e: SSLContext::available() at C:\Users\HW\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0-rc1\libraries\ESP8266WiFi\src/WiFiClientSecure.cpp line 502 0x4020877e: SSLContext::available() at C:\Users\HW\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0-rc1\libraries\ESP8266WiFi\src/WiFiClientSecure.cpp line 502 0x4020d1c9: PubSubClient::loop() at C:\Users\HW\Documents\Arduino\libraries\PubSubClient\src/PubSubClient.cpp line 641 0x40228701: wifi_station_connect at ?? line ? 0x4020877e: SSLContext::available() at C:\Users\HW\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0-rc1\libraries\ESP8266WiFi\src/WiFiClientSecure.cpp line 502 0x40207758: ESP8266WiFiSTAClass::status() at C:\Users\HW\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0-rc1\libraries\ESP8266WiFi\src/ESP8266WiFiSTA.cpp line 333 0x402087bf: WiFiClientSecure::connected() at C:\Users\HW\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0-rc1\libraries\ESP8266WiFi\src/WiFiClientSecure.cpp line 502 0x40204ca8: MqttClient::loop() at C:\Users\HW\AppData\Local\Temp\builda1c577501961b4e0365618c3762dbe2f.tmp\sketch/MqttClient.cpp line 771 0x40201d52: delay at C:\Users\HW\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0-rc1\cores\esp8266/core_esp8266_wiring.c line 53 0x40207161: loop at C:\Users\HW\Documents\Arduino\thingsConnect_v5/Wifi.ino line 64 0x4020f73c: loop_wrapper at C:\Users\HW\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0-rc1\cores\esp8266/core_esp8266_main.cpp line 43 0x40100718: cont_norm at C:\Users\HW\AppData\Local\Arduino15\packages\esp8266\hardware\es