Arduino-esp32: How can I reset Serial2 / UART2?

[jestebanes]

Hardware:

Board: Own design based on ESP-WROOM-32
Core Installation/update date: 23/sept/2017
IDE name: Arduino IDE
SDK version: v3.0-dev-745-gc4e65d6a
Flash Frequency: 80Mhz
Upload Speed: 512000

Description:

I am using OTA to upgrade my firmware but after a software update Serial2 doesn't work properly
Apparently, if I make a hardware reset, the serial port works fine but if ESP.restart() is used the serial port doesn't work.

Doing a little investigation, ESP.restart call to esp_restart(); defined in system.h and I found, the library header. Then the problem is clear UART1 and UART2 weren't reset.

how can I reset those UARTs?

/**

• @brief Restart PRO and APP CPUs.
  *
• This function can be called both from PRO and APP CPUs.
• After successful restart, CPU reset reason will be SW_CPU_RESET.
• Peripherals (except for WiFi, BT, UART0, SPI1, and legacy timers) are not reset.
• This function does not return.
  */
  void esp_restart(void) __attribute__ ((noreturn));

Best wishes

[me-no-dev]

What do you see as errors? Old data? Generally SerialX.begin should reinit the serials.

[kurtow]

Board: ESPRESSIF ESP32-DEVKITC based on ESP-WROOM-32
Core Installation/update date: 23/sept/2017
IDE name: Arduino IDE

Hello,
I have the same problem.
After a hardware reset, the Serial2 runs with a probability of about 50%
(sometimes I had to reset three or four times before it starts)

After restarting from deepsleep Serial2 runs with a probability of 20%.
(In most cases its hanging after restart)
I mean (but I am not sure), that this problem did not occur
at my last arduino-core download of 21.08.

edited by kurtow:
here is new issue by emilekm2142
Hardware serial 2 does not work #665

kindly regards

[jestebanes]

Hello @me-no-dev , I see a lot of crap

This is a communication after one hard reset or powering up the device, using UART2 connected to pins 16 & 17 at 19200 bauds, the first line is a question sent by ESP32 to the serial client and the second line is the response of the client on the serial bus, the numbers before : are the timeline. The data is obtained with an external hardware that sniffs the serial bus. Every line ends with an ACK, client sent ACK to ESP32 and ESP32 send ACK to the client

P300138824:031D58C0156F5C 1A
P300138833:03C059C015C9E9 CA

P300138839:0300FF00C0F8FD 0A
P300138849:06C0FE001000BDA46159 CA

P300141726:031D58C0002DC8 1A
P300141733:03C059C0008B7D CA

P300142151:021D5C05F72D 1A
P300142159:05C05D05303332CCE8 CA

P300142166:026D5C052F25 6A
P300142201:05C05D05323531384D CA

After the OTA update the first command change to this sequence, you can see the same bytes but they are not sent in the correct order, is like they are rotated in the buffer. In these sequences, there is no ACK because they are wrong, ESP tries to send several times without success.

P300121672:021D58031D5800
P300121700:156F5C031D58C0
P300121727:156F5C031D58C0
P300121748:156F5C031D58C015
P300122749:6F5C031D58C0156F
P300123750:5C031D58C0156F5C
P300124753:031D58C0156F5C00
P300125759:1D58C0156F5C031D
P300126762:58C0156F5C031D58
P300127765:C0156F5C0300FF00

But the bytes read by ESP32 in serial port are wrong too if I read the last sequence with ESP32, I receive

7780: 50006100E58A1A
7801: 041150006100E5
7822: 8A1A0400500011
7844: 0043021D58C292421458C2
7869: C29242021D58C2
7897: 5000110043D81A
7922: 42CA021D58C292
7944: 041150006100E5
7971: 8A1A42CA021D58
7998: C29242021D58C2

I think there must be some kind of bits in the UART register that becomes 0 after a hard reset, but serial.begin ignores it and cause this issue.

Best regards

[me-no-dev]

Please pull the latest changes and report :) Thanks!

[jestebanes]

Hello @me-no-dev, this issue was not solved with the last commit

Best regards

[jestebanes]

Is the same thing, before the OTA update, with serial 2, the data sent is:

P300235436: 021D58C29242 1A

After the OTA update, the data is rotated in the serial 2

P300426334: 9242 CA 021D58C2

The CA is the CRC failure, same code different behavior. After a hard reset, the system goes to their normal behavior.
¿is this a problem to report to esp-idf?

Bye

[me-no-dev]

looks like data comes out of order?

[jestebanes]

The command that sent the data is Serial2.printf and I read with an external Arduino.
Is like the begin of the fifo was shifted two or three bytes and the serial2.printf command writes the bytes in a wrong place or the serial port hardware takes it from a shifted place.

[me-no-dev]

that is not really possible ;) Serial.print sends the bytes one by one in a blocking manner

[jestebanes]

Upps, not print is write
if(Serial2.write(send, tamPaq)!=tamPaq)

that calls to hardwareSerial.cpp

size_t HardwareSerial::write(const uint8_t *buffer, size_t size)
{
uartWriteBuf(_uart, buffer, size);
return size;
}

that calls to esp32-hal-uart.c

void uartWriteBuf(uart_t* uart, const uint8_t * data, size_t len)
{
if(uart == NULL) {
return;
}
UART_MUTEX_LOCK();
while(len) {
while(len && uart->dev->status.txfifo_cnt < 0x7F) {
uart->dev->fifo.rw_byte = *data++;
len--;
}
}
UART_MUTEX_UNLOCK();
}

This function isn't waiting to transmit the byte, while still room in the fifo it is pushing every byte.

[lonerzzz]

Not sure if it applies equally here but I found that over a reboot, the I2C FSM (state machine) was left in a funny state. I just submitted a pull request to add the reset function for I2C. It does so by calling the following:

periph_module_disable( PERIPH_I2C0_MODULE );
delay( 20 ); // Seems long but delay was chosen to ensure system teardown and setup without core generation
periph_module_enable( PERIPH_I2C0_MODULE );

For the UART, the same type of operation would be:

periph_module_disable( PERIPH_UART0_MODULE );
delay( 20 );
periph_module_enable( PERIPH_UART0_MODULE );

or using PERIPH_UART1_MODULE or PERIPH_UART2_MODULE depending on which UART you are using.

I had to add code to the I2C HAL layer to do so, had to include the following:

#include "driver/periph_ctrl.h"

and had to call the I2C initialization code once again after doing so. Doing something similar may be worth trying in your case.

[jestebanes]

Thanks @lonerzzz, I try but this does not work for me, There is some change in the behavior in UART2 but still desynchronized
Best regards

bool Dbus2::init(Debug *pdebug){
dbg = pdebug;
speed = 19200;

periph_module_disable(PERIPH_UART2_MODULE);
delay(20);
periph_module_enable(PERIPH_UART2_MODULE);

Serial2.begin(19200, SERIAL_8N1);

return true;
}

[lonerzzz]

One other thing worth looking at is in uartBegin in the file esp32-hal-uart.c. The code there is such that if the queue has been created, it won't be created again so if you were out of sync and the internal queue is unchanged, then you are likely facing some unexpected data right from the start.

    if(queueLen && uart->queue == NULL) {
        uart->queue = xQueueCreate(queueLen, sizeof(uint8_t)); //initialize the queue
        if(uart->queue == NULL) {
            return NULL;
        }
    }

It would be worth ensuring that the queue is empty as a part of resetting.

[me-no-dev]

OK I think it's time you show some code :) both on ESP and AVR ;)

[jestebanes]

I'm out for a while.
I've tried to read the port status in both conditions, with a hardware reset and a software reset by an OTA update, but I can't find some significative differences in the status of the UART registers. This is why module_disable and enable doesn't work.
I use this piece of software into the esp32-hal-uart.c

void uartConfiguration(uint8_t uart_nr, char resul, int size)
{
if(uart_nr != 2) {
return; //Not return NULL
}
uart_t uart = &_uart_bus_array[uart_nr];
UART_MUTEX_LOCK();
snprintf(resul, size, "\nUart2 register values:\nUART_CONF0 %08X\nUART_CONF1 %08X\nUART_CLKDIV %08X\nUART_FLOW_CONF %08X\nUART_SWFC %08X\nUART_SLEEP_CONF %08X\nUART_IDLE_CONF %08X\nUART_RS485_CONF %08X\nUART_STATUS %08X\nUART_AUTOBAUD %08X\nUART_FIFO_REG %08X\nUART_MEM_CONF %08X\nUART_MEM_CNT %08X\nAT_CMD_PRECNT %08X\nAT_CMD_POSTCNT %08X\nAT_CMD_GAPOUT %08X\nAT_CMD_CHAR %08X\nINT_RAW %08X\nINT_ST %08X\nINT_ENA %08X\nINT_CLR %08X\n",
ESP_REG(UART_CONF0_REG(uart_nr)), ESP_REG(UART_CONF1_REG(uart_nr)), ESP_REG(UART_CLKDIV_REG(uart_nr)),
ESP_REG(UART_FLOW_CONF_REG(uart_nr)), ESP_REG(UART_SWFC_CONF_REG(uart_nr)), ESP_REG(UART_SLEEP_CONF_REG(uart_nr)),
ESP_REG(UART_IDLE_CONF_REG(uart_nr)), ESP_REG(UART_RS485_CONF_REG(uart_nr)),
ESP_REG(UART_STATUS_REG(uart_nr)), ESP_REG(UART_AUTOBAUD_REG(uart_nr)), ESP_REG(UART_FIFO_REG(uart_nr)),
ESP_REG(UART_MEM_CONF_REG(uart_nr)), ESP_REG(UART_MEM_CNT_STATUS_REG(uart_nr)), ESP_REG(UART_AT_CMD_PRECNT_REG(uart_nr)),
ESP_REG(UART_AT_CMD_POSTCNT_REG(uart_nr)), ESP_REG(UART_AT_CMD_GAPTOUT_REG(uart_nr)), ESP_REG(UART_AT_CMD_CHAR_REG(uart_nr)),
ESP_REG(UART_INT_RAW_REG(uart_nr)), ESP_REG(UART_INT_ST_REG(uart_nr)), ESP_REG(UART_INT_ENA_REG(uart_nr)),
ESP_REG(UART_INT_CLR_REG(uart_nr)));
UART_MUTEX_UNLOCK();
}

Following the @lonerzzz suggestion I change the uartBegin to delete the oldest structures and I insert this code in uartBegin with no results at all

if !CONFIG_DISABLE_HAL_LOCKS

if(uart->lock != NULL) {
    vSemaphoreDelete(uart->lock);
    uart->lock=NULL;
}
if(uart->lock == NULL) {
    uart->lock = xSemaphoreCreateMutex();
    if(uart->lock == NULL) {
        return NULL;
    }
}

endif

if(uart->queue != NULL){
    vQueueDelete(uart->queue);
    uart->queue = NULL;
}
if(queueLen && uart->queue == NULL) {
    uart->queue = xQueueCreate(queueLen, sizeof(uint8_t)); //initialize the queue
    if(uart->queue == NULL) {
        return NULL;
    }
}else{
    xQueueReset(uart->queue);
}

[jestebanes]

Hi @me-no-dev, I'm out for a while, your right but my code is about 10k. I'm going to write a short version that it only checks the trouble so you can try in your esp32
Bye

[nkostis]

We are facing a similar issue with the Serial2 port using the latest (as of yesterday, 30/Oct) arduino-esp32 base.

After an ESP.restart() call, if we write X bytes to the serial port, instead of seeing those X bytes at the output, we see an equal number of bytes that have been previously sent pushed out, i.e. bytes that are past values (before restart) of the tx fifo.
If we sent X more bytes, we’ll see equal number of bytes pushed to the output starting from the point where the previous (wrong) TX operation ended, and so on.

Calling Flush() on the serial port seems to have no effect as well.

If we push the board’s ‘Enable’ button or power cycle the board, no such problems occur.

[stickbreaker]

@nkostis is this problem related to hardware fifo? It looks to me that there is stale data in the FIFO from before the reboot. The hardware reference describes a 1024 byte FIFO buffer shared between the three UARTS.
I would look through the UART init code to see if a TX_FIFO reset operation was preformed.

UART_CONF0_REG.UART_TXFIFO_RST = 1;
UART_CONF0_REG.UART_TXFIFO_RST = 0;

A software reset just restarts the processor, a hardware reset initializes the hardware to a know state.

It looks to me that the UART is sending old data.
Chuck.

[jestebanes]

Hello, I don't think is a problem related to USART hardware, because, if you read the history messages, I made some test and I can't found differences between the machine registers in hardware restart and software restart.
I made some code test to help to solve this trouble, the ESP32Test2.ino is running on an ESP32 duino or any other board and ESP32_testB in a classic arduino UNO. (The extensions are changed to support the .ino file) Serial2 ESP32 (pins io16 & io17) need to connect to UNO (pins io10 & io11)

ESP32Test2.ino.txt
ESP32_testB.ino.txt

If everything is ok the arduino UNO must receive the sequence 021D58C29242CA, but after the first software reset the received data is corrupted. Eventually after an indeterminate number of resets the sequence it's recovered.

My theory is about the software driver and the struct uart, that some variable is not properly initiated, but in a hardware reset this variable is forced to zero by memory behavior.

Best regards

[jestebanes]

Another detail, this behavior only responds to ESP32 Arduino UART driver, in esp-idf this issue doesn't happen. I translate the code to esp-idf and UART2 works fine after software reset.

uart_serial2_main.zip

Best regards

[nkostis]

hi @stickbreaker

It looks to me that there is stale data in the FIFO from before the reboot

i don't think it's stale data, as if you don't reboot and continue writing to serial2 everything is fine. It's only after reboot that this happens, which makes me think that this is incorrect FIFO pointers initialization (probably its memory is accessed via a circular queue and not both pointers (head and tail) are initialized?).

I would look through the UART init code to see if a TX_FIFO reset operation was preformed.

UART_CONF0_REG.UART_TXFIFO_RST = 1;
UART_CONF0_REG.UART_TXFIFO_RST = 0;

Yes, the arduino code for Flush() (i think it's named uartFlush()) does this but seems to have no effect

[shimarin]

Hello, I've found this can be relevant to this issue in esp-idf.
https://github.com/espressif/esp-idf/issues/1202
It seems there's hardware issue and recently software workaround was merged.
https://github.com/espressif/esp-idf/commit/4052803e161ba06d1cae8d36bc66dde15b3fc8c7
If what I guess is right, applying same workaround to arduino-esp32 maybe worth.

[jestebanes]

I propose a change in the Serial.flush function to avoid the trouble with serial2, is very similar to change made in uart.c in esp-idf git

esp32-hal-uart.c

void uartFlush(uart_t* uart)
{
if(uart == NULL) {
return;
}

UART_MUTEX_LOCK();
while(uart->dev->status.txfifo_cnt && uart->dev->mem_cnt_status.tx_cnt);

// uart->dev->conf0.txfifo_rst = 1;
// uart->dev->conf0.txfifo_rst = 0;

// uart->dev->conf0.rxfifo_rst = 1;
// uart->dev->conf0.rxfifo_rst = 0;
uint8_t c;
while(uart->dev->status.rxfifo_cnt && uart->dev->mem_cnt_status.rx_cnt){
READ_PERI_REG(UART_FIFO_REG(uart->num));
}
while(uxQueueMessagesWaiting(uart->queue)){
xQueueReceive(uart->queue, &c, 0);
}

UART_MUTEX_UNLOCK();

}

[PhilColbert]

Thanks for the work , I have this issue aswell...

Tried adding this code and on first boot the board comes up ok, second boot after a software restart, it just hangs.... any ideas?

[PhilColbert]

Tried to debug it to find out where its hanging, however serial fails so cant see any printf statements coming out.....

Found if I left the old

uart->dev->conf0.txfifo_rst = 1;
uart->dev->conf0.txfifo_rst = 0;

uart->dev->conf0.rxfifo_rst = 1;
uart->dev->conf0.rxfifo_rst = 0;

code in, it doesnt hang, but the serial is still in a mess :(

Not the best that serial2 is this unstable!

( the system hangs after its starts Serial and then tries to start Serial2.

Thanks

[jestebanes]

Hi Phil, I read in another issue that the reason for that behavior, is a hardware failure. It is due to a wrong connection in the internal reset of serial 1 & 2.
This is the reason why I suggest a change in the code similar to the one done in esp-idf library to avoid this reset, but that was two months ago and we are still waiting for a solution.
Regards.

[PhilColbert]

Thanks for the reply.

Serial 2 kind of works, but its very unstable, sometimes after boot it works, sometimes characters come in all over the place.

Unfortunately I need 3 UARTS in my project running at 115200 :(

Did the ESPIDF way of running the uart work ok still ? Will have to move to that :)

[jestebanes]

I think the solution will come with the version 2 of the chip because the failure is hardware related and I hope that EXPRESSIF is working to solve it, as other troubles like the inaccuracy of analog reads or the inability of analog2 channels when you are using WiFi.
ESP32 Arduino is developed as a component of ESPIDF, at this moment maybe it had some bugs but runs the same libraries than ESPIDF.
In my case moving to ESPIDF is not an option because I got thousands of lines of code from ESP8266 Arduino and I can't afford the cost of change all these stuff.
As you experience after software boots the UART2 fails but after hardware boots it runs most of the times, in my case I have designed a board that uses UART2 pins and my solution came when I change the external connection from UART2 to UART1.
Before that, I tried some stupid ideas like force a reset hardware with a transistor when I detect the failure in the port (it works pretty cool).
Question, do you need the 3 UARTS all the time? Or you need 3 UARTs but you can; connect, read one device, disconnect, read another device, etc. In the second case, you can have one permanent connection to UART0 and use UART1 to connect to the other 2 ports with the begin sentence.
Regards

[PhilColbert]

Sounds about right !

Did you get the ESPIDF code to run ok however ? I can move one of the ports to that code , thats not an issue, just checking before I go ahead.

Unfortunately I need 3 UARTS, one is for gps, one for radio board ( in and out ) and one for comms to a flight instrument ( in and out ) .

Alternatively I could add one of the ports to be software serial I guess.

Thanks

[JacoFourie]

I solved this by adding a flush before each send. All 3 my UARTS work now.

[PhilColbert]

Thanks, sends seem to be better, its reading thats the issue.... and the restart issue :(

[JacoFourie]

Mmmm maybe I also have this issue. I can not get the MODBUS library to read. I can set a register but can not read one. so maybe the read is the problem. I can send and read on 2 of the UARTS and I can send on all 3 of them. Let me test if I can receive on the 3rd one withou MODBUS.

[xinort]

I'd thought it was just me! I've been trying to figure this out for a while. Works fine with an external reset but after ESP.restart, serial2 is a mess. I'll try the flush before each send.

[JacoFourie]

OK I see the same thing on the read. That is why my MODBUS is not working. I can send. But nothing comes back into the read buffer. I can use all 3 UART to send but only 2 will receive data.

https://github.com/espressif/arduino-esp32/issues/1189

[PhilColbert]

All 3 receive data here.... update your espidf and arduino, that should solve that issue.

However, Serial 2 sometimes works ok, then after a software reboot or even sometimes on boot, characters come into the esp in a semi random order, characters missing aswell.

Would really appreciate a fix ! :)

I tried the fix recommended by jestebanes - however it hangs on boot when Serial 2 is brought up after a software reset.

Just tested this directly in code by adding a GPS to the esp32 - assigning serial 1 to the gps RX pin, datra comes in with no corruption at all, then switching to Serial 2, data is corrupted, same pins, same GPS.

Unfortunately I really need Serial2 !

Anyone ? :)

[PhilColbert]

Tried using the flush statement before every read or write, first 2 reboots it worked perfectly, and then data corruption came back on the third reboot.

Next reboot, data is ok again and then all corruption is back again....

Surely this needs looking into ASAP - serial ports are kind of fundamental to a chip ! :)

[thomasschalch]

I have a similar problem with UART2. The RX ringbuffer (and/or) FIFO behaves strange once the buffer flew over. I did not manage to get out of this condition (uart_flush and even esp_restart does not help). I found out that going to deep sleep and waking up after a second does help (i.e. esp_deep_sleep(1000*1000))!
So I try to detect this condition and do a reset using esp_deep_sleep. That's a acceptable workaround for me, since it happens very rarely. Probably you could give it a try!

[JacoFourie]

@thomasschalch did you try the workaround https://github.com/espressif/arduino-esp32/issues/1189 ?

[thomasschalch]

@JacoFourie thanks for your reply! I'm using esp-idf v3.0-rc1 without arduino libraries. The mentioned issue is already fixed there.
But I found out my issue is fixed in the commit 492db0591d0c5643cb863b76600ab0bf5a204b64 on esp-idf master. The UART FIFO was not reset properly when an overflow occurred. Using this commit my UART2 seems to work perfectly now!
Maybe this bugfix can also help fixing your issues with UART2?

[JacoFourie]

Yes in the workaround we use the code from the ESPIDF fix and apply it to the wrapper.

[9a4gl]

The above modification in esp32-hal-uart.c didn't help me :(

I had the same issue and I read somewhere that using ESP-IDF uart API works fine. So, I tried to implement my HardwareSerial using ESP-IDF API and this really works, you can do ESP.restart() and UART2 works after software reset.

I pushed my implementation called ESP32Serial here https://github.com/9a4gl/ESP32Serial