Arduino-esp32: ESP32 HTTP Server request interrupts Telnet session

Created on 8 Oct 2020 · 24Comments · Source: espressif/arduino-esp32

I have an ESP32 application that uses a web server as a user interface. For auditing and debugging during development I use Serial; however, that is impractical in the installed system. So, I decided to use Telnet (TCP/IP) to monitor what is happening when Serial is impractical.

The issue is that once a TCP/IP session has been established, when a HTTP request is received, the TCP/IP session is aborted.

I have reproduced the issue in a minimal sketch here:

#include "Arduino.h"
#include <WiFi.h>
#include <WiFiClient.h>
#include <WebServer.h>

WiFiClient telnetClient;
WebServer server(80);
WiFiServer telnetServer(23);

void telnetPrint(const char* text)
{
  if (telnetClient)
  {
    telnetClient.print(text);
  }
}

void telnetPrintln(const char *text)
{
  telnetPrint(text);
  telnetPrint("\r\n");
}

void _print(const char* text)
{
  Serial.print(text);
  telnetPrint(text);
}

void _println(const char* text)
{
  Serial.println(text);
  telnetPrintln(text);
}

void _println(IPAddress myIP)
{
  char rgIPTxt[32];
  sprintf(rgIPTxt,"%u.%u.%u.%u",myIP[0],myIP[1],myIP[2],myIP[3]);
  _println(rgIPTxt);
}

const char *rootFmt="\
<html>\
  <head>\
    <title>Test</title>\
  </head>\
  <body>\
    <h1>Test</h1>\
    <div>\
      <h2>Heap size=%d<br><br>\
    </div>\
  </body>\
</html>\
";

char rootMessage[1024];

void updateRootMessage()
{
  _println("updateRoot");
  snprintf(rootMessage, sizeof(rootMessage), rootFmt,
           heap_caps_get_free_size(MALLOC_CAP_8BIT)
           );
}

void handleRoot() 
{
  _println("Entered handleRoot");
  updateRootMessage();
  server.send(200, "text/html", rootMessage);
  _println("Leaving handleRoot");
}

void wifiSTASetup(const char*ssid, const char*password)
{
  WiFi.mode(WIFI_AP_STA);
  WiFi.begin(ssid, password);
  telnetServer.begin();
  telnetClient=telnetServer.available();
  _println("");

  // Wait for connection
  while (WiFi.status() != WL_CONNECTED) 
  {
    delay(500);
    _print(".");
  }
  _println("");
  _print("Connected to ");
  _println(ssid);
  _print("IP address: ");
  _println(WiFi.localIP());
  server.on("/", handleRoot);
  server.begin();
}

#define BAND    915E6

void setup() 
{
  Serial.begin(115200);
  wifiSTASetup("yourssid", "yourpass");
}

void loop() 
{
  server.handleClient();
  if (!telnetClient)
  {
    telnetClient=telnetServer.available();
    if (telnetClient)
    {
      _println("telnetClient obtained");
    }
  }

}

This is the observed behavior:

When this sketch is running, before establishing a Telnet session, HTTP requests are processed as expected and Serial output is as expected. When a Telnet session is established, "telnetClient obtained" appears on the terminal (and via Serial) but when an HTTP request is processed, the Telnet terminal receives "Entered handleRoot" and MAY receive "updateRoot" and "Leaving handleRoot" but then the telnet session is aborted.

My actual application logs quite a bit of information, reacting to messages received via LoRa and on I/O pins. Everything there works as expected until an HTTP request is served which results in the connection abort.

stale

Source

evortriede

Most helpful comment

WebServer is really not designed to persist sessions. I don't know exactly why it is disconnecting (perhaps you are starving the connection and it times out?), but I would strongly recommend you use EspAsyncWebServer if you intend to have multiple services running.

It is not the WebServer connection (which is transient by its nature) that is aborting, but the Telnet (raw TCP/IP) session which should be persistent, that is aborting. It is established once within the loop method and should persist indefinitely. In the example sketch that I attached, there is virtually no activity over this connection; however, in the actual application, there is a great deal of uninterrupted traffic which is terminated by 1) closing the Telnet terminal or 2) by sending a HTTP request to port 80. To my mind, sending a request to port 80 should not interfere with the connection on port 23, but it does. That is the issue.

evortriede on 10 Oct 2020

👍2

All 24 comments

I think you need to use server.hasClient() for persistent tcp connections. See https://github.com/AlphaLima/ESP32-Serial-Bridge/blob/master/ESP32-Serial-Bridge.ino and https://github.com/lasselukkari/aWOT/blob/master/examples/KeepAlive/KeepAlive.ino for examples.

lasselukkari on 9 Oct 2020

I think you need to use server.hasClient() for persistent tcp connections. See https://github.com/AlphaLima/ESP32-Serial-Bridge/blob/master/ESP32-Serial-Bridge.ino and https://github.com/lasselukkari/aWOT/blob/master/examples/KeepAlive/KeepAlive.ino for examples.

no. hasClient() only tells you if available() will return a connected client or a client object which evaluates to false.

jandrassy on 9 Oct 2020

I may have phrased my reply wrong. What I mean that a working solution will require you to use hasClient() to make difference between new connections and the existing ones. Maybe this has nothing to do with the actual problem here. I think the connection should stay open as long as the destructor is not being called.

lasselukkari on 9 Oct 2020

WebServer is really not designed to persist sessions. I don't know exactly why it is disconnecting (perhaps you are starving the connection and it times out?), but I would strongly recommend you use EspAsyncWebServer if you intend to have multiple services running.

lbernstone on 9 Oct 2020

WebServer is really not designed to persist sessions. I don't know exactly why it is disconnecting (perhaps you are starving the connection and it times out?), but I would strongly recommend you use EspAsyncWebServer if you intend to have multiple services running.

evortriede on 10 Oct 2020

👍2

So here is modified example of the most simple WifiServer example that comes with the board. I have added another server to port 23 and there is no trouble maintaining the connection. Does this work for you?

/*
 WiFi Web Server LED Blink

 A simple web server that lets you blink an LED via the web.
 This sketch will print the IP address of your WiFi Shield (once connected)
 to the Serial monitor. From there, you can open that address in a web browser
 to turn on and off the LED on pin 5.

 If the IP address of your shield is yourAddress:
 http://yourAddress/H turns the LED on
 http://yourAddress/L turns it off

 This example is written for a network using WPA encryption. For
 WEP or WPA, change the Wifi.begin() call accordingly.

 Circuit:
 * WiFi shield attached
 * LED attached to pin 5

 created for arduino 25 Nov 2012
 by Tom Igoe

ported for sparkfun esp32 
31.01.2017 by Jan Hendrik Berlin

 */

#include <WiFi.h>

const char* ssid     = "";
const char* password = "";

WiFiServer server(80);

WiFiServer telnetServer(23);
WiFiClient telnetClient;

void setup()
{
    Serial.begin(115200);
    pinMode(5, OUTPUT);      // set the LED pin mode

    delay(10);

    // We start by connecting to a WiFi network

    Serial.println();
    Serial.println();
    Serial.print("Connecting to ");
    Serial.println(ssid);

    WiFi.begin(ssid, password);

    while (WiFi.status() != WL_CONNECTED) {
        delay(500);
        Serial.print(".");
    }

    Serial.println("");
    Serial.println("WiFi connected.");
    Serial.println("IP address: ");
    Serial.println(WiFi.localIP());

    server.begin();

    telnetServer.begin();
}

int value = 0;

void loop(){
 WiFiClient client = server.available();   // listen for incoming clients

  if (client) {                             // if you get a client,
    Serial.println("New Client.");           // print a message out the serial port
    String currentLine = "";                // make a String to hold incoming data from the client
    while (client.connected()) {            // loop while the client's connected
      if (client.available()) {             // if there's bytes to read from the client,
        char c = client.read();             // read a byte, then
        Serial.write(c);                    // print it out the serial monitor
        if (c == '\n') {                    // if the byte is a newline character

          // if the current line is blank, you got two newline characters in a row.
          // that's the end of the client HTTP request, so send a response:
          if (currentLine.length() == 0) {
            // HTTP headers always start with a response code (e.g. HTTP/1.1 200 OK)
            // and a content-type so the client knows what's coming, then a blank line:
            client.println("HTTP/1.1 200 OK");
            client.println("Content-type:text/html");
            client.println();

            // the content of the HTTP response follows the header:
            client.print("Click <a href=\"/H\">here</a> to turn the LED on pin 5 on.<br>");
            client.print("Click <a href=\"/L\">here</a> to turn the LED on pin 5 off.<br>");

            // The HTTP response ends with another blank line:
            client.println();
            // break out of the while loop:
            break;
          } else {    // if you got a newline, then clear currentLine:
            currentLine = "";
          }
        } else if (c != '\r') {  // if you got anything else but a carriage return character,
          currentLine += c;      // add it to the end of the currentLine
        }

        // Check to see if the client request was "GET /H" or "GET /L":
        if (currentLine.endsWith("GET /H")) {
          digitalWrite(5, HIGH);               // GET /H turns the LED on
        }
        if (currentLine.endsWith("GET /L")) {
          digitalWrite(5, LOW);                // GET /L turns the LED off
        }
      }
    }
    // close the connection:
    client.stop();
    Serial.println("Client Disconnected.");
  }

  if (telnetServer.hasClient()) {
    if(telnetClient) {
      telnetClient.stop();
    }

    telnetClient = telnetServer.available();
  }

  if (telnetClient.available()) {
    Serial.write(telnetClient.read());   
  }
}

lasselukkari on 10 Oct 2020

This is strange. I tested my own sketch again. After a few requests the connection became unresponsive. What I mean is that no data is printed anymore although the telnet client does not detect the connection lost until i resetted the ESP.

lasselukkari on 10 Oct 2020

You should be using WiFi.setSleep(false) on a "server"

lbernstone on 10 Oct 2020

WiFi.setSleep(false) did not change the behaviour for me. Also if after a reset I make a few requests with the a browser to the http port first and then create the connection to the port 23 after that the telnetServer.hasClient() never returns true. Unfortunately I do not have time to debug this further at the moment.

lasselukkari on 10 Oct 2020

Ahh silly me. In the sketch the http server stays in loop until the connection is lost. Chrome opens up a new TCP connection automatically and leaves it open for the next request and the main loop is stuck there. I need to modify the example sketch a bit more.

lasselukkari on 10 Oct 2020

I recreated the example using my own HTTP server library. It seems to work as expected. There is no trouble maintaining the connection.

#if defined(ESP8266)
#include <ESP8266WiFi.h>
#else
#include <WiFi.h>
#endif
#include <aWOT.h>

#define WIFI_SSID ""
#define WIFI_PASSWORD ""

WiFiServer server(80);
WiFiServer telnetServer(23);
WiFiClient telnetClient;
Application app;

void index(Request &req, Response &res) {
  res.print("Hello World!");
}

void setup() {
  Serial.begin(115200);

  WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println(WiFi.localIP());

  app.get("/", &index);
  server.begin();
  telnetServer.begin();
}

void loop() {  
  WiFiClient client = server.available();

  if (client.connected()) {
    app.process(&client);
  }

  if (telnetServer.hasClient()) {
    if(telnetClient) {
      telnetClient.stop();
    }

    telnetClient = telnetServer.available();
  }

  if (telnetClient.available()) {
    Serial.write(telnetClient.read());   
  }
}

lasselukkari on 10 Oct 2020

I recreated the example using my own HTTP server library. It seems to work as expected. There is no trouble maintaining the connection.

but the issue is with WebServer library

jandrassy on 10 Oct 2020

but the issue is with WebServer library

How do you know?

lasselukkari on 10 Oct 2020

Now that I finally ran the the original sketch I'm unable to reproduce the issue with even that. I maybe should have done that in the first place...

lasselukkari on 10 Oct 2020

Now that I finally ran the the original sketch I'm unable to reproduce the issue with even that. I maybe should have done that in the first place...

What hardware platform are you using and which libraries? I am using Heltec Automation WiFi LoRa 32(v2) with the Heltec supplied libraries (which appear to be current, but I'm not sure).

evortriede on 10 Oct 2020

I recreated the example using my own HTTP server library. It seems to work as expected. There is no trouble maintaining the connection.

I suspect that you are not understanding the problem since your sketch closes the Telnet connection in the loop if it exists. The idea is to establish the Telnet connection and keep it indefinitely while serving any HTTP requests that come in. The behavior that I observe is that the Telnet connection, once established, remains in place until an HTTP request is served at which point the Telnet connection is aborted.

If you are not seeing the same behavior when you run my sketch, I would assume that it is an issue that has been addressed or introduced between the versions of the libraries that we are using.

evortriede on 10 Oct 2020

I recreated the example using my own HTTP server library. It seems to work as expected. There is no trouble maintaining the connection.

I suspect that you are not understanding the problem since your sketch closes the Telnet connection in the loop if it exists. The idea is to establish the Telnet connection and keep it indefinitely while serving any HTTP requests that come in. The behavior that I observe is that the Telnet connection, once established, remains in place until an HTTP request is served at which point the Telnet connection is aborted.

If you are not seeing the same behavior when you run my sketch, I would assume that it is an issue that has been addressed or introduced between the versions of the libraries that we are using.

It only closes the connection if a new client is connected. It's only done for the sake of being able to open "hijack" an existing connection. If there is only one client the connection is kept alive.

I have ESP32 core v 1.0.4 compiled with arduino ide 1.8.12. I was using the MH-ET LIVE D1 board with the ESP32 Dev Module config.

lasselukkari on 10 Oct 2020

I'm using Arduino 1.8.13... as stated earlier, I am not sure what vintage the Heltec libraries are. (How would I determine that, trivially?)

I have modified my example sketch to more accurately reflect the dynamics of the actual application - specifically by creating output every 5 seconds. Interestingly, the behavior seems to have changed slightly in that upon the first HTTP request after the Telnet connection is established, the output to the terminal stops, but the terminal (I'm using putty) does not report the connection being closed. All subsequent Telnet connections are immediately closed upon an HTTP request being served. (i.e., putty reports "connection closed by remote host")

Here's the new sketch:

#include "Arduino.h"
#include <WiFi.h>
#include <WiFiClient.h>
#include <WebServer.h>

WiFiClient telnetClient;
WebServer server(80);
WiFiServer telnetServer(23);

void telnetPrint(const char* text)
{
  if (telnetClient)
  {
    telnetClient.print(text);
  }
}

void telnetPrintln(const char *text)
{
  telnetPrint(text);
  telnetPrint("\r\n");
}

void _print(const char* text)
{
  Serial.print(text);
  telnetPrint(text);
}

void _println(const char* text)
{
  Serial.println(text);
  telnetPrintln(text);
}

void _println(IPAddress myIP)
{
  char rgIPTxt[32];
  sprintf(rgIPTxt,"%u.%u.%u.%u",myIP[0],myIP[1],myIP[2],myIP[3]);
  _println(rgIPTxt);
}

const char *rootFmt="\
<html>\
  <head>\
    <title>Test</title>\
  </head>\
  <body>\
    <h1>Test</h1>\
    <div>\
      <h2>Heap size=%d<br><br>\
    </div>\
  </body>\
</html>\
";

char rootMessage[1024];

void updateRootMessage()
{
  _println("updateRoot");
  snprintf(rootMessage, sizeof(rootMessage), rootFmt,
           heap_caps_get_free_size(MALLOC_CAP_8BIT)
           );
}

void handleRoot() 
{
  _println("Entered handleRoot");
  updateRootMessage();
  server.send(200, "text/html", rootMessage);
  _println("Leaving handleRoot");
}

void wifiSTASetup(const char*ssid, const char*password)
{
  WiFi.mode(WIFI_AP_STA);
  WiFi.begin(ssid, password);
  telnetServer.begin();
  telnetClient=telnetServer.available();
  _println("");

  // Wait for connection
  while (WiFi.status() != WL_CONNECTED) 
  {
    delay(500);
    _print(".");
  }
  _println("");
  _print("Connected to ");
  _println(ssid);
  _print("IP address: ");
  _println(WiFi.localIP());
  server.on("/", handleRoot);
  server.begin();
}

#define BAND    915E6

void setup() 
{
  Serial.begin(115200);
  wifiSTASetup("ssid", "pass");
}

void loop() 
{
  server.handleClient();
  if (!telnetClient)
  {
    telnetClient=telnetServer.available();
    if (telnetClient)
    {
      _println("telnetClient obtained");
    }
  }
  if ((millis() % 5000) == 0)
  {
    char msg[256];
    sprintf(msg, "heap available = %d",
            heap_caps_get_free_size(MALLOC_CAP_8BIT));
    _println(msg);
    delay(1);
  }
}

So... I just ran it again and the first HTTP request closed the connection as before. So the behavior is somewhat dynamic rather than entirely deterministic.

evortriede on 10 Oct 2020

After playing around a bit more I noticed that the behaviour you describe happens if I first send any data from the telnet client. First everyting works as expected but after sending any data from the client the connection gets closed when the next HTTP request comes in.

If I add a read() call to the loop like this it keeps on working as expected.

void loop() 
{
  server.handleClient();
  if (!telnetClient)
  {
    telnetClient=telnetServer.available();
    if (telnetClient)
    {
      _println("telnetClient obtained");
    }
  } else {
    telnetClient.read();
  }

}

lasselukkari on 10 Oct 2020

👍1

If I add a read() call to the loop like this it keeps on working as expected.

lasselukkari - you have nailed it!!!! Thank you very much!!!

evortriede on 10 Oct 2020

but it is still a bug

jandrassy on 11 Oct 2020

but it is still a bug

I agree. The same code on ESP8266 works as expected without the need to call read.

lasselukkari on 11 Oct 2020

So, I'll re-open it.

evortriede on 11 Oct 2020

[STALE_SET] This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.