Espeasy: OpenHAB MQTT Trouble

[Fusseldieb]

Hi,
I'm using Home Assistant and saw that the OpenHAB MQTT option works the best, but I just found out that he's the source of trouble, or at least seems like it.

I experience random NodeMCU freezes. The MQTT no longer updates and when navigating with Chrome to it, it simply says that "The connection has timed out". Be noted that this isn't a new issue, I experience this since I first knew ESPEasy some months ago.

This sometimes happens if I try to restart Home Assistant, and sometimes when it is normally running. I need to pull it from the power and put it back in. It's quite unreliable.

Our router is a powerful NETGEAR Nighthawk running OpenWRT, so the problem definitively doesn't rely there. I never had to restart it since I put OpenWRT on it, it works super reliable. The NodeMCU connection to the router is also good (-72 dB).

I'm using the build from yesterday (ESP_Easy_mega-20181203_normal_ESP8266_4096.bin).

I don't know why this is happening. It is quite random.

If any additional info is needed, just let me know.

Thanks in advance.

[Grovkillen]

What broker are you running? OpenWRT + Mosquitto or on a separate server?

[Fusseldieb]

@Grovkillen I'm running Mosquitto (4.0) on Home Assistant. But since earlier versions this bug exists.

It seems like it's an issue with ESPEasy, other devices didn't crash or stop responding.

[TD-er]

It is an issue which has been reported by several other users also.
Not with the exact same symptoms, but in general this is what happens:

• Hardware/Software watchdog crash when data is either lost or will never arrive during connection setup.

This can have several causes:

• Poor WiFi reception
• Slow or offline server
• Busy WiFi network
• Lots of data to be sent

[workgroupengineering]

I my little experience with Mosquitto , I have many MQTT issue with the ESPEasy and NodeMCU LUA firmware.
I solved, replacing Mosquitto with emqttd.

[TD-er]

I have Mosquitto working here on a R'pi running a standard Domoticz image.
It is running fine here.
So maybe there are some differences on how MQTT is implemented in the controllers?
What controller are you running?
Some controllers need to send stuff a value at a time, so that may impose a lot more stress on the network layer in the ESP.
That's also why I added the delay option between messages in the controller settings.

[Fusseldieb]

@TD-er I'm using following settings

I even increased the time between data, but it still crashes (a few times _a day_)
It's definitively not sending a lot of data.
The only sensor connected is a Water Pressure sensor through an analog port and that's it.
The connection isn't bad either.

[TD-er]

Why such a large minimum send interval?
The default is 100 msec, but almost all local running mqtt brokers can handle 10 - 20 msec just fine.
Longer interval means more memory usage.

[Fusseldieb]

@TD-er I just experimented out with random values. All of them crash equally.

I've restarted the ESP 20 minutes ago, it just stopped responding again. Sigh :(

[TD-er]

And without MQTT enabled the wifi stability is OK?
No WiFi reconnects?

[Fusseldieb]

@TD-er I've started a proxy to see how the Webpage fetches the sensors, since I found nothing documented. (https://www.letscontrolit.com/wiki/index.php/ESP_Easy_web_interface#JSON_page_.28hidden_prior_to_version_2.0.2B.29)
Now I'll use that JSON and feed it into Home Assistant directly without using MQTT (Disabling the MQTT controller) and will report back when it'll crash again, but it probably won't since I'm almost sure that MQTT is the culprit.

Please leave this issue open. I'll report back if:

• It crashes again
• It makes through 7 days without crashing

Thanks

EDIT: For those wanting to do the same with Home Assistant (hass.io), I've made following configuration:

sensor:
  - platform: rest
    resource: http://192.168.1.253/json?view=sensorupdate
    name: Water Pressure
    value_template: '{{ value_json.Sensors[2].TaskValues[0].Value }}' # Sensors[X] is the sensor index as seen in the frontend page
    scan_interval: 1 # Not sure if this is supported by the RESTful sensor, but putting it there anyways

[Fusseldieb]

I think waiting for 7 days is a little overkill, since it has already proved that it works now.

I've been using the JSON interface since that day and it didn't crash once. It's been running longer than _any_ of my MQTT attempts.

(Now it's fetching the JSON every 1 second)

@TD-er The problem is indeed the ESPEasy's MQTT.

[Fusseldieb]

Bump. @TD-er @Grovkillen Sorry to bother, but was there any change made to fix this issue? I would love to test MQTT again (Currently fetching over JSON, it's running over a month now).

[TD-er]

There have not been changes in MQTT, but there have been some minor changes in how WiFi reconnects are being dealt with.
Also builds based on core 2.5.0 could also have some fixes with respect to WiFi and the network layer.

[Fusseldieb]

@TD-er Then the bug probably persists, because the WiFi is stable here on build "20181203_normal".

As you can see, it now ran a good time fetching with JSON without crashing or loosing connection (on build mentioned above):


_(7 days sensor graph)_

But I'll try with MQTT on the latest build again, maybe... who knows...

I'll report back.

[TD-er]

@Fusseldieb Maybe you can also try to increase the MQTT buffer size (PubSubClient.h line 26)
Change it to:

#define MQTT_MAX_PACKET_SIZE 1024 

[Fusseldieb]

@TD-er Is there any guide on how to compile it myself after modifying this variable? What do I need to have/make?
(This same issue also prevents me from actively contributing to changes. I don't see a guide and I don't know how to start... I have knoweledge in Arduino, VSCode, Git and stuff, but well, every project is unique)

I will then report back with both original and the modified version, if anything changed to better...

[TD-er]

What OS are you running on?

[Fusseldieb]

@TD-er It seems like the MQTT bug is gone... I am using the build from 10/Jan/19 and it's now running for almost 3 days without any problem. I will report further later...

What OS are you running on?

Windows 10 x64

[TD-er]

You did increase the buffer size?

[Fusseldieb]

You did increase the buffer size?

@TD-er Actually no, I just downloaded the latest build mentioned above and it seems to have solved the problem. I still can't say for sure because it has not yet been running a long time and I have only been testing MQTT on one device out of two.

EDIT 10 days later: Still running fine with that one device, will try to enable MQTT on my other devices with ESPEasy also and report back 🥇

[Misiu]

@Fusseldieb any updates? How the devices are working? I need to build a simple sensor on Wemos and I want to use ESPEasy to send data to openHAB.

[Fusseldieb]

any updates? How the devices are working?

Yes, the devices are still in use and the problems seem to have gone. I'm polling them over MQTT with HASSIO since weeks and I had zero problems until now.

[TD-er]

Just to be clear, what build version are you now running?

[Fusseldieb]

Just to be clear, what build version are you now running?

ESP_Easy_mega-20190212_normal_ESP8266_4096.bin (Aka build 20103)