Espeasy: Core 2.4.1 vs 2.4.2 What are the related issues?

[TD-er]

The last 2 weeks, the nightly builds are based again on:

• Core 2.4.1
• LWIP 1.4

The builds before that were based on:

• Core 2.4.2
• LWIP 2.0 low memory

Major changes from 2.4.1 => 2.4.2:

• Arduino cont stack overlaps system stack by 4 kB, freeing 4 kB of user memory. ( https://github.com/esp8266/Arduino/pull/4622 )
• Move lots of strings to flash, saving RAM.
• I2C: improve bus recovery procedure
• numerous WiFi bug fixes.
• Improve hardware serial
• New waveform generation code supporting multiple analogWrite, Servo, Tone at the same time (https://github.com/esp8266/Arduino/pull/4640, https://github.com/esp8266/Arduino/pull/4872)
• Add ESP.deepSleepMax() function to get maximum possible deep sleep time ( https://github.com/esp8266/Arduino/pull/4627 )

Possible breaking changes:
Overlap of stacks between system and Arduino cont:

• We define a number of rather large objects on the system stack.
• It is not entirely sure this behavior will introduce other stability issues.
• Large objects could be moved to the heap when in doubt.

New waveform generation code:

• (#1653) changes stability of PWM signal.

Issues fixed in 2.4.2 and broken in 2.4.1:

• (#1882) since 20181004 NeoPixel SK6812/WS2812B pixel errors (2.4.1 vs 2.4.2)
• (#1642) Restarts due to notification LED [mega-20180809]
• More available free memory makes the unit faster and perhaps less likely to crash due to allocation errors.

Are there any other issues which may help in deciding what version of the core library to use?
It is also possible to make different choices for different builds.
For example, for typical Sonoff builds (1M flash) it may be useful to have the extra memory. Although 2.4.2 builds may be bigger in size making it harder to perform OTA.

[v-a-d-e-r]

For me the best combination is core 2.4.2 and LwIP 1.4.0-RC2. Running stable on 20 different nodes. Current core 2.4.2+ from Git seems also running good, but takes 3-4% more CPU load with the same config as with core 2.4.2. :-/ All with current ESPeasy from Git. LwIP 2.0 and even 2.1 seem to respond slower on HTTP/UDP as 1.4.

[s0170071]

Thank you @TD-er to sum up the pros and cons of each core version. I think this is a more fundamental question which poses itself again whenever there is a new core. Migrating to a new version is a big step and of course means coping with new bugs. But on the other hand, new versions often have old bugs sorted out.

For those who don't know: LWIP is the LightWeight IP stack, a library which handles all TCP/IP wifi related stuff. It is compiles along with the core. Arduino IDE users can change it using the drop down menu. Core is the Arduino hardware abstraction layer for the ESP8266. You change that using the boards manager.

In my opinion there are three ways to go about it:

A: stick to a known version, i.e. 2.4.1 and LWIP 1.4

Both versions are obsolete. The advantage is that the bugs are known and often you can deal with them (using workarounds). The down side is that you don't benefit from any improvements. (LWIP 2 is way faster, core 2.4.2 brings more free memory (=heap)
Using this option makes only sense if we plan on skipping core 2.4.2 totally and maybe some more. Question is: how long can you avoid migrating to a current version?

B: using latest stable core (2.4.2 ) and latest stable LWIP (v2)

You already guessed it ;-) its faster and gives more memory. And other improvements. Of course they introduced new bugs too, like the PWM issue.
Unless there is no way around current core issues, things won't work, e.g. folks who depend on the PWM and can't figure out a fix have to use an older ESPEasy version. For now at least. But folks who want to use new features like e.g. endless serial buffers can get started...

C: mix versions every other day

Not to get people confused of course, but to distribute available bugs more evenly :-) And the devs can play a funny core version guessing game whenever there is a new issue. When core version 42 is available we finally use it for good. It should have everything sorted out- per definition.

[TD-er]

Do you know any other issue or use case which cannot run on either 2.4.1 or 2.4.2?
Just to help finding a good compromise in deciding what version to use (now) for what application.

For example, typical use can be plugin X or use case Y => use version Z.

[Grovkillen]

What I think could be a way forward is to stick with 2.4.1 and be able to release a proper stable version (finally! 😄) and straight after that release we could bump the core and start digging from there. I reckon keeping on bumping dependencies will make the release "forever" in front of us. Just my top of the mind feeling.

[ascillato]

Hi,

In my experience, core 2.4.2 is way more stable than 2.4.1.

[TD-er]

Hi,

In my experience, core 2.4.2 is way more stable than 2.4.1.

And what LWIP version?

[s0170071]

Shouldn't we re-evaluate the stability now that the stack seems fixed?

[TD-er]

@s0170071 Agreed, but testing it on a few nodes will not give the full picture.
And thus this is also a question whether we should make a few builds also using core 2.4.2?

I am running one node here on 2.4.2/LWIP1.4 and I already saw a HW watchdog reboot.

[ascillato]

With 2.4.2 lwip 2 we had some issues with a mqtt library but now we are using the new pubsubclient and lwip 2 hb is working great.

We found less issues with wifi repeaters with this lwip2.

Now, also the actual stage core is working fine with lwip2. They will release 2.5.0 in december. The great thing in this new core is that light_modem_sleep works fine. No issues.

[s0170071]

December? How about we release two versions in December? One 2.4.1/lwip1.4 and one 2.5.0/lwip2.0

[amigatommy]

I think too, we should not go backwards with core and lwip 2. Usually the plugin developer, in my case the "Neopixel plugin", should change something on his code to get it work. I think, he only has to change some timing relevant variables, maybe different values for each platform (ESP8266 / ESP32 ...) too.
Lets see, what happens with core2.5.0 ... ;o)

[micropet]

I am not a developer, but I like to try the different possibilities.

I've been using Core 2.4.2 LWIP 1.4.0-RC2 and 160 MHz for a couple of weeks, compiled with PlatformIO.

My units are now running for more than 14 days without reboot.
Before that, they boot every few hours.

Peter

[TD-er]

@ascillato What version of PubSubClient do you now use? You said: "but now we are using the new pubsubclient" Is it different from the one included in ESPeasy?

I had yesterday one of my nodes running core 2.4.2 + LWIP 1.4 and that's a big no-go combination.
Max uptime on that node was 140 minutes. All "Hardware Watchdog" reboots.
The same source, compiled with core 2.4.2 and LWIP 2.0 high bandwidth and still not a single reboot (9+ hours)

[ascillato]

@TD-er

What version of PubSubClient do you now use?
The last from ESPEasy

We had no watchdog issues with core 2.4.2 in any lwIP. May be you have a library with conflict with that core. We had one old mqtt library that were using espconn.h that also is in the core and that make a conflict with lwip 2 hb but with lwip 1,4 was working fine. We replace that library and all problems gone.

The core 2.4.2 is stable in all lwip, so you should look for a conflicting library IMHO.

[s0170071]

@TD-er can you pls test your no-go binary with (in this order) :
1) another power supply
2) previous power supply but a factory reset
3) another ESP module ?

[TD-er]

@s0170071 Both tests were run on the same ESP, same settings and same power supply.
My test boards run from a 25cm quality USB cable off an Anker 60 Watt USB3 hub.

I will be away most of today, but later this evening I will test with a clean install.

[ascillato]

@TD-er

The guys from the esp8266 core are going to change the webserver parsing API for v2.5.0. They are proposing some changes. May be you are also interested on them (https://github.com/esp8266/Arduino/issues/5261#issuecomment-432187111)

[arendst]

@TD-er the PubSubClient version now used with Tasmota is yours! ;-)

[d-a-v]

@s0170071
Is there an opened related PWM issue in the arduino core repository ?

[TD-er]

@arendst Bedankt Theo :)
@ascillato I will have a look at it later this evening.
@d-a-v Welcome to the discussion here ;)

[s0170071]

@d-a-v Somebody was experiencing a unstable PWM in core 2.4.2. I guess its caused by https://github.com/esp8266/Arduino/issues/5140

[TD-er]

Suggestion:
For the 1M builds, we should go back to core 2.4.2 and LWIP2.0
Then we still have to decide if it is going to be LWIP low memory or high bandwidth.

Any pro/con arguments about this?

[s0170071]

@TD-er did you do the test? https://github.com/letscontrolit/ESPEasy/issues/1931#issuecomment-432137907 ?

[TD-er]

@s0170071 No not yet a clean install.

[s0170071]

latest ESPEasy and latest core with v3 NonOS SDK. No plugins active --> 22.2k Ram free
:-)

@TD-er how's your test coming ?

[uzi18]

@d-a-v @s0170071 I think PWM is related. to https://github.com/esp8266/Arduino/pull/5248 and #1653

[TD-er]

Left half is the same config with core 2.4.1, right half with core 2.4.2
The reason of the reboot of core 2.4.2 was due to USB hub port reset, not a HW watchdog reset.

Both running LWIP 1.4.0.
So the current version running on a config which used to give a lot of reboots is running:
ESP82xx Core 2_4_2, NONOS SDK 2.2.1(cfd48f3), LWIP: 1.4.0-RC2
And the strange thing is, this was first my "big no go" combo.

So what has changed? Not the config, not the USB cable and not the plugged in sensors.
The only thing changed was:

• Upgrade PIO
• Complete clean before build and building it twice, just to be sure.

[TD-er]

Anyone objects when I switch all builds to 2.4.2?
And add a 1M and 4M build with the "normal" plugin set for those in need of PWM.

Is there a plugin needing PWM not in the "normal" list?

[uzi18]

@TD-er and what about 3.0, any test?

Also have found "Complete clean before build" resolves lots of troubles, PIO bug?

[Grovkillen]

@TD-er we're good to go with the normal plugin set for the 2.4.1 builds. Close and go?

[TD-er]

I will add the core 2.4.2 as default and 2 "normal" builds based on core 2.4.1 for those needing PWM.

• normal_core_241_ESP8266_1024
• normal_core_241_ESP8266_4096

@uzi18 The 3.0 is included in "stage" ?

I think for now, we should try to get a stable release out and then focus on newer core library

[uzi18]

@TD-er don't know, @s0170071 mention it above

[ascillato]

The SDK 3.0.0 you meant? Yes, it is included in the actual STAGE version

[TD-er]

@ascillato Do you have any preference (or good arguments) to use LWIP 1.4 or 2.0 (high bandwidth/low mem) ?

[ascillato]

Mainly The extra free memory. lwip 2 HB has more free memory and also shows the webpage in Tasmota faster.

[TD-er]

Hmm, I would expect "Low memory" to use less memory ;)
I will select High Bandwidth as default setting for now.

[ascillato]

I also think that it is a good idea to get a stable version and then moving to the next core.

The actual Stage core has some things that are different like the webserver argument parsing.

[ascillato]

We have to make some code adaptation mainly for Alexa

[Grovkillen]

I close this discussion