Espeasy: Analog Input - internal returns 0 first time after Power on / Reboot

[racer73]

Using the "ESP_Easy_mega-20190928_normal_ESP8266_4M1M.bin"

I have experianced that the "Analog Input - Internal" device always returns 0
the first time after a Reboot / Power on situation.

So when using Deep Sleep the returned value is always 0 from the ADC

The Analog Input is used to monitor the battery used to power the ESP.

The Internal sensor is setup to check every 60 seconds

The issue i fully repeatable over several devices, that have been running "mega-20180809" before without any issues.

I have tried full "Reset" of the device after updating the ESP Easy to start clean but the issue is the same.

Log:
rllœž|Œlà|Œlìb|Ž‚ì’r’bŒbŒònnžlnnœâìbplblrlpònà‚lŒœbnâ|lìŽbŒònnîlŒŽl<STX><DC2><DC2>nn<FF>lnrŽ’’nbœlr’’nbœlœŒllìŒl`ü‚nœªU54 : INIT : Booting version: mega-20190928 (ESP82xx Core 2_5_2, NONOS SDK 2.2.1(cfd48f3), LWIP: 2.1.2 PUYA support)
55 : INIT : Free RAM:33376
56 : INIT : Cold Boot - Restart Reason: External System
58 : FS : Mounting...
82 : FS : Mount successful, used 75802 bytes of 957314
467 : CRC : program checksum ...OK
502 : CRC : SecuritySettings CRC ...OK
609 : INIT : Free RAM:30264
610 : INIT : I2C
611 : INIT : SPI not enabled
650 : INFO : Plugins: 46 [Normal] (ESP82xx Core 2_5_2, NONOS SDK 2.2.1(cfd48f3), LWIP: 2.1.2 PUYA support)
755 : WIFI : Set WiFi to STA
788 : WIFI : Connecting Krusagarden attempt #0
789 : IP : Static IP : xxx.xxx.x.xx GW: xxx.xxx.x.xxx SN: xxx.xxx.xxx.x DNS: xxx.xxx.x.xxx
1031 : ADC : Analog value: 0 = 0.000
1036 : Domoticz: Sensortype: 1 idx: xxx values: 0.00
1317 : Domoticz: Sensortype: 2 idx: xxx values: 24.13;57.50;1
2788 : WD : Uptime 0 ConnectFailures 0 FreeMem 26304 WiFiStatus 6
4755 : WIFI : Connected! AP: xxxxxxxxxxx (xx:xx:xx:xx:xx:xx) Ch: 11 Duration: 3768 ms
4757 : WIFI : Static IP: xxx.xxx.x.xx (Garage-0) GW: xxx.xxx.x.xxx SN: xxx.xxx.xxx.x duration: 197 ms
4763 : Webserver: start
4765 : firstLoopConnectionsEstablished
32127 : WD : Uptime 1 ConnectFailures 0 FreeMem 22512 WiFiStatus 3
60722 : ADC : Analog value: 322 = 322.000
60726 : Domoticz: Sensortype: 1 idx: xxx values: 3.90
60932 : Domoticz: Sensortype: 2 idx: xxx values: 24.18;57.70;1
62126 : WD : Uptime 1 ConnectFailures 0 FreeMem 22512 WiFiStatus 3
92126 : WD : Uptime 2 ConnectFailures 0 FreeMem 22512 WiFiStatus 3

System Info:
ESP8266MOD
AI-THINKER

Build:⋄
20104 - Mega
System Libraries:⋄
ESP82xx Core 2_5_2, NONOS SDK 2.2.1(cfd48f3), LWIP: 2.1.2 PUYA support
Git Build:⋄
mega-20190928
Plugins:⋄
46 [Normal]
Build Time:⋄
Sep 28 2019 02:15:30
Binary Filename:⋄
ESP_Easy_mega-20190928_normal_ESP8266_4M1M.bin

[clumsy-stefan]

This could probably happen when the ADC is called while WiFi is still calibrating and not yet connected. Issue is discussed / described here:
https://github.com/letscontrolit/ESPEasy/issues/2621#issuecomment-533887066
and @TD-er explained a possible cause here:
https://github.com/letscontrolit/ESPEasy/issues/2621#issuecomment-533902513

Not sure if this is really a bug or it really needs to be taclled, but I think it would be nice if it could be somehow checked if the ADC is already available or not and therefore delay the request until it's available.

[TD-er]

I've also seen it happen here.
Problem is that it takes some time for the ADC to give proper values after boot.
And like @clumsy-stefan already linked, reading the ADC during WiFi connection attempt may lead to crashes.
So I don't know yet what to do with this.
Even when returning no value (or last value), it is hardly ever anything other than 0 (raw value) on the initial run.
What I can do is re-schedule the ADC read plugin when it returns 0 and thus report no success when the raw value is 0.
But I'm not really sure if there is a valid reason for outputting 0.

[clumsy-stefan]

Hmm.. I think 0 is a valid value for the ADC... The question is more if there is a possibility within the ESP/core to actually check if reading the ADC is a "real" value or if the ADC is not yet initialized/already in use for something else (some kind of register or similar).

BTW: I also saw this phenomen with other sensors (like temp/hum sensors).

The issue for me is with deep-sleep units which give a one-shot for sensors (if you don't use workarounds).

Another possible solution would be if one could specify some "initial wait time" or "condition" (eg. wifi connected) which has to be met before the (first) reading is done.

[TD-er]

The plugin "knows" if there has been a valid value.
And a plugin can re-schedule itself.

But there is also a bit more elegant check at the return value of the PLUGIN_READ function.
If this returned true, it means the value has been updated. Or at least it should be like that (not all plugins obey this rule)

So when in the state of "single fire then go to sleep", we could set a re-schedule of the read command in the handling of these same read calls if the returned value was false.
But then we get at the very same question, how often should we re-schedule? And what interval? The set interval in the plugin? That may leave the node awake for a lot longer than needed.
Should we try to get at least a single read after waking up of all plugins? For how long do we need to test?
Should we stop after 1 (successful) read?

[clumsy-stefan]

Hmm...

Different issues here... If a plugin can not read the value, probably it should be called once again just before going to sleep.

rescheduling on deep-sleep units can be easily done by setting the interval small enough (eg. when the awake time is 10sec, I set the "Interval" parameter (for sending to the controller) to 3 sec. Like this the Plugin is called at least 3 times before going to sleep again, and hopefully gets a valid value (but that's jsut a workaround).

Other possibility would be to collect the values and build some median function (like other plugins do) and only send it to the controller once before going to sleep...

however even then, when the controller can't be reached the first time the reading is lost... therefore the workaround above works quite well... except that it should not even queue a value that could not be read (eg. when the ADC is busy/not ready)..

IMHO

[TD-er]

rescheduling on deep-sleep units can be easily done by setting the interval small enough (eg. when the awake time is 10sec, I set the "Interval" parameter (for sending to the controller) to 3 sec. Like this the Plugin is called at least 3 times before going to sleep again, and hopefully gets a valid value (but that's jsut a workaround).

And that's not how it should work.
Problem is that if it is working fine, it will then send out values 3 times and maybe blocking the one value you're missing because the buffers are full, or the node goes back to sleep again.
The interval timer is exactly that, an interval.
There should be an timeout applicable for all plugins to determine when no new value will come and maybe the plugin should be queried if there is a chance there will be a new value. (e.g. if the sensor is not reacting, there will be no value)

Tweaking interval timers to use this is not the way how it should work.
Interval timers should be applicable for long time operations and maybe we also need a single sample timeout per plugin to support deep sleep operations
Your work around is actually assigning two different interpretations to the same value and that will in the end lead to bugs since from a developer point of view it is impossible to handle all different interpretations of a parameter.

[clumsy-stefan]

And that's not how it should work.

But it does 😄

because the buffers are full

Exactly, that's why I set buffers large enough and awake time laarge enough...

But I agree, it's not how it was intended and should work...

Your work around is actually assigning two different interpretations to the same value

I don't understand what you mean here.... My (deepsleep) nodes collect the values 2 or three times and send it that often to the controller...

[TD-er]

I don't understand what you mean here.... My (deepsleep) nodes collect the values 2 or three times and send it that often to the controller...

Well, like I said, it is a work-around, meaning there is something fundamentally missing in ESPeasy.
What I mean is, the parameter is intended to define an interval, suggesting multiple readings.
Your work-around is to make sure it is taking a single reading and thus you use/interpret the parameter "interval" as something different compared to its intended meaning.
And thus the parameter "interval" is used for something completely different than to define an interval => multiple interpretations.

For sure it is helping you to get done what you need to do, but from a programmer's perspective it is a potential future bug, if parameters are being used for something else as they were intended.
So it is not something you're doing wrong here, it is something wrong (or missing) in the design.

Parameters should be used solely for what they were intended or else something will be broken in the future.
I can only make optimizations in the code (which are essentially short cuts) if I can make assumptions.
If those assumptions are wrong, because it is also used for other purposes or interpretations, then we have a potential future bug.

The hardest problem in programming is finding good names for functions/parameters etc. and this is a nice example of a well named parameter which indicates something else is missing by its "mis-use" :)

[clumsy-stefan]

I don't agree completely...

I'm not using it "wrongly", if so the "stay awake time" would not make sense. If it would be intended that tasks fire off exactly once when the unit is defined for deep sleep, then the stay awake time would not make sense.

it could well be, that a unit is awake for say 5min. to take a number of samples (eg. run a task a number of times) and do some other stuff and then goes to sleep again for a while. So it would possibly be wrong if the tasks only fire once and against what the user intends to do.

thereore my units do exactly what they should, the wake up for 10sec. take at leat three samples, which which I can do whatever I want (eg. calculate a median or similar) and go to sleep again.

Like this a number of issues can be tackled which can never be "programmed" in the code, like warming up a sensor, taking a number of readings to calculate some median, running/switching an output and waiting until something changed due to this (and dtherefore needing multilpe samples) etc... and all of this only once in a while, so goig to deepsleep inbetewen (probably due to battry powered device)...

[TD-er]

I'm not stating there are no valid use cases to tweak the interval timer.
It is just that we're apparently missing some parameter here.
We are missing a "sensor read retry timeout" (preferably one with a better and shorter name) after which a sensor read should be retried and maybe also a number of retries.
Without it, a battery powered node (thus power consumption is key factor) may be left awake longer than needed just hopefully get some reading.

Now we have only the awake time and interval timer to tweak and those are not enough to reduce power consumption to a minimum and still get predictable results.

Also for the analog input (this issue), it may be useful to delay the wifi connection time.
Maybe even delay it until it has become clear there will be useful data to send anyway.

[Domosapiens]

For these type of sleeping nodes ....

The dogma is, because the ESP8266 has WiFi, WiFi connection must be immediately.

Looking for the "evaporating cloud": https://en.wikipedia.org/wiki/Evaporating_Cloud
(a logical diagram representing a problem that has no obvious satisfactory solution)

The flow for sleeping nodes

• node awakes
• node collects data
• node connects to WiFi
• node disconnects WiFi
• node goes sleeping

I suggest some tweaking possibility:

On System#Boot do
WiFi_suspend //WiFi ""wait to connect"" and ""build a queue"" command 
ReadYourAnalogFirstCode
DoYourOtherThingCode
EndOn

On TimerOrOther#Reason do
WiFi_proceed //WiFi regular behavior/connection command
EndOn

On WIFI#QueueEmpty do 
BackToSleep
EndOn

[TD-er]

Thing is, connecting to WiFi may take some time (1 - 10 seconds) and I already made it asynchronous.
So the data collection is started and the WiFi connection is initiated.
Now - with the ADC- we have the first plugin not able to collect data during WiFi setup.
I may make this period a bit shorter, since it really is only during RF calibration when the ADC should not be read, but I don't have an event to see when RF calibration has finished.
I can however distinguish between "connected" and "got IP", so for slow DHCP it can save off 20 - 3000 msec.
In this time the ADC can also collect some samples, so there is some data.
Data will only be sent when there is WiFi, but that doesn't mean there isn't any data in the controller buffers.
So what I should do at least for the ADC plugin is to not output any value (and thus not send anything) as long as there has not been a correct reading.

This would fix this specific issue, but still we have to look into the apparent missing feature to stall WiFi connection and initiate a reading retry.

[clumsy-stefan]

There is also a (probably rare) case of nodes that have no (permanent) WiFi connection and for example collect data or control other things (with GPIO's) based on measurements it takes. In such cases you should'nt/couldn't wait for WiFi connection or even a valid IP-Address...

Therefore it would probably be best, if we could find out, how long the RF-Calibration takes (at most) until ADC is read for the first time (but this also "only" solves this particular issue...

[TD-er]

I don't know if the RF calibration duration is a constant.
I can imagine it must have some stable reading before it can use the data. (or else WiFi stability may be affected a lot by fluctuating power)

[clumsy-stefan]

agree, I guess it more a heuristic value to be found, how long it takes until the first ADC Value gives non-zero (when it actually shoudl give non-zerio).. Just recieving a "zero" value from ADC doesn't mean it's wrong I guess as it's also a valid value (ADC connected to ground), or can it be differnentiated?

[TD-er]

The problem is you should not read the ADC, so it is then hard to see when it does not give 0 anymore :)
I suspect it will return valid values before you can safely read it, since it is interfering with the WiFi RF calibration.

[clumsy-stefan]

yes, that's wh I guess we need to "prorgam" a fixed value by doing some heuristic tests with a unit beforehand. Eg. sepcify a fied initial delay before reading the ADC the first time. IMHO this does not have to be specifiable but can be a fixed constant (probably user-defined at compile-time).

Again, just IMHO....

[clumsy-stefan]

@TD-er : I had a look at the ADC plugin. Some things I'm not clear (or do not understand) why it's there:
in P002_performRead() only a valid value is returned if !wifiConnectInProgress, but as I understand, the AD is only "busy" when the initial calibration is in progress. So why shouldn't the ADC be read when a (re-)connection is in progress?

however in P002_getOutputValue() if there is no oversampling the value will be returned without checking if P002_performRead()actually returned a valid value or not. the returncode of the function is not checked, so it always returnes 0 I guess.

I think as you stated above, the logic of the ADC plugin should probably be rethought and altered, taking into account, that the ADC could be busy when it's tried to be read..

[TD-er]

There are 2 main unknowns here:

• Unsure when the reading is stable
• Unsure when the RF calibration has finished.

The first one may prevent a very simple fix of forcing a single read immediately after boot, before starting WiFi connect.
The second one makes it hard to cut down on the "unavailable period".
Maybe the simplest would be to not return success when calling the PLUGIN_READ function and no value has been read.

[clumsy-stefan]

Maybe the simplest would be to not return success when calling the PLUGIN_READ function and no value has been read.

Agree on this... I think that would be a good first step.

Also probably do oversampling and use a median function could help..

[TD-er]

Well we already use a very basic filtering in the oversampling:

• Only use valid values
• Keep lowest and highest value
• Sum all values and count nr. of values used

Output is:
(sum - highest - lowest) / (count - 2)
Well it is a bit more elaborate than this, but you get the idea.

Keeping track of the median value requires to keep at least half + 1 of the samples in memory and since you don't know the total number of samples you also cannot optimize for "half + 1" samples.
So this approach I chose is the "poor man filtering" to get rid of single fluke values.

[clumsy-stefan]

I guess that filtering would be "ok"... for me it's still unclear how this works on the deepsleep units, as obviously also the "one-shot" does not work as expected (if the plugin send interval is smaller than awaketime/2 it gets executed multiple times) and how the oversampling plays in there...

[TD-er]

It is just counting the number of valid samples until it is read. Then it clears the min/max/count and sum values.

[clumsy-stefan]

and gets executed really 10times per second? also on deep-sleep units?

[TD-er]

Yep, but it only returns a value (in the 10/sec call) when it is not connecting to WiFi.
Main problem is that the read function may return a "success" value of 0 if there has been not a single valid reading.

[clumsy-stefan]

ok, thx.. I'l also have a look at it tomorrow... probably I can propose a "fix" for that...

[clumsy-stefan]

I started PR #2665 with a first commit. At this time enhancing the code only to be able to return false from PLUGIN_READ if no value has been read from ADC.
Still TBD: decide when a value is actually considered "ok" after reading it and capture a read-failure (not sure yet how to do)...

[TD-er]

I already merged it so it can be tested a bit leaving room for some patching before tomorrow's build if needed :)

[clumsy-stefan]

I can't currently test with a serial-connected DS-Unit, but I wonder if this already changes something regarding the issue that @racer73 described.

[clumsy-stefan]

I am a little step further on this With my changes to the ADC at least it skips the initial "0" value (see bleow), still the readings that are done before WiFi is connected do not seem to be queued and sent later to the controller after the connection has been established. Only readings that are captures aftrer connections are sent (in below example the 8 system values never get to the controller).

Not sure how to tackle this, but this is definitively also an issue.

Any Idea why the first call to the sysinfo plugins do not get queud in the controller queue?

INIT : Booting version: SMY_6.02T Oct 16 2019 08:14:29 (ESP82xx Core 40f456ac, NONOS SDK 2.2.2-dev(38a443e), LWIP: 2.1.2)
71 : INIT : Free RAM:31960
72 : INIT : Rebooted from deepsleep #2 Last Task: Background Task - Restart Reason: Deep-Sleep Wake
74 : FS   : Mounting...
96 : FS   : Mount successful, used 75802 bytes of 957314
434 : CRC  : program checksum       ...OK
467 : CRC  : SecuritySettings CRC   ...OK 
564 : INIT : Free RAM:28760
566 : INIT : I2C
567 : INIT : I2C custom clockstretchlimit:1000
569 : INIT : SPI not enabled
571 : EVENT: System#NoSleep=20
584 : INFO : Plugins: 26 [Normal] [4M OTA SMY] 24tasks (ESP82xx Core 40f456ac, NONOS SDK 2.2.2-dev(38a443e), LWIP: 2.1.2)
587 : EVENT: System#Wake
693 : WIFI : Set WiFi to STA
726 : WIFI : Connecting clumsy_ap2 attempt #0
729 : OTA  : Arduino OTA enabled on port 18266
744 : EVENT: System#Boot
851 : WIFI : Connecting clumsy_ap2 attempt #1
994 : SYS  : 31.00,26072.00,100.00,0.00

everything looks fine up to here, then the Tasks Nr. 1&2 are called (sysinfo plugin)

998 : EVENT: sysinfo#rssi=31.00
1025 : EVENT: sysinfo#freeheap=26072.00
1030 : EVENT: sysinfo#load=100.00
1035 : EVENT: sysinfo#uptime=0.00
1187 : SYS  : 1.00,3360.00,0.00,0.00
1189 : EVENT: sysinfo#web=1.00
1214 : EVENT: sysinfo#freestack=3360.00
1218 : EVENT: sysinfo#ip3=0.00
1222 : EVENT: sysinfo#ip4=0.00
1371 : ADC  : No value received 

last line shows the ADC is not ready yet, so no value is returned

2846 : WD   : Uptime 0 ConnectFailures 0 FreeMem 24856 WiFiStatus 6
4796 : WIFI : Connected! AP: clumsy_ap2 (E4:8D:8C:49:4F:BA) Ch: 5 Duration: 3750 ms
4798 : EVENT: WiFi#ChangedAccesspoint
4824 : EVENT: WiFi#ChangedWiFichannel
4826 : WIFI : DHCP IP: 10.0.10.116 (wemos-mini-16-16) GW: 10.0.0.2 SN: 255.255.0.0   duration: 28 ms
4846 : NTP  : NTP replied: delay 11 mSec Accuracy increased by 0.569 seconds
4848 : Time adjusted by -1571245538740.52 msec. Wander: -436457094.09 msec/second
4850 : Current Time Zone:  DST time start: 2019-03-31 02:00:00 offset: 120 minSTD time start: 2019-10-27 03:00:00 offset: 60 min

events below here are beeing sent to the controller, events above not!

4853 : EVENT: Time#Initialized
4858 : EVENT: WiFi#Connected
4863 : Webserver: start
4865 : firstLoopConnectionsEstablished
4866 : EVENT: System#NoSleep=20
4871 : WD   : Uptime 1 ConnectFailures 0 FreeMem 20104 WiFiStatus 3
4874 : EVENT: Clock#Time=Wed,19:05
21527 : EVENT: Clock#Time=Wed,19:06
24867 : WD   : Uptime 1 ConnectFailures 0 FreeMem 20528 WiFiStatus 3
24873 : EVENT: System#Sleep
24875 : SLEEP: Powering down to deepsleep...

[clumsy-stefan]

After looking at the sysinfo plugin, I have a first guess: what happens with plugins that have no periodical call (eg. oncepersecond, every30seconds,etc) but only return the value on a PLUGIN READ? I think they will never be called again after connection is established as all others here (ind ESPEasy.ino:

  // Deep sleep mode, just run all tasks one (more) time and go back to sleep as fast as possible
  if ((firstLoopConnectionsEstablished || readyForSleep()) && isDeepSleepEnabled())
  {
#ifdef USES_MQTT
      runPeriodicalMQTT();
#endif //USES_MQTT
      // Now run all frequent tasks
      run50TimesPerSecond();
      run10TimesPerSecond();
      runEach30Seconds();
      runOncePerSecond();
  }

As the comment says, they should be called "once *more""....

[clumsy-stefan]

and again me 😄 sorry...

BUt I think I found a (possible) solution. addding the (currently commented out) function SensorSendAll() to above block after runOncePerSecond() solves the above described issue..

I'll do a few more checks and a quick PR (#2668) for this, so probably @TD-er can have a look at it...

log after changing above:

INIT : Booting version: SMY_6.02T2 Oct 16 2019 19:49:25 (ESP82xx Core 40f456ac, NONOS SDK 2.2.2-dev(38a443e), LWIP: 2.1.2)
69 : INIT : Free RAM:31960
70 : INIT : Rebooted from deepsleep #2 Last Task: Background Task - Restart Reason: Deep-Sleep Wake
72 : FS   : Mounting...
94 : FS   : Mount successful, used 75802 bytes of 957314
429 : CRC  : program checksum       ...OK
439 : CRC  : SecuritySettings CRC   ...OK 
537 : INIT : Free RAM:28760
539 : INIT : I2C
540 : INIT : I2C custom clockstretchlimit:1000
542 : INIT : SPI not enabled
544 : EVENT: System#NoSleep=20
555 : INFO : Plugins: 26 [Normal] [4M OTA SMY] 24tasks (ESP82xx Core 40f456ac, NONOS SDK 2.2.2-dev(38a443e), LWIP: 2.1.2)
558 : EVENT: System#Wake
664 : WIFI : Set WiFi to STA
697 : WIFI : Connecting clumsy_ap2 attempt #0
700 : OTA  : Arduino OTA enabled on port 18266
715 : EVENT: System#Boot
822 : WIFI : Connecting clumsy_ap2 attempt #1
962 : SYS  : 31.00,26128.00,100.00,0.00
966 : EVENT: sysinfo#rssi=31.00
996 : EVENT: sysinfo#freeheap=26128.00
1001 : EVENT: sysinfo#load=100.00
1006 : EVENT: sysinfo#uptime=0.00
1163 : SYS  : 1.00,3360.00,0.00,0.00
1166 : EVENT: sysinfo#web=1.00
1194 : EVENT: sysinfo#freestack=3360.00
1197 : EVENT: sysinfo#ip3=0.00
1201 : EVENT: sysinfo#ip4=0.00
1355 : ADC  : No value received 
2825 : WD   : Uptime 0 ConnectFailures 0 FreeMem 24856 WiFiStatus 6
4774 : WIFI : Connected! AP: clumsy_ap2 (E4:8D:8C:49:4F:BA) Ch: 5 Duration: 3748 ms
4775 : EVENT: WiFi#ChangedAccesspoint
4800 : EVENT: WiFi#ChangedWiFichannel
4803 : WIFI : DHCP IP: 10.0.10.116 (wemos-mini-16-16) GW: 10.0.0.2 SN: 255.255.0.0   duration: 86 ms
4821 : NTP  : NTP replied: delay 10 mSec Accuracy increased by 0.710 seconds
4822 : Time adjusted by -1571248611906.16 msec. Wander: -436457947.75 msec/second
4824 : Current Time Zone:  DST time start: 2019-03-31 02:00:00 offset: 120 minSTD time start: 2019-10-27 03:00:00 offset: 60 min
4828 : EVENT: Time#Initialized
4833 : EVENT: WiFi#Connected
4838 : Webserver: start
4840 : firstLoopConnectionsEstablished
4842 : EVENT: System#NoSleep=20
4847 : WD   : Uptime 1 ConnectFailures 0 FreeMem 20104 WiFiStatus 3
4850 : EVENT: Clock#Time=Wed,19:56
4858 : SYS  : -60.00,20728.00,100.00,1.00
4861 : EVENT: sysinfo#rssi=-60.00
4887 : EVENT: sysinfo#freeheap=20728.00
4889 : EVENT: sysinfo#load=100.00
4892 : EVENT: sysinfo#uptime=1.00
4928 : SYS  : 4.00,3424.00,10.00,116.00
4930 : EVENT: sysinfo#web=4.00
4956 : EVENT: sysinfo#freestack=3424.00
4959 : EVENT: sysinfo#ip3=10.00
4961 : EVENT: sysinfo#ip4=116.00
4997 : ADC  : Analog value: 1014 = 4.411
4999 : EVENT: sysinfo#lipo=4.41
8498 : EVENT: Clock#Time=Wed,19:57
24844 : EVENT: System#Sleep
24848 : SLEEP: Powering down to deepsleep...

[stefbo27]

Hi there,
I use the FEATURE_ADC_VCC to monitor the voltage. After deepsleep the voltage is -1, after the regular interval a correct voltage is shown. So this could be a similar issue.
Looking at ESPEasy.ino I see that getVcc is called after the Plugins are initialized. Wouldn't it make sense to call this function earlier?
Stefan

[TD-er]

The getVcc is not part of this plugin and thus it also is not affecting the values collected in the PLUGIN_TEN_PER_SEC call of the ADC plugin.
I don't think calling a read earlier will make the ADC yield a proper value earlier after boot.
If it does effectively initialize the ADC, then it should be called in the init function of the ADC plugin.

@clumsy-stefan
If the plugin_READ call does return a valid reading, then it should send it to the controller.
If the controller does not accept a value before the WiFi is ready, then we may have some bug here I guess. (and it could very well be the case here, some optimization I once made)
If this is actually the bug, then we don't have to have something you proposed in your PR, but we should just allow storing to the controller queue before the WiFI is ready.

[clumsy-stefan]

If this is actually the bug, then we don't have to have something you proposed in your PR, but we should just allow storing to the controller queue before the WiFI is ready.

I agree, but I can't really differntiate which of both happens. As you see in the logs, the readings are done, but never sent.

Also for things like the ADC it obviously helps calling PLUGIN_READ at a later time as the ADC (and probably the same with other sensors) had more time to initialize.

Also this does not solve the problem of complete offline nodes (eg. if you have a node that controls watering at a remote location and uses deepSleep, however, not sure if there is any application of such special usecases), there it would never run this block of code, as WiFi is never ready.

[clumsy-stefan]

I just saw, that PluginInit() is called before CPluginInit(), so probably changing the sequence here would solve the problem of not queued values. However some values (as the IP-Adress) will only be available after WiFi connection, so it would be important anyway to call such a plugin again after having a connection to update the values accordingly (same as with ADC). So probably a combination of fixing the queue and calling the plugins again after having a connection would deliver the most reproducable resuts..

EDIT: confirmed. Initializing the Controller Plugins before the Plugins solves the issue of task results not queud in controller queues. but still then, if some values change after connecting (like IP, ADC, etc.) they are never sent, as PLUGIN_READ is not called anymore (if the Interval setting within the task is bigger than the awake time). As written, a combination of changing the init-sequence and calling all tasks again after connection is established produces (for me) the best results.

EDIT2: why should the controllers be initialized at all before a wifi connection is established? wouldn't it be sufficient to do that after connecting and then call all tasks at least once (more)? E.g why queue anything at all if a wifi connection will probably never be established?

[TD-er]

Initialization of a controller also makes it available to collect data in its queue.
So the plugin can only send data to a controller's buffer if it is initialized, but like you said it may never be flushed if there is no WiFi connection.

I think the controllers should be activated first, then the plugins and if there's no WiFi before going to sleep, then queued data is lost.
There is however one possible issue if we change this.
If the unit is running for a long time without WiFi, then the buffers will consume a lot more memory compared to normal operations.
So we may need to check the memory availability when queue'ing and maybe add a TTL value to queued data.
Also some controllers do queue the data rather inefficient (e.g. complete JSON strings per message).

[clumsy-stefan]

I think the controllers should be activated first, then the plugins and if there's no WiFi before going to sleep, then queued data is lost.

Agee to this, at least initializing controllers before notifications and plugins should be done, so the queue's can be used.

But I still think that all takss should be run once (more) before going to sleep, as you do it with 10persec, oncepersec, etc. also the PLUGIN_READ should be called, as there are plugins which have no periodical calls in it.

[TD-er]

I am afraid we cannot have a universal number of reads which is optimal.
So it should be up to the plugin to signal if it needs another read.
Maybe something like "PLUGIN_PREPARE_SLEEP" or something like that.
For example some sensors actually need to be put to sleep to conserve energy and it can also be used to call for the last read if no read was successful.

[clumsy-stefan]

that could be the way to go! One final call to the plugins so it knows it has to finalize whatever, then flush all controller queues, then sleep...

also then it would probably make sense to change the meaning of sleep_awake_time to the time that it waits after finalizing everything. as now I have one node which has about 10-12 sec. to update and send all values, (varies), so I have to use some 15sec. for sleep awake, to make sure that it has enough time to do all stuff.

[TD-er]

But when should the prepare sleep call be sent?
We actually have several stages of "awake":

1. Start plugins/controllers
2. Start WiFi connection
3. Send data via controllers
4. call for prepare sleep
5. Flush buffers
6. Sleep.

What should then be the sleep_awake_time ?
I would say the sleep_awake_time should be the last moment to call step 4.
It then serves as a timeout for making the WiFi connection also.
Step 5 (flush buffers) should also have a timeout, but I have no idea what to use for it.
The timeout in the controller is per message and if it has 10 queues messages it means 10x the set timeout, or even more if it retries per message or sends a message per value instead of per sample (some controllers do it like that)

[clumsy-stefan]

one possibility could be to have the sleep_awake_time starting after all tasks have been run once and all controllers tried to get rid of theyr queue's for once. After that you wait sleep_awake_time sec. to eventually get WiFI connection and read tasks again in a normal loop, and after that ulitmately go to sleep.

as all controllers also have a timeout, this could also account for some delay in wifi connection.

EDIT: on the other hand, if after the first try all sensors could be read and all queue's emtied (normal case) the unit could go to sleep immediately, without waiting any longer... probably a flag needed for this...

[TD-er]

I just added a possible fix for this issue.
See PR #2667
In short, I added a call to the read function from the PLUGIN_INIT, which is being called before WiFi is started.

I have not yet marked the PR to actually fix this issue, since it does contain a lot of insightful discussions on possible other improvements.

At least the controllers are now initialized before the plugins and the ADC is at least read once before WiFi is started.

The other parts about plugin calls related to sleep/wake should be moved to a new issue.
@clumsy-stefan Do you feel up for collecting that info into a new issue, or do you think I should do it?

[clumsy-stefan]

Are you sure that the ADC is not busy after booting the node? Is the calibration only starting when wifi is initialized?

And yes, I'm happy to open a new issue, we just need to make sure the knowlede from this thread is not getting lost (can reference it). You just need to give me a day or two time ;)

[TD-er]

At the moment the plugins are initialized, the WiFi is off, so I'm pretty sure it will not affect the RF calibration.
What I'm not sure about, is if the ADC value is already stable at that point.
The node is already running for a while at that moment (few-100 msec), so I guess it should be stable.

[racer73]

Now the ADC does not return 0 anymore, and as has been stated the discussion should continue in #2694