Arduino: Implement low power mode api

[devyte]

Basic Infos

• [x] This issue complies with the issue POLICY doc.
• [x] I have read the documentation at readthedocs and the issue is not addressed there.
• [x] I have tested that the issue is present in current master branch (aka latest git).
• [x] I have searched the issue tracker for a similar issue.
• [x] If there is a stack dump, I have decoded it.
• [x] I have filled out all fields below.

Platform

• Hardware: All
• Core Version: latest git today
• Development Env: All
• Operating System: All

Problem Description

PR #6989 provides a low power usage reference example, and it has been merged. Per @d-a-v 's comment, it should be investigated how to unify the current experimental pseudo modes with the reference code in the example, and how to provide a consistent api to the users. Once that api is available, the example in #6989 should be migrated to make use of that api without changing the current behavior of the tests in that example sketch.

MCVE Sketch

See the example merged in #6989.

[devyte]

CC @Tech-TX

[devyte]

Related to discussion in #6642

[Tech-TX]

(laughs) That's not exactly an MCVE, but it'll do. Timed Light Sleep (timer or GPIO wake) and Forced Light Sleep (GPIO wake) would benefit from API wrappers; the rest of the modes are already encapsulated. Both of those Light Sleep modes need the WiFi off, so it'd be more friendly if it were an ESP API and not a WiFi API. Similarly, an ESP API for Forced Modem Sleep would be nice if you're not using WiFi (the last section in the README).

Additionally I'm still researching the low-power boot to see if I can get that working again. I saw something last night in another thread that doesn't solve it exactly, but it certainly helps. During most of the boot the current is below 37mA. The RTC write does _something weird_ to the modem and current only peaks at 53mA then drops to 30mA, but I can't drop it to the 15-18mA 'Forced Modem Sleep' range after that's been done (yet). It's zeroing the target in the RTC for Deep/Light Sleep wakeup to _mostly disable_ the modem. On the plus side, the current only hits ~53mA for 650us which is filterable by an electrolytic cap.

Unfortunately it's not entirely consistent. 1/5 to 1/8 times it does this instead:

I've duplicated the write to RTC inside preinit() and it's occasionally not working the same. The first write in RF_PRE_INIT() seems to always work.

[Tech-TX]

Done! :sleeping: I beat igrr's results by ~20mA, and it works today. It barely peaks at 54mA, should be able to run with a 50mA PSU if you're not using WiFi. An electrolytic will smooth that ~50mA peak out. Except for the brief peak, the maximum current during boot is ~35mA, and is lower on successive resets after power on. This fixes the issue originally raised here:
https://github.com/esp8266/Arduino/issues/2111

#include <user_interface.h>

#define _R (uint32 *)PERIPHS_RTC_BASEADDR
#define valRTC 0xFE000000  // normal value during CONT (modem turned on)

RF_PRE_INIT() {
  *(_R + 4) = 0;  // value of RTC_COUNTER for wakeup from light/deep-sleep (sleep the modem)
  system_phy_set_powerup_option(2);  // stop the RFCAL at boot
  wifi_set_opmode_current(NULL_MODE);  // set Wi-Fi to unconfigured, don't save to flash
  wifi_fpm_set_sleep_type(MODEM_SLEEP_T);  // set the sleep type to Forced Modem Sleep
  // moving the two SDK commands up here saves another 670us @ 8 mA of boot current :-)
}

void preinit() {
  *(_R + 4) = 0;  // sleep the modem again after another initialize attempt before preinit()
  wifi_fpm_open();  // enable Forced Modem Sleep, not in RF_PRE_INIT or it doesn't sleep
  wifi_fpm_do_sleep(0xFFFFFFF);  // enter Modem Sleep mode, in RF_PRE_INIT causes WDT
  // won't go into Forced Modem Sleep until it sees the delay() in setup()
}

void setup() {
   *(_R + 4) = valRTC;  // Force Modem Sleep engaged, set the modem back to 'normal' briefly
  delay(1);  // and sleep the modem at 18mA nominal substrate current
}

void loop() {
  yield();
}

zoomed out to show the power on:

_I'll just leave this here_ for the ultra-low-power folks. There's no clean way to implement this in an API as it's scattered across 3 different routines. The 4 SDK calls in preinit() that do modem sleep are essentially the same as WiFi.forceSleepBegin() without loading the WiFi library. The maximum instantaneous power is slightly higher with other modules, and the operating current will be higher than the 18mA I'm getting if you have a USB chip, voltage regulator or power LED.

[Tech-TX]

@d-a-v I'll play with the Light Sleep modes (the only two currently not encapsulated) to make sure the optional items can go _before_ the base sleep functions. The Timed Light Sleep needs more setup than the simpler Forced Light Sleep (wakeup with GPIO only). The optional item for Forced Light Sleep are whether you wanted a callback, which GPIO to use for the interrupt, and whether it's LOLEVEL or HILEVEL. I'll see if it'll work out of the order I have them in currently.

Timed Light Sleep was twitchy to get running, and if they don't use a callback the timed sleep is weird (first sleep(x millis), then delay(x millis)) so I'd make the callback required, even if it's empty. Although the SDK call passes micros for the sleep time, it has to be followed by delay(time in micros + 1 milli) so there's no reason to send it micros. Timed Light Sleep can also be woken by an optional GPIO interrupt (also LOLEVEL or HILEVEL). If that's too ugly to add as optional passed parameters then it might work having them set the GPIO interrupt before the API is called. I'll test that and add a message with the results. Give me a few days to verify the variations thoroughly.

If I had the first clue how to encapsulate those two in an ESP class I'd do it, but I'm not up on C++.

Your pseudo modes work great, no need for anything else on them. I can edit the .rst file so that they're documented, if you'd like.

[Tech-TX]

It appears there's WAY more involved in bringing it out of Light Sleep than I thought. What I'd been fighting is that the top two timers on the os_timer list are from inside the SDK, and just copying the values back to the list doesn't re-attach them. I haven't used the os_timers before, so I didn't know that wouldn't work. The top/fastest timer (check_timeouts_timer) is the one that controls everything else on the list. Without that timer, the os_timer list falls apart. Google to the rescue, once I had the right search terms. :wink:

It turns out the NodeMCU crowd did it 2 years ago, nodemcu/nodemcu-firmware#1231 (2000 additions, 80 deletions for that PR...) The ugliest part of it is saving and restoring the os_timer list. They had to walk the list and disarm everything, save it, then re-attach all of the timers again after sleep. Here's the short description of their version of Timed Light Sleep: https://nodemcu.readthedocs.io/en/master/modules/wifi/#wifisuspend
and the doc for their version of Forced Light Sleep: https://nodemcu.readthedocs.io/en/master/modules/node/#nodesleep
(same low-power modes, merely different API names than Espressif used for Light Sleep). That PR is a long read.

Looks like they were at SDK 2.2.0 when this went through, and they had support for all of the different interrupt types for the Forced Modem Sleep (wakeup with GPIO), not just HILEVEL/LOLEVEL. I'll dig inside and see how they did that. They're currently at SDK 3.0 as of September.

You don't have to disarm anything to get either of the Forced Light Sleep modes to work, and for Timed Light Sleep you only need to point the timer_list to the last timer on the list. The SDK functions that put it to sleep disable the os_timers after that, but we DO need to save them so we can re-attach them afterwards. The first SDK call wifi_fpm_set_sleep_type(LIGHT_SLEEP_T) disables nearly all os_timers, and the rest are disabled after one of the later Sleep calls (I couldn't find which one). With Devyte's simple copy of the timer struct something weird was going on, and only the top 2 timers were recoverable (two SDK timers). The others on the list was being clobbered (by the SDK?) in the copy. They were there immediately after saving a copy but were gone after sleep.

Any API encapsulation of Forced Light Sleep (either mode) will need to save and restore/restart the os_timers.

Here's the code I was using to save and then view the timer_list:

extern os_timer_t *timer_list;  // get the list of os_timers
os_timer_t * orig_list = timer_list;  // make a copy of the list
 while (timer_list != 0) {  // point at the last os_timer in the list so the SDK can halt them
  Serial.printf("timer_address = %p\n", timer_list);
  Serial.printf("timer_expire  = %u\n", timer_list->timer_expire);
  Serial.printf("timer_period  = %u\n", timer_list->timer_period);
  Serial.printf("timer_func    = %p\n", timer_list->timer_func);
  Serial.printf("timer_next    = %p\n", timer_list->timer_next);
  Serial.printf("=============\n");
  timer_list = timer_list->timer_next;
}

and to restore the list after sleep just

timer_list = orig_list;

but unfortunately the copy only contains the top two (SDK) timers after sleep.

[ArnieO]

@Tech-TX Please forgive me in advance if this is not the right place to comment on LowPowerDemo (#6989 ).

I've been searching for a solid way to master sleep modes (in particular Auto and Forced Modem Sleep), and found your great work today.
I'm about to debug an application where Auto Modem Sleep behaves differently on two Wifi networks, and want to use LowPowerDemo to investigate the issue.

I ran into the following issue:
When uploading this to a virgin ESP-01, I did not succeed connecting to Wifi.
Arduino IDE 1.8.13. I tried several compiler options, including erasing both sketch and Wifi settings.
I also tried using another simpler sketch to connect, then recompiling LowPowerDemo with compiler option that only erase sketch (not Wifi settings) - but still no connection.
The only way I found to work was to comment out lines 408 and 421 (avoiding the "if"-test).
So now I can run the code. However, this code is at an advancement level where I fail to see if this influences on the "Wifi connect time" results - which is important for my investigation.

Do you have an idea why I fail to connect to Wifi with your original code?

Side comment regarding Line 414:
WiFi.setOutputPower(10); // reduce RF output power, increase if it won't connect
I fail to see where in the code RF output power is increased, as this seems to be the only call to WiFi.setOutputPower(x).
Using Wifi connection code that adjusts RF output power sounds potentially useful, so it would be great if you could clarify this point.

[vks007]

@Tech-TX , Thanks for all the pointers. I am trying to write code to go into light sleep with the ability of waking up either via GPIO interrupt or on expiry of a fixed time. So I have to look for a way to suspend and then re-enable the timers after sleep. If I use just the pointer method suggested by you I get back 3 out of 4 timers (instead of 2). These probably are internal timers.
I think suspending and restoring the timers is a bit more complicated than this. I looked at the code nodemcu team have written here for the same requirement and there's a lot going on. There is special handling of the timers with callback too. I am not an advanced user to be able to adapt that code to the need here. maybe somebody else can help.

[ericbeaudry]

@Tech-TX , That's great work there on managing to reduce power at boot! So good that I'm actually struggling to establish a connection once in that state. Would you mind clarify how to safely restore the modem when it's needed.
ESP.forceSleepWake(); isn't cutting it and neither are :

wifi_fpm_close;
wifi_set_opmode(STATION_MODE);
wifi_station_connect();

I must be missing sometime because once in that mode, I cannot reconnect to any network. My guess is I'm unable to wake it up.

[vks007]

@ericbeaudry , you should start with the low power example which comes with the IDE. Load that and see if everything is fine, then remove the parts you don't want form the program and adapt it to your needs.