Esp-idf: [TW#13852] RTC calibration and precision

[HubbyGitter]

I made a test with the following time recording instances.

• Terminal emulator with time stamps (1 s) on desktop (running chrony)
• ESP_LOG time stamps.
• The ULP looping approximately every second via ulp_set_wakeup_period(), SLEEP and HALT, reporting its cycles and RTC_CLK via RTC SLOW Mem.
• A task sleeping for 1 s via vTaskDelayUntil(), logging ULP cycles and its RTCCLK` snapshot.
• No sleep of main system.

There are no significant temperature changes to be expected; the test was done after resets with the same firmware already running. _Number of cycles for_ RTC_SLOW_CLK _calibration_ was set to 1024. Chip revision is 0.

I let this run for a few minutes and when I stopped it, the result was the following.

• FreeRTOS and desktop / chrony timings are almost identical; due to the granularity in the terminal emulator log it's impossible to say whether there's an actual difference.
• ULP cycles are very accurate and in line with the FreeRTOS timing.
• The RTC is 6.5% ahead of everything else (wrt a reference freq of 150 kHz).

I found this thread which is pretty old as compared to the speed of software improvements, so I'm not sure what the current state is. But it mentions an RTC drift of 0.3 % which is very different from 6.5 %.

What could be the reason for this? Should the calibration setting be longer? Is something going wrong during calibration?

[dottools]

What ESP32 board are you using? Does it have an external 32.768KHz RTC oscillator? If so is it enabled (CONFIG_ESP32_RTC_CLOCK_SOURCE_EXTERNAL_CRYSTAL=y)?

Because I have a sparkfun ESP32 Thing (chip rev 0) board that does have the external RTC oscillator and its enabled in my project's build settings. The program boots up and acquires current time via SNTP after a wifi connection. I've let the board run for just over a day and the RTC doesn't clock drift for me as it's still accurate to the second. Using CONFIG_ESP32_TIME_SYSCALL_USE_RTC_FRC1=y.

Now to be fair I haven't tried deep sleeping and waking up to test time measurements like you have, yet. I'm still figuring out the ulp assembly.

[igrr]

The RTC is 6.5% ahead of everything else (wrt a reference freq of 150 kHz).

How exactly are you measuring this? Using gettimeofday, or by direct access to RTC_TIME registers?

RTC 150kHz oscillator is somewhat tunable, and we are tuning it to approximately 160kHz because it has better temperature stability at that frequency (+/- 5% over the range of operating temperatures, IIRC). Normally the fact that it runs at 160kHz instead of 150kHz is not an issue, because the calibration procedure measures whatever frequency it has. However if you read RTC_TIME and assume that it ticks at 150kHz, there may be a problem.

[HubbyGitter]

@dottools Thank you for sharing. That's apparently not the root cause.

@igrr That is exactly the point, thank you. Quoting myself:

• The ULP looping approximately every second via ulp_set_wakeup_period(), SLEEP and HALT, reporting its cycles and RTC_CLK via RTC SLOW Mem.

And then I divided that by 150k on the main CPU as a first test.

Where do I find the frequency; is that in a documented register? I had searched the headers upfront but found nothing that caught my attention (except for the RTC and its snapshot control register).

[igrr]

Where do I find the frequency; is that in a documented register?

At the moment the last available calibration value can be obtained from this function:
https://github.com/espressif/esp-idf/blob/master/components/esp32/include/esp_clk.h#L43
Note that there is a disclaimer in this header file which says the function is not part of the public API; if you have a use case for calling this function from application code, instead of using gettimeofday, for example, please let me know.
Calibration value you get from this function is the RTC_SLOW_CLK clock period, in microseconds, multiplied by 2^19.

[HubbyGitter]

@igrr Oh my, this simple rename here didn't catch my attention.

Thank you, so it's essentially a register.

One use case is to have the ULP autonomously collect data for you for some time, which is (I believe) even mentioned in ESP docs as an alternative to e.g. ADC DMA. In the details, depending on the requirements, there may be two scenarios.

• It may be desirable to have the ULP adjust its SLEEP time, e.g. due to calculations of various length, with the chance to fill an array of values without time stamps, still maintaining an accurate timing. Many more examples come to my mind.
• If the ULP does not care about timing details, the main CPU obviously will. The ULP can nicely take RTC snapshots and store them (inefficiently, though) along with measured values for the main CPU to post process, but then the main CPU needs to understand what these time stamps really mean.

It's not an XOR if you ask me so having both features officially would be nice.

Yes one can have relative time stamps by means of an ULP maintained counter but with the limited sync between ULP and main CPU, the accuracy drops to the frequency of that counter, and the time base of the counter cannot be precise, especially e.g. if one repeats I2C commands due to implausible results.

Hope I inspired you to see quite a few professional use cases here...

[dottools]

Update now that I have my project deep sleeping majority of the time. I'm still not seeing any clock drift, only that the RTC time jumped back 28 seconds when I switched power from the microUSB to a lipo battery on my sparkfun ESP32 Thing by connecting the lipo battery pack then removing the microUSB connector. Also at the time I disconnected the monitor tool (_connected vi_ a make flash monitor) via Ctrl+] just before the power swap, so could be possible disconnecting the monitor program caused it and not the power swap.

My program project on boot up establishes wifi connection, acquires current time via SNTP (_same as example_), and sends an email. I have a PIR motion sensor connected to a GPIO of which the ULP coprocessor always watches (_using code from system/ulp example_) and so after a low-high-low pulse from PIR sensor the SoC is woken up. On wake-up wifi is again connected, an email is generated and sent, wifi stopped, and goes back to deep sleep.

Each email has a header Date: field that is generated using this code:

char        datestr[64];
struct tm   timeinfo = {0};
time_t      now;

time(&now);
localtime_r(&now, &timeinfo);
strftime(datestr, sizeof(datestr), "%a, %d %b %Y %H:%M:%S %z", &timeinfo);

So I compare the email header Date: field against my local email server's appended Received: header by the time it gets to my inbox to check the clock offset. The program uptime was 1 day, 3 hours, 34 minutes, and 35 seconds.

I'll post another update when I've let the board run just off of the battery pack over the weekend to see what happens since I rebooted it (_via reset button_) and see if any clock drift/offsets occur again.

[dottools]

@dottools you only do SMTP sync on first boot or on each boot?

I'm only doing SNTP sync on power up boot process (POWERON_RESET) by using static RTC_DATA_ATTR uint32_t sm_Persistent = 0 variable and checking with an enum flag Persist_TimeSet = (1 << 1), to make sure time was successfully synced. It's apart of a state machine I have setup to automate handling of making sure network is up before sending an email and making sure there's a valid time to set the Date: header field to. And to make sure everything works as expected there's plenty ESP_LOGI() usage to print to monitor tool for debugging and watch what's going on.

[negativekelvin]

@dottools figured so and deleted my comment because the fact you are using RTC crystal is major factor in drift.

[HubbyGitter]

@negativekelvin Have you made measurements regarding the difference between using external crystal for RTC vs not using it?

[dottools]

After one-time SNTP time sync, powered entirely on a lipo battery, and deep sleeping majority of the time for 5 days, 15 hours, 50 minutes, and 27 seconds the time has drifted ~5 seconds ahead of my home server (running ntpd).

Again this is using the external 32.768KHz oscillator on the sparkfun ESP32 Thing board. The board was set aside and left alone near my work space except for the occasional motion sensor trigger to make it wake-up and send an email couple times each day. Room temperatures ~74 F to 76 F through each day (middle Tennessee, US weather).

[hwmaier]

@dottools The 5s over that period would translate to a clock drift of ~10ppm. Most crystals have an accuracy of 20ppm, so this would be well within the accuracy of the crystal and an acceptable drift.

[igrr]

Closing, since esp_clk_slowclk_cal_get function is now in a public header file, which is what was requested in https://github.com/espressif/esp-idf/issues/769#issuecomment-313313474.