Linux: DVFS/cpufreq appears to be broken on Raspberry Pi 3

[lgeek]

Hi,

I've tried and failed to replicate Gareth's excessive overheating test (https://www.reddit.com/r/raspberry_pi/comments/48lhot/raspberry_pi_family_thermal_analysis_thermal/). The SoC on my board stays around 60-70 degrees.

Howerver, I think I've figured out why: DVFS/cpufreq appears to be broken on my board. Even if set to the performance governor, or to the userspace governor with the frequency set at 1.2GHz, if I poll /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_cur_freq, it will only show 1200 MHz very rarely, maybe once in 20-40 samples and 600 MHz most of the time. The behaviour is identical if I try to set the frequency to 600 MHz, it still goes up to 1200 MHz from time to time. This is happening both when idle and when under load and at any operating temperatures (even perfectly reasonable ones like 40 degrees).

I'm not really sure what performance I should expect from benchmarks, but I'm getting very high measurement noise, consistent with DVFS randomly switching the operating frequency while short benchmarks are running.

I've tried both a kernel I've built from this repository and the one distributed with Raspbian.

[clivem]

I run a bunch of 'thermal' tests yesterday. The firmware starts throttling at 80degC. I couldn't complete a kernel compile or my 4x core transcoding tests, with a board in the open (not cased) at an ambient of 21degC, without core temp hitting 80degC and throttling arm core speed. Small heatsink and airflow definitely required if you want to actually sustain running under load at 1.2G.....

[P33M]

You will be throttled to 600MHz if an undervoltage event occurs. Does the rainbow square appear top right (if display attached) or does the red PWR LED blink?

[clivem]

$ cpupower frequency-info
analyzing CPU 0:
driver: BCM2835 CPUFreq
CPUs which run at the same hardware frequency: 0 1 2 3
CPUs which need to have their frequency coordinated by software: 0 1 2 3
maximum transition latency: 355 us.
hardware limits: 600 MHz - 1.20 GHz
available frequency steps: 600 MHz, 1.20 GHz
available cpufreq governors: conservative, ondemand, userspace, powersave, performance
current policy: frequency should be within 600 MHz and 1.20 GHz.
The governor "performance" may decide which speed to use
within this range.
current CPU frequency is 1.20 GHz.
cpufreq stats: 600 MHz:99.03%, 1.20 GHz:0.97% (805)

$ while true; do vcgencmd measure_temp && vcgencmd measure_clock arm; sleep 1; done
temp=51.4'C
frequency(45)=1200000000
temp=52.5'C
frequency(45)=1200002000
temp=51.9'C
frequency(45)=1200000000
temp=51.9'C
frequency(45)=1200000000
temp=51.9'C
frequency(45)=1200000000
temp=51.9'C
frequency(45)=1200000000
temp=51.4'C
frequency(45)=1200000000
temp=51.9'C
frequency(45)=1200000000
.........
temp=79.9'C
frequency(45)=1200000000
temp=79.9'C
frequency(45)=1200000000
temp=80.4'C
frequency(45)=1182000000
temp=80.4'C
frequency(45)=1170000000
temp=81.0'C
frequency(45)=1187000000
temp=81.0'C
frequency(45)=1173002000
temp=80.4'C
frequency(45)=1128000000
temp=80.4'C
frequency(45)=1162000000
temp=80.4'C
frequency(45)=1128000000
temp=81.0'C
frequency(45)=1117000000
temp=81.0'C
frequency(45)=1124000000
temp=81.0'C
frequency(45)=1115998000
temp=79.9'C
frequency(45)=1130000000
temp=81.0'C
frequency(45)=1121000000
temp=81.0'C
frequency(45)=1110000000
temp=81.0'C
frequency(45)=1105000000
temp=81.0'C
frequency(45)=1100000000
..........
temp=82.0'C
frequency(45)=951000000
temp=82.0'C
frequency(45)=973000000
temp=82.6'C
frequency(45)=973000000
temp=83.1'C
frequency(45)=937000000
temp=82.0'C
frequency(45)=929998000
temp=82.6'C
frequency(45)=956000000
temp=82.6'C
frequency(45)=982000000
temp=82.6'C
frequency(45)=975000000

[clivem]

@lgeek If wanting accurate arm core speed, you want to be getting it from the firmware, not the kernel. It doesn't get reported back to the kernel when the firmware is throttling. "cat ......../cpuinfo_cur_freq" will still be showing 1.2G, even when firmware has throttled back to 900M. Not that this is your problem, just saying.

[lgeek]

Thanks @P33M, that was it. I've confirmed with a multimeter that the voltage drops quite significantly across the USB cable I'm using and the red LED does indeed blink, which I hadn't noticed before.

Closing this as not-a-bug, although getting a warning in the kernel log would be nice.

[pelwell]

I've seen serial port corruption due to clock reduction because of under-voltage, possibly exacerbated by an inline power switch. Switching to a 2.5A supply solved the problem for me.

[Tito1337]

I agree, there should be more warning than a little flick of the LED. Is there a way to adjust the voltage threshold for this throttling? Mine seems to throttle at 4.9V ! (measured at the Pi, powered by a 3A bench power supply).

I can also put the Pi in a weird loop where the extra current needed for 1.2Ghz is enough to trigger the throttling at 600Mhz, but immediately after, it tries to go back to 1.2G because the voltage drop accross the thick wires dropped also...

Test conditions : PSU set at 5.1V, directly on the GPIO headers. Multimeter on two other headers for voltage measurement. When throttling 4.99V @ 410mA / when trying really hard 4.80V @ 740mA. And it goes back and forth. Total cable resistance measured at 0.27Ohm (consistent with the voltage drop)

I suspect many people will never see the 1.2Ghz of the Pi3 because even high quality USB cables can exhibit a drop of 0.25V, and we're not even talking about the quality of the PSU.

[popcornmix]

There should be warning square in the top right of screen when throttling occurs (multi-coloured for under-voltage, yellow for moderately over-temperature and red for over-temperature).
However there is a bug that means this sometimes doesn't get shown. It is fixed in rpi-update and will be fixed in the apt-get upgrade that will appear imminently.

The voltage threshold is 4.65V and it is not configurable. It is possible your multimeter is not seeing brief voltage drops due to bursts of higher power usage.

It is expected behaviour for the throttle to be removed if the voltage has been above the threshold for a couple of seconds, and it may well throttle again. Running with the under-voltage triggering is not recommended and should be resolved at the power supply end (the official 2.5A power supply will solve the issue).

[ghost]

There should be more of a warning for headless Raspberry Pis. I never use a screen (I have a standard image with Avahi enabled that I use) and there is no way of telling if the pi is underpowered if you are connected with SSH.

[popcornmix]

Run vc_gencmd get_throttled to check if throttled has occurred when headless. See:
https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=147781&p=972822#p972790