Linux: Make the RPI POE Hat Fan user-controllable and adjustable

[hansamann]

I am controlling my kids gecko/hamster cams with a RPI and got POE hats for these PIs. All was great when the fan was still not controlled, e.g. turned off at all times. Now, with the october raspbian update the fan was on all time as it seems the temp it turns on is 45 and that pretty much the case at all times.

As I was running the PEO hats for several weeks with the fan completely off and my cpu temp is just a little above the 45 degrees, it seems fair to at least adjust the fan temps to some degree. I would hope that raspi-config would allow me to turn the fan-control on/off completely and for on I would love to see different levels (aggressive fan, less aggressive).

I really think this needs fixing as many will have RPIs with POE hats around during night time and especially for all private users at home (potentially with kids) the subtle humming / cracking of the fan can be very disturbing.

[hansamann]

For now I have unplugged the fan and of course I destroyed one of the connectors. Can someone also please let me know the exact name of the connector so I can order a replacement connector with cables so I can easily solder it back on?

[XECDesign]

If you don't need the fan, you can turn it off by putting disable_poe_fan=1 in config.txt. The behaviour of the fan is defined in the device tree overlay for it, which can be edited as necessary.

Device tree can be a bit tricky, so some of the options have been made adjustable through config.txt. That feature hasn't made it into the pre-built kernel yet.

Some links which could be useful:
https://www.raspberrypi.org/forums/viewtopic.php?t=219050&start=25#p1354556
https://www.raspberrypi.org/forums/viewtopic.php?f=45&t=219050&start=50#p1356682 (a little out of date)
https://github.com/raspberrypi/linux/pull/2700

[sbromberger]

Hi there,

I've tried setting

dtoverlay=rpi-poe,poe_fan_temp0=60000,poe_fan_temp1=68000

in /boot/config.txt but it doesn't seem to be having any effect: /sys/class/thermal/cooling_device0/... shows current state of 2 when temp is 51C.

Is there another way to set the fan thresholds? This is on an RPi4 with the POE hat, using 4.19.56-v7l+.

EDIT:

dtc -I fs /proc/device-tree yields, in part,

        rpi-poe-fan@0 {
                cooling-min-state = < 0x00 >;
                compatible = "raspberrypi,rpi-poe-fan";
                cooling-levels = < 0x00 0x96 0xff >;
                firmware = < 0x07 >;
                status = "okay";
                phandle = < 0xcb >;
                cooling-max-state = < 0x02 >;
                #cooling-cells = < 0x02 >;
        };

[sbromberger]

Also, I'm a little confused by the default settings:

(57.0'C) seth@orange:/sys/class/thermal/thermal_zone0$ cat trip_point_0_temp
55000
(56.0'C) seth@orange:/sys/class/thermal/thermal_zone0$ cat trip_point_1_temp
50000

The dtb file shows these reversed (that is, trip point 0 should be 50k, trip point 1 55k). I'm also seeing this switch when I edit the dtb to increase the temps directly. Is this expected?

[patschi]

I've tried it on my RPi4 with the official PoE HAT as well, and I indeed see the same behavior:

# cat /boot/config.txt | grep poe
dtoverlay=rpi-poe,poe_fan_temp0=60000,poe_fan_temp1=68000
# uname -r
4.19.50-v7l+
# vcgencmd measure_temp
temp=48.0'C
# cat /sys/class/hwmon/hwmon0/pwm1
150

(I guess the temp was 50°C, but just went down to 48°C when executing the command)

The fan seems still to turn on at 50 °C, even the fact I've overritten it to 60 °C.

In my environment the fan always turns on for a few seconds every few minutes, as it always comes across the trip0 temperature limit for a relatively short time. So increasing the trip0 should basically solve that for me - but unfortunately does not seem to work as expected.

[sbromberger]

I managed to work around this by decompiling the overlay with

dtc -I dtb -O dts /boot/overlays/rpi-poe.dtbo > rpi-poe.dtb

editing the rpi-poe.dtb to switch the values (and increase them appropriately),
recompiling using

dtc -@ -I dts -O dtb -o rpi-poe-reversed.dtbo rpi-poe.dtb

and then moving the rpi-poe-reversed.dtbo to /boot/overlays/rpi-poe.dtbo. Remember to back up your original in /boot/overlays first.

[JamesH65]

@XECDesign Any thoughts?

[pelwell]

I can take a look in the morning.

[pelwell]

It looks like there might be two issues here - the overlay parameters and the trip point ordering.

1. Contrary to the documentation, the overlay doesn't currently have any parameters. Instead it gives the base dtb some parameters of its own. This makes more sense when you realise that the POE HAT EEPROM causes the overlay to be loaded automatically, so there isn't an obvious way (*) to pass the parameter values. Replacing your dtoverlay line with dtparam=poe_fan_temp0=60000,poe_fan_temp1=68000 should do what you are trying to do, at least as far as the DT contents goes:

pi@raspberrypi:~$ od -An --endian=big -td4 /proc/device-tree/thermal-zones/cpu-thermal/trips/trip0/temperature
       60000
pi@raspberrypi:~$ od -An --endian=big -td4 /proc/device-tree/thermal-zones/cpu-thermal/trips/trip1/temperature
       68000

To make the documentation correct and to avoid future confusion, I've added real parameters to the overlay.

1. The trip points are appearing in reverse order. This sounds like a thermal subsystem/Device Tree change, which needs more investigation.

(*) It is possible to pass parameters to a HAT overlay - the overlay is "in scope" until the next dtoverlay=..., so a dtparam=... before the first dtoverlay=... will refer to the overlay loaded for a HAT or, if no matching parameter is found, the base Device Tree.

[sbromberger]

@pelwell - thanks for the response. I have the dtparam line above as the first uncommented line in config.txt, but in my /proc/device-tree/.../cpu-thermal output, I have 50000 for trip0 and 55000 for trip1, so I don't think they're actually being set.

I take that back. I forgot to comment out the dtoverlay line below it, so I guess it was getting reset or something.

[pelwell]

I believe the order of the trip points is irrelevant - their content and that of the cooling maps defines the implicit ordering. If you (or anyone else) feels the cooling isn't working as it should then please correct me.

[sbromberger]

@pelwell - thanks so much for your help here. I've run using the dtparams config mentioned above for the past couple of days and can confirm the trip points are being set correctly (despite the ordering issue). At this point there is no need to modify the dtb file directly; the dtparams handles everything.

Man, those PoE fans are loud :)

[pelwell]

Closing, as I think the issues raised have been addressed.

[kjetilk]

I placed dtaram=poe_fan_temp0=60000,poe_fan_temp1=68000 in my /boot/config.txt, and get

# od -An --endian=big -td4 /proc/device-tree/thermal-zones/cpu-thermal/trips/trip0/temperature
       60000
# od -An --endian=big -td4 /proc/device-tree/thermal-zones/cpu-thermal/trips/trip1/temperature
       68000

but the fan is running at much lower temperatures. Is there a more complete documentation for how this is done from start to end somewhere?

[pelwell]

How are you measuring the temperature?

[kjetilk]

How are you measuring the temperature?

With

vcgencmd measure_temp

Also, I hear the fan running when the RPi4 is pretty much idle (i.e. has 0 load average).

[DSlatkin]

@pelwell Just an FYI the line dtaram=poe_fan_temp0=60000,poe_fan_temp1=68000 suggested earlier to add to /boot/config.txt to set the temperature thresholds has a typo in it, dtaram should be dtparam.

@kjetilk My output with od actually wasn't correct until I fixed that typo. You may have to reboot with sudo shutdown -r now for the changes to take effect.

Summary of my now-fixed situation: using my Pi with a PoE hat to run my UniFi Controller and Pi-hole. Average idle temps were around 49.4 °C and constantly creeping over the default 50 °C threshold causing the fan to trigger constantly. After updating the threshold to 60 °C with Device Tree overlays with the dtparams above, I'm sitting at about 54.8 °C at idle with no case and I'm no longer needing to listen to the fan loudly spin while trying to sleep. 🙂

Since this basically sits between at 55 °C on idle with no case, would it make sense to increase the default threshold to 60 °C instead of 50 °C? That's still under 70 °C limit for the board. Maybe get some more life out of the fan this way too if it's not constantly spinning up.

[pelwell]

Well spotted - I've corrected the example.

[kjetilk]

Thanks, but I did in fact correct it when I entered into the boot config. :-) Just didn't think about it when I did a cutnpaste here. So that has been correct all along.

Is there anything else I need to do with the overlays, or something? FWIW, my current device tree says:

        rpi-poe-fan@0 {
                cooling-min-state = < 0x00 >;
                compatible = "raspberrypi,rpi-poe-fan";
                cooling-levels = < 0x00 0x96 0xff >;
                firmware = < 0x07 >;
                status = "okay";
                phandle = < 0xc8 >;
                cooling-max-state = < 0x02 >;
                #cooling-cells = < 0x02 >;
        };

[bennetts2]

Hey, I don't know if anyone cares at this point, but I was previously controlling the fan speed with:

echo 50 > /sys/class/hwmon/hwmon1/pwm1

This seems to be enough pwm for the fan to start (at lower levels it would sometimes stall) but is low enough that you really cannot hear it. The small amount of airflow with the pwm set to 50 is enough extra cooling to stop my Pi from spinning the fan up once a minute even when it's idle.

The latest firmware (probably not very recent, this particular Pi hasn't rebooted since September I think) seems to have removed this option for speed control (grr), but inspired by @sbromberger and @kjetilk I tried updating my rpi-poe.dtbo to change the "level 1" fanspeed from 150 (0x96) to 50 (0x32).

It appears to work great! (based on approx 5 mins of evidence, YMMV, don't blame me).
My device tree looks like this:

        rpi-poe-fan@0 {
                cooling-min-state = < 0x00 >;
                compatible = "raspberrypi,rpi-poe-fan";
                cooling-levels = < 0x00 0x32 0xff >;
                firmware = < 0x07 >;
                status = "okay";
                phandle = < 0xc9 >;
                cooling-max-state = < 0x02 >;
                #cooling-cells = < 0x02 >;
        };

I guess a better alternative might be to instead set the fanspeed for "level 0" to 0x32, then the cooling will still kick in at the appropriate levels, it'll just never switch off completely.

I've not looked to see if there's a kernel command-line option to set the 'cooling levels' values: that might be a simpler way to tinker with these parameters.

[Cheddoleum]

For now I have unplugged the fan and of course I destroyed one of the connectors. Can someone also please let me know the exact name of the connector so I can order a replacement connector with cables so I can easily solder it back on?

I've been trying to work this out. Based just on looking at images online it looks a lot like a 2-pin molex picoblade, as used here, and if it weren't for COVID-19 I'd swing by my local electronics supply to try one out. I'd like a spare connector as I've experimented with using a larger and slower fan for silent operation (and it works like a dream) but to do that connector-wise I've just been impaling the wire ends on the pins and that's not a good permanent arrangement.

Edit: changed connector identification

[AndrewFarley]

Can we consider re-opening this this bug/issue?

I would prefer more granularity in the ability to control the fan, specifically...
• I would like to be able to have more than two intensity levels of the fan
• I would like to be able to implement more aspects of an intensity curve
• I would like to be able to run the fan in an extremely low-rpm (and thus low noise) fashion for environments that can not tolerate noise but need a bit more thermal dissipation than passive.

Possible solutions to the above
• Add a new param to each tempX level that allows setting the intensity of the fan. Eg: poe_fan_temp0_intensity=10
• Add a few more tempX levels (eg: lets say 2 or 3 more)
• Document how you could disable this poe i2c module and have some sample code of how to talk to the i2c to control the fan, so the user could implement their own tiny daemon to do the same

Thoughts?

[pelwell]

It sounds like you haven't noticed that the rpi-poe overlay was updated at the end of March, adding two new temperature thresholds and fan speeds:

Name:   rpi-poe
Info:   Raspberry Pi PoE HAT fan
Load:   dtoverlay=rpi-poe,<param>[=<val>]
Params: poe_fan_temp0           Temperature (in millicelcius) at which the fan
                                turns on (default 40000)
        poe_fan_temp0_hyst      Temperature delta (in millicelcius) at which
                                the fan turns off (default 2000)
        poe_fan_temp1           Temperature (in millicelcius) at which the fan
                                speeds up (default 45000)
        poe_fan_temp1_hyst      Temperature delta (in millicelcius) at which
                                the fan slows down (default 2000)
        poe_fan_temp2           Temperature (in millicelcius) at which the fan
                                speeds up (default 50000)
        poe_fan_temp2_hyst      Temperature delta (in millicelcius) at which
                                the fan slows down (default 2000)
        poe_fan_temp3           Temperature (in millicelcius) at which the fan
                                speeds up (default 55000)
        poe_fan_temp3_hyst      Temperature delta (in millicelcius) at which
                                the fan slows down (default 5000)

[AndrewFarley]

@pelwell Hey hey, that's pretty good didn't know that. I'll have to check if this works and/or update my overlay so it can. Still sorely missing the ability to set the fan speed though. Thanks!

[Cheddoleum]

The ability to set the fan speed per threshold has been present all along; the Feb 19 comment above from @bennetts2 explains it in detail.

Even without updating to the new overlay, I've been getting results on a Pi 4 that suit my pretty ordinary use case very well, simply by setting

cooling-levels = <0x00 0x20 0xff>;

and

dtparam=poe_fan_temp0=57000,poe_fan_temp0_hyst=7000,poe_fan_temp1=67000,poe_fan_temp1_hyst=6000

This causes the fan to come on very quietly indeed at 57°, and cool it below 50° in less than 20 seconds even under the highest CPU load in my use case, around 200%. If for some reason the temperature were to continue to climb, the fan will increase to full at 67°, still well short of throttling speed. I don't expect that to happen in the real world but it's reassuring to have it there as a backstop.

Under a more typical CPU load range of 10%-80% the temperature stabilizes passively around ~50° and the fan never needs to come on at all. My reading around the web suggests this is fairly typical, so I'm not overly thrilled with the new defaults in the 25 Mar commit . This has it start spinning -- albeit fairly quietly -- at 40° which suggests to me it'd be coming on more frequently than necessary under conditions that I'd guess are pretty common.

[AndrewFarley]

I'm sorry if I feel that decompiling an overlay and recompiling it with edits isn't my idea of fun. :P The "howto" modify the cooling-levels value isn't the most straightforward, and I don't really enjoy the idea of disassembling my Pi cases to eject the memory card to fix them if I f-up the overlay. If possible can we still move this cooling-levels into a dtparam?

[pelwell]

You never need to decompile any of our overlays - the source is in the kernel tree: https://github.com/raspberrypi/linux/blob/rpi-5.4.y/arch/arm/boot/dts/overlays/rpi-poe-overlay.dts

You really can't mess up the overlay enough to prevent a Pi from booting - it would take knowledge and intent. If somebody really wants to submit a PR adding parameters (with README documentation) to control the fan speeds then I will consider merging it, but I won't be adding them myself.

[sveyret]

Don't know if I need to open a new ticket or continue this one, but I have a problem when setting temperature threshold for the fan: the poe_fan_temp3 always keeps its value of 55000, whichever value I set to it.

# od -An --endian=big -td4 /proc/device-tree/thermal-zones/cpu-thermal/trips/trip3/temperature 
55000

[pelwell]

Are you saying that the other values change correctly, just not the trip3 temperature? Can I see the contents of your config.txt? You can leave out anything which is obviously unrelated.

[sveyret]

Sorry, false alert. After more investigation, it seems to be a problem of line length. The following definitions are working for temperatures 0 to 2, but not for temperature 3:

dtoverlay=rpi-poe,poe_fan_temp0=50000,poe_fan_temp1=58000,poe_fan_temp2=64000,poe_fan_temp3=68000

or:

dtparam=poe_fan_temp0=50000,poe_fan_temp1=58000,poe_fan_temp2=64000,poe_fan_temp3=68000

But if I split the parameters into multiple lines, it works correctly:

dtoverlay=rpi-poe
dtparam=poe_fan_temp0=50000
dtparam=poe_fan_temp1=58000
dtparam=poe_fan_temp2=64000
dtparam=poe_fan_temp3=68000

[kiptonk]

Sorry, false alert. After more investigation, it seems to be a problem of line length. The following definitions are working for temperatures 0 to 2, but not for temperature 3:

I'm seeing this also. If the dtparams are set on the same line as dtoverlay, the temp3 setting is ignored and stays at the default 55000. Setting them one per line works as expected. Can confirm by checking /sys/class/thermal/thermal_zone0/trip_point_* (note that sequencing is reversed). Not sure if this is an issue specific to the fan overlay or a general issue with long lines for overlay params? Maybe a new issue should be created?

[pelwell]

There is a line length limit for config.txt - it used to be 80 but (from what I remember) it's more like 100 now. Splitting the parameters onto separate lines avoids hitting this limit, but otherwise it makes no difference.

[Nooblet-69]

Just a quick note for the people like me that are a little slow. If you dont overwrite the default limit for all 4 thresholds it will used the default. I didn't want the fan to kick in until 60C and set temp0 and temp1 correctly. however because temp2 was set at 50C the fan was running anyway. make sure you overwrite all for levels.