Linux: One core is running lxpanel process at 100+ % CPU usage after kernel update 4.14.30-v7+

[E3V3A]

After having updated to the latest Kernel 4.14.30-v7+, one processor core was constantly running an lxpanel process at 100+ % CPU usage. (Yes, that's right it often show more than 100%.) I found the process running to be: lxpanel --profile LXDE-pi process.

Checking with htop:

top - 10:13:17 up 2 days, 29 min,  2 users,  load average: 1.06, 1.03, 1.00
Tasks: 120 total,   2 running,  75 sleeping,   0 stopped,   0 zombie
%Cpu(s): 14.4 us, 10.8 sy,  0.0 ni, 74.8 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :   949476 total,   384624 free,   109396 used,   455456 buff/cache
KiB Swap:   102396 total,   102396 free,        0 used.   765540 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
  778 pi        20   0  148544  32428  19840 R 100.2  3.4  49:47.31 lxpanel --profile LXDE-pi

I then disabled a number of (bloat) services and rebooted. Now the system is quasi-normal again.
So I have no idea what caused this, and have no way to reproduce it.

[ThomasKaiser]

Since you ignore a severe hardware problem (extreme under-voltage) I wonder whether your installation is already corrupted?

Ever checked installation integrity?

VerifyRepairExcludes="/etc/|/boot/|cache|getty/"
dpkg --verify | egrep -v -i "${VerifyRepairExcludes}"

[E3V3A]

It's all good. :)

Please try to refrain from using words like severe and extreme, we all know there is nothing of such sort. :1st_place_medal:

??5??????   /usr/share/alsa/alsa.conf
??5??????   /usr/share/plymouth/themes/pix/splash.png
??5??????   /usr/share/vte/termcap-0.0/xterm

[JamesH65]

Feel free to continue to think that an unvoltage condition is OK.

(It's not, btw, but you can continue to think that if you wish)

[E3V3A]

@JamesH65

Feel free to argue that all the issues here on github are related to a suddenly faulty PSU, which only happened after the severely untested update you provided. In this extremely constructive way of addressing issues, I am sure a lot of people want to continue help and all 176+ open issues can be closed at a snap.

Now, let's step back and assume that I really do have a super uber severely extreme power problem. How does that explain the issue here? Why are there no other indications anywhere about such issues? Like SD card read/write errors, Wifi lost packets, slow starting processes? Flickering screens? Well, you name it. Your PSU arguments are totally irrelevant to the issue at hand.

[JamesH65]

1. 4.14 was tested for multiple months, both in house and in the field. It was available for anyone to try during that period.
2. When we need to diagnose problems, the first thing we ALWAYS check is power supply. Supply problems can cause all sorts of weird issues, on SD cards, ethernet , wifi etc. Without stable power, we cannot know for certain whether power supply might be causing an issue and could spend days trying to find a software error that was actually a power supply issue.

You have a power supply problem, probably entirely unrelated to the issue here. It's that probably bit that is causing the problem, we cannot be entirely sure its NOT causing the problem, so the first thing to do it fix the problem we know about, then once that is eliminated, we can move on to finding the real issue. This is an absolutely standard diagnostic process. Fix all the obvious problems first.

[E3V3A]

I'm closing this issue as it doesn't seem reproducible. At least now it has been logged in case this happen to someone else.

Because it is using lxpanel, and thus indirectly using x11, this issue may also be related to the xset one in #2517.

[hofnarr132]

I also have had exactly this issue and on a recently updated RPi3:
Linux raspberrypi 4.14.98-v7+ #1200 SMP Tue Feb 12 20:27:48 GMT 2019 armv7l GNU/Linux

If anyone is interested..

I do use an external USB sound card integrated into an HDMI KVM switch. When I removed the "Volume Control (ALSA/BT)" applet from the panel, usage immediately dropped to normal - so obviously some sort of bug there.

Reproducing the issue is a bit hit and miss, but usually requires fiddling around with the volume level and the switching screens on the KVM (which has the effect of disconnecting the USB sound card) and then switching back (reconnecting).

[pelwell]

Any ideas, @spl237?

[spl237]

I suspect it is due to a device which is under ALSA control suddenly vanishing; I would not be surprised for the ALSA volume controller in the plugin to then get into a state whereby it attempts to reconnect in a loop, continually failing, and pushes a core to max load in the process.

The answer is probably "ALSA is a bit flakey", which doesn't come as much of a surprise - I'll do some testing and see if I can repeat this.

[hofnarr132]

Thanks Phil & Simon, appreciated. Yes I guess ALSA and the LxPanel applet are the culprits - happy to take a steer if there is a better alternative. There is another applet “Volume Control” which I have not tested fully, but this does not have the choice of sound outputs on right click (the Analog, HDMI, USB Sound Card list) but instead has menu items that call the mixer controls.

BTW reproduced this on the other RPis on the KVM switch - a 2B, 3B, 3B+ and 3A+.

[spl237]

I suspect the other Volume Control applet will behave in the same way - the code for the Volume/BT applet which is installed by default is a (substantial) set of modifications I applied to the original Volume Control applet. If it doesn't behave the same way, then I've introduced the problem with my mods to it. Which is certainly not impossible... ;) On the other hand, if the problem does also appear with the original Volume Control, then we are probably looking at an issue somewhere in ALSA itself.

[spl237]

I can repeat the problem by inserting a USB audio device, using the panel plugin to select it as the default output, and then unplugging it - CPU load then consistently goes to 25%.

Unfortunately, it is not obvious how to fix this - the problem is that ALSA does not (as far as I can tell) provide any mechanism to inform you that your selected device is no longer valid, so short of some sort of polling approach in the plugin, I can't see a way to tell the sound subsystem to reinitialize and look for an alternative device, which is what needs to happen to fix the abnormal CPU load; the load appears to be in ALSA itself rather than anywhere in the plugin code. Polling might work, but it's ugly and likely to put unnecessary load on the system.

For now, the answer is "don't unplug a USB audio device which is in use", or if you do so, then use the volume selector to choose an alternative device - this causes the CPU load to drop back to zero.

[hofnarr132]

Thanks Simon - appreciate your time on this. I guess this was the expected outcome - the Linux kernel has got a lot more resilient to the USB disconnect/reconnect events, but some of the sub systems built on top (like ALSA) need a little more work ;) I guess there are higher priorities.

Anyway as far as I am concerned this can be closed. My 'fix' is to switch from using the USB audio (I can turn the device off on the KVM switch) and use a TV and set sound over HDMI. Having tested, this seems to work when inputs are switched - in this case I don't thing the audio channel is disconnected, just the output switched, where as the USB device was disconnected from one 'source' and connected to the next available 'source' (which also happens with the keyboard and mouse with this particular KVM). Shame as I had a better quality monitor connected with a HDMI-DVI connector, but no audio.

Thanks again.

Neil

[spl237]

No problem - thank you for understanding that this isn't trivially fixable!

Switching HDMI devices shouldn't produce any such problem, as the HDMI audio output is always available on the Pi, regardless of whether or not a monitor is actually connected, so ALSA doesn't ever find the device missing.

[FishOrBear]

I have the same problem

[yutayu]

lxpanel --profile LXDE-pi writes constantly according to iotop.

[yutayu]

lxpanel --profile LXDE-pi writes constantly according to iotop.

https://www.raspberrypi.org/forums/viewtopic.php?p=1064436
I fixed by disabling wifi panel.
Thanks.

[war59312]

Confirmed. Disable wifi panel and no more high cpu usage.

This needs to be fixed.

[yutayu]

Confirmed. Disable wifi panel and no more high cpu usage.

This needs to be fixed.

:)

[JamesH65]

OK, we seem to have two issues here. The Alsa issue (which can be replicated by pulling a USB audio device out ) is not, as far as we can tell, fixable.

The WIfi one we have been unable to replicate, so will need some sort of sequence of actions that can make it happen before we can progress. Although, it may be that the same problems will appy, with no way of fixing it.

So, if people can provide some sort of mechanism to replicate, we can progress, if not this issue will be closed within 30 days unless further interactions are posted.

[war59312]

It only happens when I create a new account, that is not using the built-in pi account.

I simply created a new account and logged in and then click and interacting a little bit with the wifi panel and a few seconds later it starting using high CPU.

[pelwell]

@spl237 Do you have any thoughts? The non-standard account is clearly something that _could_ trigger odd behaviour.

[spl237]

I suspect it is due to something now being unable to access some hardware due to the relevant group permissions being unavailable in the new account. For reference, any new account should be added to the following groups (substitute the new account name for the $1):

sudo adduser $1 adm
sudo adduser $1 dialout
sudo adduser $1 cdrom
sudo adduser $1 sudo
sudo adduser $1 audio
sudo adduser $1 video
sudo adduser $1 plugdev
sudo adduser $1 games
sudo adduser $1 users
sudo adduser $1 input
sudo adduser $1 netdev
sudo adduser $1 spi
sudo adduser $1 i2c
sudo adduser $1 gpio

I'd suggest trying those lines one-by-one and seeing if the problem goes away. If it does (which I suspect it will), please let us know which line fixed it!

[Moneymaker1968]

sudo chown [user]:[user] -R /home/[user]
solved it for me

[DKFrederik]

I can comfirm that hiding/disabling WiFi panel drops CPU usage on one core from 100/103% down to a tolerate level (3-5%)

[mtolava]

Hi, I Have the same problem.
I have Raspberry 3B version and I use it to play a video with advertisement in my work, It is connected to a TV with HDMI port.
But my Raspberry doesnt fulfil 80% but is really slow.

[xplreitr]

I can report similar behavior. LDXE panel running the cpu at 100%. However, hiding/disabling the WiFi panel did not work for me, but when I added an object to the panel, (the CPUFreq frontend), the process terminated and the pi reverted to historically normal load averages.

[Paraphraser]

See this?

It's from a Raspberry Pi 3B+

$ uname -a
Linux octopi 5.4.51-v7+ #1333 SMP Mon Aug 10 16:45:19 BST 2020 armv7l GNU/Linux

The data being displayed is the Pi's vcgencmd measure_temp (via cron every 5 minutes ➡️ mosquitto ➡️ NodeRed ➡️ InfluxDB ➡️ Grafana).

See those periods above 55C? The cause is "lxpanel" running at 100% CPU.

Tried all manner of things to clobber the pesky little CPU-hogging sucker but nothing seemed permanent, until:

1. sudo raspi-config
2. Boot options
3. Desktop/CLI
4. Choose "console"
5. Finish

6. Reboot

   Now ps -ax | grep lxpanel yields blissful silence.🚮

True, this kills the GUI but I run the Pi headless and only access it via ssh so I really do not care.