Nixpkgs: NixOS don't use CPUs full potential

Created on 20 Aug 2019 · 13Comments · Source: NixOS/nixpkgs

Describe the bug

I bought a Workstation with Intel i9-9900K CPU.

On Windows or elementary OS (based on Ubuntu), the CPU runs at 4.8 GHz under load. In NixOS it don't goes beyond 4 GHz.

powerManagement.cpuFreqGovernor = "performance"; makes no difference.

To Reproduce
Steps to reproduce the behavior:

buy that 500€ CPU and system that supports it
install windows
run NixOS from installer usb drive
produce load with for i in {1..10}; do while : ; do : ; done & done
see if "CPU MHz" goes beyond 4000

[root@testtst-host:~]# lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
Address sizes:       39 bits physical, 48 bits virtual
CPU(s):              16
On-line CPU(s) list: 0-15
Thread(s) per core:  2
Core(s) per socket:  8
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               158
Model name:          Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
Stepping:            12
CPU MHz:             4000.000
CPU max MHz:         5000.0000
CPU min MHz:         800.0000
BogoMIPS:            7200.00
Virtualization:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            16384K
NUMA node0 CPU(s):   0-15
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp flush_l1d arch_capabilities

root@elementary:~# lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              16
On-line CPU(s) list: 0-15
Thread(s) per core:  2
Core(s) per socket:  8
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               158
Model name:          Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
Stepping:            12
CPU MHz:             4888.011
CPU max MHz:         5000.0000
CPU min MHz:         800.0000
BogoMIPS:            7200.00
Virtualization:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            16384K
NUMA node0 CPU(s):   0-15
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp flush_l1d arch_capabilities

root@elementary:~# uname -a
Linux elementary 4.15.0-36-generic #39-Ubuntu SMP Mon Sep 24 16:19:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Expected behavior
NixOS should use the CPUs max frequency like other linux distros!

Metadata
[root@testtst-host:~]# nix run nixpkgs.nix-info -c nix-info -m

system: "x86_64-linux"
host os: Linux 4.19.62, NixOS, 19.03.git.cc26704 (Koi)
hangs there because the system has no internet access

It's on this branch: https://github.com/NixOS/nixpkgs/pull/66205

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

bug

Source

davidak

👍1

All 13 comments

Do you mind collecting more information about this during your tests? Comes in mind cpupower frequency-info and cpupower monitor.

mmahut on 20 Aug 2019

👍1

Ironically, I had to disable power manager to avoid NixOS hanging (https://github.com/NixOS/nixpkgs/issues/59395) and run 2.7GHz mobile processor at 4.2GHz and I will be happy to slow it down to save battery time

volth on 21 Aug 2019

Ironically, I had to disable power manager to avoid NixOS hanging (#59395) and run 2.7GHz mobile processor at 4.2GHz and I will be happy to slow it down to save battery time

Do a sudo cpupower -c 0-7 frequency-set -u 2.7GHz.

jcumming on 21 Aug 2019

My broadwell (E5-2650v4) has similar problems:

[jack@needle:~]$ sudo cpupower monitor
    | Nehalem                   || Mperf              
 CPU| C3   | C6   | PC3  | PC6   || C0   | Cx   | Freq  
   0|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2500
  12|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2499
   1|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2500
  13|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2500
   2|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2499
  14|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2500
   3|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2500
  15|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2500
   4|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2499
  16|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2500
   5|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2500
  17|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2500
   6|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2500
  18|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2500
   7|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2500
  19|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2500
   8|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2500
  20|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2499 
   9|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2500
  21|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2500  
  10|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2499
  22|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2500
  11|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2500
  23|  0.00|  0.00|  0.00|  0.00|| 99.97|  0.03|  2499

jcumming on 21 Aug 2019

Hmm, i seem to be in the same boat as well, (although my max is 4.2Ghz, so less of a loss).

$ watch -n.1 "cat /proc/cpuinfo | grep \"^[c]pu MHz\"" shows me at only 3.99-4Ghz

Although, I'm okay with that since my temps are around ~80C. Those large nix-reviews really get every ounce out of the silicon :)

jonringer on 21 Aug 2019

Modern Intel CPUs, which run on the intel_pstatedriver, treat frequency limits more like a recommendation. The on-chip thermal management unit eventually decides about the (max) clock frequency. One reason may be that more than one core is active (see here).

@davidak Can you get a higher clock frequency with a different Linux distribution?

markuskowa on 21 Aug 2019

@mmahut i tested again with nixos-minimal-19.09pre189585.1412af4b2cf-x86_64-linux.iso

I have one process running which generates 100% load on one core: while : ; do : ; done

[nixos@nixos:~]$ nix run nixpkgs.linuxPackages.cpupower -c sudo cpupower frequency-info
analyzing CPU 0:
  driver: intel_pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency:  Cannot determine or is not supported.
  hardware limits: 800 MHz - 5.00 GHz
  available cpufreq governors: performance powersave
  current policy: frequency should be within 800 MHz and 4.00 GHz.
                  The governor "performance" may decide which speed to use
                  within this range.
  current CPU frequency: Unable to call hardware
  current CPU frequency: 4.00 GHz (asserted by call to kernel)
  boost state support:
    Supported: yes
    Active: yes

[nixos@nixos:~]$ nix run nixpkgs.linuxPackages.cpupower -c sudo cpupower monitor
    | Nehalem                   || Mperf              || Idle_Stats                                                   
 CPU| C3   | C6   | PC3  | PC6   || C0   | Cx   | Freq  || POLL | C1   | C1E  | C3   | C6   | C7s  | C8   | C9   | C10   
   0|  0.00|  0.78|  0.00|  0.00||  0.06| 99.94|  4000||  0.00|  0.00|  0.00|  0.00|  0.00|  0.00|  0.40|  1.18| 98.39
   8|  0.00|  0.78|  0.00|  0.00||  0.09| 99.91|  3999||  0.00|  0.53|  0.02|  0.00|  0.83|  0.00|  0.79|  0.00| 97.75
   1|  0.01|  0.15|  0.00|  0.00||  0.00|100.00|  3947||  0.00|  0.00|  0.00|  0.00|  0.09|  0.00|  0.00|  0.00| 99.91
   9|  0.01|  0.14|  0.00|  0.00||  0.00|100.00|  3999||  0.00|  0.00|  0.00|  0.00|  0.00|  0.00|  0.00|  0.00|100.00
   2|  0.00|  0.00|  0.00|  0.00|| 99.95|  0.05|  3998||  0.00|  0.00|  0.00|  0.00|  0.00|  0.00|  0.00|  0.00|  0.00
  10|  0.00|  0.00|  0.00|  0.00||  0.00|100.00|  3972||  0.00|  0.00|  0.00|  0.00|  0.00|  0.00|  0.00|  0.00|100.00
   3|  0.01|  0.83|  0.00|  0.00||  0.06| 99.94|  3996||  0.00|  0.00|  0.00|  0.00|  0.08|  0.00|  0.00|  0.00| 99.86
  11|  0.01|  0.81|  0.00|  0.00||  0.02| 99.98|  3954||  0.00|  0.00|  0.00|  0.00|  0.84|  0.00|  0.81|  0.00| 98.34
   4|  0.00|  0.00|  0.00|  0.00||  0.01| 99.99|  3997||  0.00|  0.00|  0.00|  0.00|  0.00|  0.02|  0.00|  0.00| 99.97
  12|  0.00|  0.00|  0.00|  0.00||  0.01| 99.99|  3997||  0.00|  0.00|  0.00|  0.00|  0.00|  0.00|  0.00|  0.00| 99.99
   5|  0.00|  0.42|  0.00|  0.00||  0.06| 99.94|  3999||  0.00|  0.00|  0.00|  0.00|  0.28|  0.00|  0.10|  0.00| 99.57
  13|  0.00|  0.40|  0.00|  0.00||  0.01| 99.99|  3971||  0.00|  0.00|  0.00|  0.00|  0.05|  0.00|  0.00|  0.00| 99.94
   6|  0.06|  0.28|  0.00|  0.00||  0.01| 99.99|  3971||  0.00|  0.00|  0.00|  0.00|  0.27|  0.00|  0.00|  0.00| 99.72
  14|  0.06|  0.27|  0.00|  0.00||  0.00|100.00|  3964||  0.00|  0.00|  0.00|  0.05|  0.10|  0.00|  0.00|  0.00| 99.85
   7|  0.06|  0.09|  0.00|  0.00||  0.00|100.00|  3999||  0.00|  0.00|  0.00|  0.00|  0.00|  0.00|  0.00| 12.84| 87.16
  15|  0.06|  0.08|  0.00|  0.00||  0.00|100.00|  3953||  0.00|  0.00|  0.00|  0.05|  0.00|  0.00|  0.00|  0.00| 99.95

@markuskowa thanks for the link, very informative.

even with just one process at 100% load i don't get beyond 4.00 GHz! The one other linux distro i tested (elementary OS, based on Ubuntu) used up to 4,68 with 16 threads at 100%, like windows. so it's a NixOS issue and not a hardware or UEFI settings issue

davidak on 21 Aug 2019

Hrm. My laptop (skylake) (i7-6820HQ) running 19.09.git.51186de does turbo to what ark.intel.com says it should (3.6GHz). Under a single threaded load:

     | Nehalem                   || Mperf              || Idle_Stats                                                   
 CPU| C3   | C6   | PC3  | PC6   || C0   | Cx   | Freq  || POLL | C1   | C1E  | C3   | C6   | C7s  | C8   | C9   | C10   
   0|  0.00|  0.00|  0.00|  0.00|| 98.31|  1.69|  3603||  0.00|  0.00|  0.00|  0.00|  0.00|  0.00|  0.00|  0.00|  0.00
   4|  0.00|  0.00|  0.00|  0.00||  0.05| 99.95|  3182||  0.00|  0.00|  0.00|  0.00|  0.57|  0.00|  0.70|  0.00| 98.62
   1|  0.01|  1.18|  0.00|  0.00||  0.18| 99.82|  3001||  0.00|  0.00|  0.01|  0.00|  0.00|  0.00|  3.28|  0.53| 95.91
   5|  0.01|  1.18|  0.00|  0.00||  0.18| 99.82|  2988||  0.00|  0.01|  0.00|  0.00|  1.24|  0.00|  2.57|  0.00| 95.93
   2|  0.00|  1.89|  0.00|  0.00||  0.22| 99.78|  2980||  0.00|  0.23|  0.08|  0.00|  1.69|  0.00|  1.78|  0.00| 95.87
   6|  0.00|  1.89|  0.00|  0.00||  0.10| 99.90|  3070||  0.00|  0.00|  0.00|  0.00|  0.29|  0.00|  0.79|  0.74| 98.01
   3|  0.03|  0.13|  0.00|  0.00||  0.02| 99.98|  2770||  0.00|  0.00|  0.00|  0.00|  0.15|  0.00|  0.00|  0.00| 99.77
   7|  0.03|  0.13|  0.00|  0.00||  0.18| 99.82|  3103||  0.00|  0.00|  0.00|  0.04|  0.00|  0.00|  0.00|  0.00| 99.69

... might the kernel config require some tuning?

jcumming on 22 Aug 2019

https://bugzilla.kernel.org/show_bug.cgi?id=110941 indicates that it might be microcode updates that are causing this.

jcumming on 22 Aug 2019

69048 fixed the problems on my E5-2650v4.

jcumming on 20 Sep 2019

Does #69048 fix the problem for everyone?

Not that I'm qualified to help. I'm just wondering if this is fixed.

bbigras on 29 Jan 2020

😄1

Hello, I'm a bot and I thank you in the name of the community for opening this issue.

To help our human contributors focus on the most-relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.

The community would appreciate your effort in checking if the issue is still valid. If it isn't, please close it.

If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me". If you'd like it to get more attention, you can ask for help by searching for maintainers and people that previously touched related code and @ mention them in a comment. You can use Git blame or GitHub's web interface on the relevant files to find them.

Lastly, you can always ask for help at our Discourse Forum or at #nixos' IRC channel.

stale[bot] on 28 Jul 2020

Does #69048 fix the problem for everyone?

don't look like it's fixed :grimacing:

[davidak@gaming:~/code/nixos-homepage]$ nix run nixpkgs.linuxPackages.cpupower -c sudo cpupower frequency-info
[1 copied (0.2 MiB), 0.1 MiB DL]
[sudo] password for davidak: 
analyzing CPU 0:
  driver: intel_pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency:  Cannot determine or is not supported.
  hardware limits: 800 MHz - 5.00 GHz
  available cpufreq governors: performance powersave
  current policy: frequency should be within 800 MHz and 4.00 GHz.
                  The governor "performance" may decide which speed to use
                  within this range.
  current CPU frequency: Unable to call hardware
  current CPU frequency: 4.00 GHz (asserted by call to kernel)
  boost state support:
    Supported: yes
    Active: yes

system: "x86_64-linux"
host os: Linux 5.7.19, NixOS, 20.03.2868.ff6a070b4ef (Markhor)
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.3.6
channels(root): "home-manager-20.03, nixos-20.03.2868.ff6a070b4ef, nixos-hardware, nixos-unstable-20.09pre239318.c59ea8b8a0e"
nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos