Corefreq: Linux 5.5.0 module build failure w/ 1.73.0

[chrisleup]

Hey Cyril,
Just tried to build on 5.5.0 and got this ->
make -j1 -C /lib/modules/5.5.0/build M=/home/cpickert/CoreFreq modules make[1]: Entering directory '/usr/src/linux-headers-5.5.0' CC [M] /home/cpickert/CoreFreq/corefreqk.o /home/cpickert/CoreFreq/corefreqk.c: In function ‘CoreFreqK_Limit_Idle’: /home/cpickert/CoreFreq/corefreqk.c:8685:36: error: ‘struct cpuidle_state’ has no member named ‘disabled’ CoreFreqK.IdleDriver.states[idx].disabled = false; ^ /home/cpickert/CoreFreq/corefreqk.c:8688:36: error: ‘struct cpuidle_state’ has no member named ‘disabled’ CoreFreqK.IdleDriver.states[idx].disabled = true; ^ /home/cpickert/CoreFreq/corefreqk.c:8696:35: error: ‘struct cpuidle_state’ has no member named ‘disabled’ CoreFreqK.IdleDriver.states[idx].disabled = false; ^ make[2]: *** [scripts/Makefile.build:266: /home/cpickert/CoreFreq/corefreqk.o] Error 1 make[1]: *** [Makefile:1693: /home/cpickert/CoreFreq] Error 2 make[1]: Leaving directory '/usr/src/linux-headers-5.5.0' make: *** [Makefile:33: all] Error 2

[cyring]

Hello,
It's indeed a change in kernel 5.5 which requires refactoring & non regression tests with the disabled member struct removed.
Meanwhile you may build by inhibiting CONFIG_CPU_IDLE
https://github.com/cyring/CoreFreq/blob/007d58d4fa25ce78211ee5742cd5f76f84fa6e1c/corefreqk.c#L8677

I need to analyse this change and offer a new 5.5 conditional build.

Will come back shortly to you with a testing version.

Regards

[cyring]

Hello,

Here's the CoreFreq function which should fix the 5.5 kernel compatibility.
PLease copy/past at the following line then fully rebuild.
https://github.com/cyring/CoreFreq/blob/007d58d4fa25ce78211ee5742cd5f76f84fa6e1c/corefreqk.c#L8674

static long CoreFreqK_Limit_Idle(int target)
{
    long rc = -EINVAL;
#ifdef CONFIG_CPU_IDLE
    int idx, floor = -1;

    if ((target > 0) && (target <= CoreFreqK.IdleDriver.state_count))
    {
    for (idx = 0; idx < CoreFreqK.IdleDriver.state_count; idx++)
    {
        if (idx < target) {
#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 5, 0)
        CoreFreqK.IdleDriver.states[idx].flags &= !CPUIDLE_FLAG_UNUSABLE;
#else
        CoreFreqK.IdleDriver.states[idx].disabled = false;
#endif
        floor = idx;
        } else {
#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 5, 0)
        CoreFreqK.IdleDriver.states[idx].flags |= CPUIDLE_FLAG_UNUSABLE;
#else
        CoreFreqK.IdleDriver.states[idx].disabled = true;
#endif
        }
    }
    rc = 0;
    }
    else if (target == 0)
    {
    for (idx = 0; idx < CoreFreqK.IdleDriver.state_count; idx++)
    {
#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 5, 0)
        CoreFreqK.IdleDriver.states[idx].flags &= !CPUIDLE_FLAG_UNUSABLE;
#else
        CoreFreqK.IdleDriver.states[idx].disabled = false;
#endif
        floor = idx;
    }
    rc = 0;
    }
    if ((Proc->OS.Gate != NULL) && (floor != -1)) {
    Proc->OS.Gate->OS.IdleDriver.stateLimit = 1 + floor;
    }
#endif /* CONFIG_CPU_IDLE */
    return (rc);
}

• If your Processor is an Intel with C-states capabilities, you can do non regression tests of the idle states limitation. As mentioned in the FAQ section of the READMD.md, you'll have to blacklist any idle mainstream drivers and let corefreqk.ko registers as the CPU Idle driver.
  AMD is not implemented.

[cyring]

I have upgraded to Kernel 5.5; CoreFreq can build with above lines but can not limit the C-States any more

[cyring]

Among the 5.5 commits, here is part of the refactoring :
https://github.com/torvalds/linux/commit/cbda56d5fefcebc01448982a55836c88a825b34c ; https://github.com/torvalds/linux/commit/ba1e78a1dc0ca3e92f0be82279e6ba24177af7d6 ;
https://lkml.org/lkml/2019/11/21/849

cpuidle_driver_state_disabled() appears to replace the disable field but this function is not an exported symbol that I can link with.

I then tried to set :

• the disable field of states_usage struct with CPUIDLE_STATE_DISABLED_BY_DRIVER which leads to an Oops.
• CPUIDLE_FLAG_UNUSABLE which give any result after registration. I mean on the fly changed.

My feature is broken since this major kernel change.

[cyring]

The Idle State Limiter is now fixed for Kernel 5.5
If that's ok for you, feel free to close this issue
Regards

[chrisleup]

Hi Cyril,

I've pulled the recent 1.73.5 build and am having compile issues again when I try to test.

/home/cpickert/CoreFreq/corefreqk.c:233:26: error: initialization of ‘int (*)(struct cpufreq_policy_data *)’ from incompatible pointer type ‘int (*)(struct cpufreq_policy *)’ [-Werror=incompatible-pointer-types]
  /*MANDATORY*/ .verify = CoreFreqK_Policy_Verify,
                          ^~~~~~~~~~~~~~~~~~~~~~~
/home/cpickert/CoreFreq/corefreqk.c:233:26: note: (near initialization for ‘CoreFreqK.FreqDriver.verify’)
/home/cpickert/CoreFreq/corefreqk.c: In function ‘CoreFreqK_Policy_Verify’:
/home/cpickert/CoreFreq/corefreqk.c:9185:36: error: passing argument 1 of ‘cpufreq_verify_within_cpu_limits’ from incompatible pointer type [-Werror=incompatible-pointer-types]
   cpufreq_verify_within_cpu_limits(policy);
                                    ^~~~~~
In file included from /home/cpickert/CoreFreq/corefreqk.c:22:
./include/linux/cpufreq.h:448:62: note: expected ‘struct cpufreq_policy_data *’ but argument is of type ‘struct cpufreq_policy *’
 cpufreq_verify_within_cpu_limits(struct cpufreq_policy_data *policy)
                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~
cc1: some warnings being treated as errors
make[2]: *** [scripts/Makefile.build:266: /home/cpickert/CoreFreq/corefreqk.o] Error 1
make[1]: *** [Makefile:1693: /home/cpickert/CoreFreq] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-5.5.3'
make: *** [Makefile:33: all] Error 2

Tested on debian buster and ubuntu 19.10

[chrisleup]

Also, definitely unrelated, but on Clear Linux I'm getting this ->

make -j1 -C /lib/modules/5.5.2-903.native/build M=/root/CoreFreq modules
make[1]: Entering directory '/usr/lib/modules/5.5.2-903.native/build'
  CC [M]  /root/CoreFreq/corefreqk.o
as: unrecognized option '-mbranches-within-no-boundaries'
make[2]: *** [scripts/Makefile.build:266: /root/CoreFreq/corefreqk.o] Error 1
make[1]: *** [Makefile:1693: /root/CoreFreq] Error 2
make[1]: Leaving directory '/usr/lib/modules/5.5.2-903.native/build'
make: *** [Makefile:33: all] Error 2

Being tracked elsewhere (sorta) .. They just went to 5.5 but also this deals with new mitigation processes that showed up in new 'as' & binutils builds ..

https://github.com/clearlinux/distribution/issues/1728
https://github.com/clearlinux/distribution/issues/1725

[cyring]

Impacts start here:
https://elixir.bootlin.com/linux/v5.5.3/source/include/linux/cpufreq.h#L152

And that:
https://linuxreviews.org/Linux_Distributions_Running_On_Intel_CPUs_Are_About_To_Get_A_Lot_Slower

[cyring]

Hello,
I have not upgraded to 5.5.3 yet but if you want to move on with a fix, just replace the code below at the following lines:
https://github.com/cyring/CoreFreq/blob/f447e2f5974baf17c31fe43fa5b617b6344aa16f/corefreqk.h#L3646

#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 5, 3)
static int CoreFreqK_Policy_Verify(struct cpufreq_policy_data *policy) ;
#else
static int CoreFreqK_Policy_Verify(struct cpufreq_policy *policy) ;
#endif

https://github.com/cyring/CoreFreq/blob/f447e2f5974baf17c31fe43fa5b617b6344aa16f/corefreqk.c#L9182

#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 5, 3)
static int CoreFreqK_Policy_Verify(struct cpufreq_policy_data *policy)
#else
static int CoreFreqK_Policy_Verify(struct cpufreq_policy *policy)
#endif
{
    if (policy != NULL) {
        cpufreq_verify_within_cpu_limits(policy);
    }
    return (0);
}

I should commit by the end of the day ...

[chrisleup]

Pulled a fresh 1.73.6 build with a brand new 5.5.5 kernel and everything builds/makes perfectly again, also, I do have a proper signing process for you on the other active issue I'll post up.

[chrisleup]

Hey Cyril,
Just back to let you know there'll be a few changes you may have to deal with concerning kernel 5.6 .
Newest RC gave me the following after a fresh git pull ..

/home/corefreq/CoreFreq/corefreqk.c: In function ‘SystemRegisters’:
/home/corefreq/CoreFreq/corefreqk.c:5160:11: error: ‘MSR_IA32_FEATURE_CONTROL’ undeclared (first use in this function); did you mean ‘MSR_MISC_FEATURE_CONTROL’?
    : "i" (MSR_IA32_FEATURE_CONTROL)
           ^~~~~~~~~~~~~~~~~~~~~~~~
           MSR_MISC_FEATURE_CONTROL
/home/corefreq/CoreFreq/corefreqk.c:5160:11: note: each undeclared identifier is reported only once for each function it appears in
make[2]: *** [scripts/Makefile.build:268: /home/corefreq/CoreFreq/corefreqk.o] Error 1
make[1]: *** [Makefile:1683: /home/corefreq/CoreFreq] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-5.6.0-rc5'
make: *** [Makefile:38: all] Error 2

I always want to thank you for your tremendous efforts and great software package, we all truly appreciate all of the time and sweat you put into this! :-)

[cyring]

Thanks a lot for letting me know in advance. I'm still wondering why such major MSR is disappearing from Kernel source code. Probably, they will rename it: it happened once these last 5 years.
With the help of a few macros, I should fix the problem ...

I appreciate a lot your support; thanks again.

[chrisleup]

Looks like its probably just MSR_MISC_FEATURE_CONTROL =)
Only place it's remained IA32 is ->

root@oro:/usr/src/linux-5.6-rc5# grep -r 'MSR_IA32_FEATURE_CONTROL' .
./tools/power/x86/turbostat/turbostat.c:                fprintf(outf, "cpu%d: MSR_IA32_FEATURE_CONTROL: 0x%08llx (%sLocked %s)\n",

and that's probably just an oversight =

[cyring]

https://elixir.bootlin.com/linux/v5.6-rc5/source/arch/x86/include/asm/msr-index.h#L565

MSR_IA32_FEAT_CTL is the new name for MSR_IA32_FEATURE_CONTROL. Same 0x0000003a register address

[cyring]

Rename the
MSR and its bit defines to abbreviate FEATURE_CONTROL as FEAT_CTL to
make them a little friendlier on the eyes.

The change appears @

Enable SGX...

When I'll have an access to a SGX capable processor, I'll try to reflect this feature status in _CoreFreq_

[cyring]

Fyi, last master, version 1.74 , should fix all the above issues.

[chrisleup]

Pulled 1.74.1 and built against kernel 5.6-rc6, worked like a champion! Thank you Cyril, I truly always your help and effort! Closing the issue! :-)