Zephyr: DTS metadata should be made available to Kconfig

[SebastianBoe]

Note that this is now one of the tasks listed in #8499.

The list of sub-tasks in this particular tasks, from my conversations (@SebastianBoe feel free to add others here):

• [ ] Modifying the CMake build system so that DT is processed before Kconfig
• [ ] Defining a template of Kconfig option that DT can generate so that Kconfig options can depend on. i.e. DT creates and enables CONFIG_HAS_HW_SPI and Kconfig depends on it for CONFIG_SPI
• [ ] Define and implement the interaction of DT with the build system in general
• [ ] Remove all references to Kconfig symbols from DTS

Original description:

How your SW is configured and what SW is available is dependent on what your HW looks like. It is not the other way around, that your board's HW description depends on what kind of SW you want.

Therefore we need to move DTS processing from after Kconfig to before Kconfig. This requires a lot of effort and will break all manner of things. But I believe it will result in a simpler and better integrated configuration system in the end.

A configuration architecture that this would enable (if we also start feeding DTS information into Kconfig somehow):

Use DTS to express what HW is present and feed this immutable information into Kconfig.
Use a (HW-support pruned) Kconfig tree to express that the user can enable and configure SW according to the immutable constraints of the HW.

The idea of feeding HW-information into Kconfig is not new. It is currently being done with BOARD and ARCH. This proposal simply extends the amount of HW information that is imported to take greater advantage of DTS information. When DTS information is only available after all SW configuration has completed it's usefulness in the configuration system is very minimal.

[carlescufi]

Tentatively added assignees in order to get feedback from them.

[ulfalizer]

There's an upcoming Kconfig feature (that I've had some input in) that might come in handy here.

It's a kind of simple preprocessor (that runs during parsing), where you can do stuff like this:

config FOO
        string
        default "$(shell some-cmd --foo --bar)"

After expansion, that would turn into e.g.

config FOO
        string
        default "output from some-cmd"

Things then work like they usually do. It turns out that things get super complex if you don't limit it to a simple preprocessor.

It would also support a "$(success some-cmd)" function for checking the exit status of some-cmd (which would expand to either "n" or "y"), and variables (think make variables).

I'm waiting for it to get added to the C tools before I add it to Kconfiglib. We've been arguing some details.

[SebastianBoe]

Is there an RFC or similair for this?

I'm not too worried about the mechanism for injecting DTS symbols into Kconfig. That seems trivial to me, the biggest effort is going to be porting DTS sources to not rely on Kconfig as it does today.

That effort is so large, that there must be broad agreement that this makes sense. If not we are just going to have to accept and work-around that Kconfig can not configure based on information in DTS.

Workarounds could include;

Keep information that we want Kconfig to have access to out of DTS sources (limit the scope of DTS).
Duplicate DTS information in Kconfig.
Generate DTS from Kconfig.

[erwango]

@SebastianBoe , can we consider shield case (#6837, even if not merged yet) in this discussion, where a Kconfig value that enable use of a particular shield (CONFIG_SHIELD_XXX=y) adds shield.overlay to board dts file ?

[SebastianBoe]

The shield use-case will not be possible as it is implemented today.

In the new architecture one would have to use another mechanism than Kconfig to attach shield description's to board descriptions.

[ulfalizer]

@SebastianBoe
See https://lkml.org/lkml/2018/5/17/97 for the latest version of the patchset that adds the preprocessor. It has a description too.

[SebastianBoe]

This issue re-appeared when @b0661 tried to use device tree data in https://github.com/zephyrproject-rtos/zephyr/pull/5799.

When device tree is placed after Kconfig everything becomes backwards.

Instead of using Device Tree to describe hardware, and Kconfig to configure SW we end up doing the opposite. E.g. an SoC might use KConfig to declare that SPI is supported CONFIG_SPI=y and an application might use a Device tree overlay to prune the SPI driver.

I'm not able to describe this properly, but I'm convinced that there is something seriously wrong with
the configuration system.

IMHO we need to figure out what we are doing (by resolving https://github.com/zephyrproject-rtos/zephyr/issues/7230) and then figure out what we want to do.

I suspect that we need to be more strict about what we put in DT. It is difficult to be sure, because it is difficult to understand what we are doing right now, but I suspect that if we use DeviceTree as a description language and Kconfig as a configuration language we will end up with a simpler
and more useful system than what we have now.

[erwango]

I suspect that we need to be more strict about what we put in DT

Device Tree proposes a specification to describe hardware and is used in variety of projects (https://www.devicetree.org/specifications/).
HW description should be SW agnostic and we should be able to work with dts files common with Linux for instance. So we should work with existing dts files and dts speficication.

This might appear as a constraint, but I think on the contrary that it will help providing clear and objective guidelines on what should be in DT and what should go in Kconfig.

[erwango]

In the new architecture one would have to use another mechanism than Kconfig to attach shield description's to board descriptions.

ok, as long as this remains doable.

[SebastianBoe]

"HW description should be SW agnostic and we should be able to work with dts files common with Linux for instance. So we should work with existing dts files and dts speficication.

This might appear as a constraint, but I think on the contrary that it will help providing clear and objective guidelines on what should be in DT and what should go in Kconfig."

I agree. We should include in our guidelines that DeviceTree should be application/SW agnostic.

An example of what this would mean in practice is that these files should not exist:

https://github.com/zephyrproject-rtos/zephyr/commit/7a0588ac9f6f29e07ef2b1738048c275c44f42d3

AFAICT those files are doing two things and both of those should be done with Kconfig.

Enabling the UART driver, a software component.

Configuring (not describing) the UART peripheral to use 1Mbit baud rate.
IMHO configuring peripheral address registers belongs to Kconfig, whereas describing the
hardware's supported baudrate configurations belongs in DeviceTree.

Some might object that configuring peripheral address registers belongs to DeviceTree, because
it "sounds like HW description". But it really is different. It is important for the configuration system
that we keep all description in DeviceTree and all configuration in Kconfig.

The contents of peripheral address registers are inherently application-specific, or else the IC-designer
wouldn't have bothered making the register SW configurable. And since they are application-specific they
do not belong in DeviceTree according to the above stated guidelines.

[nashif]

Something like that (very simplified)?

• We add information about hardware availability to Kconfig in the form of CONFIG_HAS_HWXYZ (bool)
• DTS runs and generates a Kconfig overlay file with all present and enabled hardware components int he for of:

CONFIG_HAS_SPI=y
CONFIG_HAS_I2C=y
CONFIG_HAS_BLAH=n

• Kconfig then loads that and create the configuration based on the available hardware.

I dislike the preprocessor approach, this will just make things very complex.

[nashif]

Some might object that configuring peripheral address registers belongs to DeviceTree, because
it "sounds like HW description". But it really is different. It is important for the configuration system
that we keep all description in DeviceTree and all configuration in Kconfig.

Similar to the UART example, the same applies to memory and flash size configuration, that what comes to mind right now but there is probably more, if we go this route, then we are back to square one, i.e. we will be downing in Kconfig hell again.

I personally dislike the the DTS overlays as shown above in the example, it just introduces yet another way to configure the system, in the past we only had Kconfig now we have to deal with another syntax and format.

So, we need somehow to meet in the middle here and see DTS as not a hardware description language, but also as the hardware configuration language that tells the system (and software components) via Kconfig (the proposal in this issue) what hardware is available, to enable/disable hardware dependent components (drivers for example).

[SebastianBoe]

We add information about hardware availability to Kconfig in the form
of CONFIG_HAS_HWXYZ (bool)

Agreed. Kconfig needs to know about available hardware and the _HAS_
pattern works well to communicate this.

DTS runs and generates a Kconfig overlay file with all present and
enabled hardware components

Agreed. Except that shouldn't it suffice with 'present' AKA _HAS_? It
is application-specific and therefore belongs to Kconfig whether a
hardware peripheral/driver is 'enabled'.

Similar to the UART example, the same applies to memory and flash
size configuration, that what comes to mind right now but there is
probably more, if we go this route, then we are back to square one,
i.e. we will be downing in Kconfig hell again.

I am not familiar with the problems of square one or or drowning in
Kconfig hell, but if it means we can start enforcing valid
configurations again and present the application developers with a
Kconfig interface instead of a DTS overlay interface then I would
welcome the challenges of this hell.

[erwango]

"HW description should be SW agnostic and we should be able to work with dts files common with Linux for instance. So we should work with existing dts files and dts specification.
I agree. We should include in our guidelines that DeviceTree should be application/SW agnostic.

I actually meant OS agnostic, my bad for this wrong choice of word which introduced ambiguity.
I think device tree should be used to described the HW and how it is configured (ie registers are populated at boot time).

Enabling the UART driver, a software component.

So this is actually something specified by device tree specification, I don't see a reason why we should not use it. One shade though: in Linux, this is used to enable the driver. Device is in the dt blob at runtime, it is just not activated. In zephyr, this is done slightly differently as device won't be compiled if stated as "disabled".

Configuring (not describing) the UART peripheral to use 1Mbit baud rate.
IMHO configuring peripheral address registers belongs to Kconfig, whereas describing the
hardware's supported baudrate configurations belongs in DeviceTree.

Again, current-speed (as clock-frequency) is part of device tree specification. I don't see a reason not using it.

The contents of peripheral address registers are inherently application-specific, or else the IC-designer wouldn't have bothered making the register SW configurable. And since they are application-specific they do not belong in DeviceTree according to the above stated guidelines.

I disagree, for instance external interrupt or pin-control configuration are done through registers and a useful part of device tree specification. I don't think this criteria is pertinent.

To sum up, my view:

• device tree should be used for HW description and boot configuration (everything that help to populate registers). One advantage to do that is that it provides a specification which limits the abuses and divergences.
• Kconfig should be used for SW configuration (do we want to use IRQ or use pooling, do we enable flow control, everything that will not end up in a device register)...

[nashif]

DT = What
Kconfig = How

[SebastianBoe]

I think device tree should be used to described the HW and how it is
configured (ie registers are populated at boot time).

Describing HW and populating registers at boot time is fine. But in
the cases where a HW configuration knob depends on the application
use-case (which is most registers) then the knob must be controllable
through Kconfig.

So [Enabling the UART driver] is actually something specified by
device tree specification, I don't see a reason why we should not use
it.

Because DT is superior to Kconfig at being a hardware description
language, whereas Kconfig is superior to DT at being a generic
configuration management tool.

Imagine this use-case. A user is developing a Zephyr product. During
development he decides that he needs a new subsystem, so he enables
it through Kconfig and glances over the configuration knobs to see if
there is anything interesting to his application.

This subsystem requires a driver that was not in use before, so
Kconfig enables it and any other dependencies that were lacking.

When configuration of what drivers are enabled is managed by
DeviceTree the user would have to learn somehow that the subsystem
depends on a particular driver, and learn how to write a DeviceTree
overlay file that enables the driver.

The usability difference between these two scenarios is what forces us
to limit the scope of DT to hardware description.

[SebastianBoe]

DT = immutable
Kconfig = mutable

I don't see why we would want to use DeviceTree to configure HW peripherals.

Peripheral configuration is done by SW, it depends on the requirements of SW, it is as soft as SW,
and for all intents and purposes it is SW and therefore under the domain of Kconfig.

[erwango]

This subsystem requires a driver that was not in use before, so
Kconfig enables it and any other dependencies that were lacking.

Then he should not forget to describe this device related to the driver in device tree... He'll have to mess with device tree anyway.
Besides, I don't see why a Zephyr developer would face different issues as a Linux developer on the use case you described.

Only difference between Zephyr and Linux in Kconfig vs DT partition is that Zephyr uses DT at pre-compilation, I think we should focus on what this impacts. And I don't think it impacts HW configuration.

[SebastianBoe]

"Then he should not forget to describe this device related to the driver in device tree... He'll have to mess with device tree anyway."

He doesn't need to, because in this use-case, the driver in question is already implemented and supported in Zephyr. Just not enabled in his application.

"Besides, I don't see why a Zephyr developer would face different issues as a Linux developer on the use case you described."

This was describing an application developer, not a Zephyr developer.

The linux kernel has a comparatively easy configuration problem to solve because they have more resources on-target and can therefore do more at runtime whereas we must do more at build time.

Also, application's do not need to configure and prune the kernel. So the usability of the configuration system is not as important in Linux as it is in Zephyr.

[erwango]

The linux kernel has a comparatively easy configuration problem to solve because they have more resources on-target and can therefore do more at runtime whereas we must do more at build time.

Also, application's do not need to configure and prune the kernel. So the usability of the configuration system is not as important in Linux as it is in Zephyr.

Agreed. I understand that everything that will impact compilation should be in Kconfig, right.
But I don't see why UART_1 "current_speed" has an impact here.
Besides you put aside in this reasonning lot of configuration that could appear as HW related but is actually HW registers SW configured, like gpio, pinmux, external interrupt, clock control ... preventing all these to be in device tree will really limit interest of using device tree. Good luck to configure that with Kconfig (and actually, this is where we're coming from, and the reason why device tree was introduced).

[nashif]

re the UART example, Baudrate is not something static that can only be configured one time and per board, someone might actually want to change the baudrate at runtime, same applies for various frequencies of peripherals. So there is a use case there, however, that does not mean it is an either or question. Baudrate could be set by DTS and reconfigured by the application if needed.

[erwango]

Baudrate could be set by DTS and reconfigured by the application if need

Sure, I think @SebastianBoe and I both meant "boot time" configuration. Then application has the hand to do whatever it need.

[b0661]

The only problem that I had in #5799 is device instantiation, and there only the DTS status = "disabled" case. All other DTS properties that I had to use did fit into the current Kconfig/ DTS scheme and were usefull at least for automatic boot time configuration.

(1) So may be as @SebastianBoe suggested a warning/ error in case Kconfig requests a device (e.g. by CONFIG_GPIO) that is not activated in the DTS could be a good starting point.

This will automatically happen if all devices of a given device type (eg. GPIO) are deactivated. A problem arises if only some are deactivated because in this case the driver gets all the necessary information from the DTS and can not detect that there is some missing (e.g. only GPIOA is deactivated). Here Kconfig would have to be more specific (e.g. CONFIG_GPIO_CONTROLLER_0 - corresponding to the first GPIO of the SoC, here GPIOA).

(2) The other way round - Kconfig prunes a driver of a device that is activated in the DTS - is aready handled.

The device driver will just not be compiled. If Kconfig prunes a device of a device type, but that device is activated in the DTS a warning/ error should be issued.

Assumed situation with drivers that use DTS:

• CONFIG_GPIO is handled by CMake:
  
  
  
  • (1) CONFIG_GPIO=y: no device activated ->compilation errors (missing DTS defines)
    
    -> ok
  
  
  • (2) CONFIG_GPIO=n: some device activated -> no warning/ error
    
    -> ok, application gets what it wants
  
  
• CONFIG_GPIO_CONTROLLER_0 is handled by the driver:
  
  
  
  • (1) CONFIG_GPIO_CONTROLLER_0=y: this device not activated -> compilation errors (missing DTS defines)
    
    -> ok
  
  
  • (1) CONFIG_GPIO_CONTROLLER_0=n: this device activated -> no warning/ error
    
    -> ok, application gets what it wants
  
  
  • (2) the CONFIG_GPIO case applies
  
  

My Conclusion:

• Driver instantiation has to be controlled by Kconfig down to single device selection (CONFIG_GPIO and CONFIG_GPIO_CONTROLLER_x). Single device selection should be done in an SoC independent way.
• CMake shall handle CONFIG_GPIO
• Driver shall follow CONFIG_GPIO_CONTROLLER_x, but not DTS status.
• warning/ error message could be improved
• Situation is not as bad as it looked like

[SebastianBoe]

It is not OK for the configuration system to produce invalid configurations that first show up as compiler errors.

[SebastianBoe]

Can I ask what currently the definition of doing status=disabled is in Zephyr, and how we are using it (in case those two are different).

I know what the DeviceTree definition is. But that is not the one we use in Zephyr AFAICT.

"disabled" Indicates that the device is not presently operational, but it might become operational in the future
(for example, something is not plugged in, or switched off).
Refer to the device binding for details on what disabled means for a given device

e.g. does status=disabled mean the same as CONFIG_SPI=n, or CONFIG_HAS_SPI=n, or something entirely different?

[erwango]

Can I ask what currently the definition of doing status=disabled is in Zephyr, and how we are using it

If a node status is set to "disabled", no define will be generated from extract_dts_include.dts for this node. Hence, if node matching Kconfig symbol is enabled (CONFIG_SPI_1=y) compilation will fail.

[SebastianBoe]

Okay, but we don't want it to work like this do we? We want the configuration system give feedback to the user about the validity of the configuration, not the compiler.

We are not in agreement about peripheral configuration.

But are we in agreement that for enabling and disabling components like drivers we should do as @nashif outlined here:

DTS runs and generates a Kconfig overlay file with all present and enabled hardware components int he for of:
CONFIG_HAS_SPI=y

Then for instance status=disabled would cause CONFIG_HAS_SPI=n to enter the Kconfig system.

The Kconfig system can then use this to hide the SPI driver from the user and in general make sure
that after Kconfig and DTS has completed we end up with a valid configuration.

If we agree on this, then it makes sense to go ahead with moving DTS before Kconfig, because we have an idea of how these two systems will integrate.

[b0661]

Then for instance status=disabled would cause CONFIG_HAS_SPI=n to enter the Kconfig system.

You would also need CONFIG_HAS_SPI_0 otherwise the compilation can still fail.

[erwango]

Then for instance status=disabled would cause CONFIG_HAS_SPI=n to enter the Kconfig system.

Fine with me!
Even if I think I got the big picture, can you clarify the interaction between:
CONFIG_HAS_SPI
CONFIG_HAS_SPI_1 (as proposed by @b0661)
CONFIG_SPI
CONFIG_SPI_1

[b0661]

Even if I think I got the big picture, can you clarify the interaction between

• CONFIG_HAS_SPI: DTS -> Kconfig (any SPI activated)
• CONFIG_HAS_SPI_1: DTS -> Kconfig (SPI_1 activated)
• CONFIG_SPI: Kconfig -> driver code (compile SPI driver)
• CONFIG_SPI_1: Kconfig -> driver code (instantiate SPI_1 device)

Logically CONFIG_HAS_SPI and CONFIG_SPI are not really necessary.

[erwango]

@SebastianBoe

So, IIUC, there would be 2 possible status for a device instance:

• IP not supported (not present in SoC dts) or "status = disabled" => not compiled
• IP supported and "status = okay" => compiled and instantiated

Assuming this is correct, this means we don't have possibility to boot with a non instantiated device that could be started later by application.
Device tree specification mentions, for "disabled": "Indicates that the device is not presently operational, but it might become operational in the future (for example, something is not plugged in, or switched off). Refer to the device binding for details on what disabled means for a given device."

We don't have that today (which is in line with minimal footprint target). Is it ok not to support it in the future?

[SebastianBoe]

Is it ok not to support it in the future?

I imagine that postponing device initialization until the application wants it to start is a valid use-case.

How would you suggest supporting it?

Another Kconfig option perhaps.

CONFIG_SPI_1_UNINIT

[erwango]

My ideal way to deal with that would be to introduce a new "status" in dt (then feed in Kconfig if required):

• ok -> Compiled and initialized
• available -> compiled but not initialized
• disabled -> not supported

So of course, "available" value would be Zephyr specific (at least at first) and we need to see how to introduce it and stay compatible with generic device tree. One way to make it possible is to consider this value would be present only in boards dts files and these files would be only available in Zephyr tree. Only SoC .dtsi files being fully Linux compatible. This is only an idea and has to be digged more in detailed.
Another idea it to add a "zephyr,no-init" property to the device node, which is in line with current dt specification.

By the way, just fyi, the rough way linux is dealing with all that is that there is a generic function in the driver that parses embedded dt blob, extract properties value and instantiate devices depending on the status. I think we should keep in mind that we have different constraints, but we're trying to achieve the same globally, in pre-compilation.

[b0661]

So of course, "available" value would be Zephyr specific (at least at first) and we need to see how to introduce it and stay compatible with generic device tree. One way to make it possible is to consider this value would be present only in boards dts files and these files would be only available in Zephyr tree.

"available" is application specific. So why not put it only in a prj.dts?

[tbursztyka]

I don't think we want to keep instance oriented Kconfig option. That is: anything like *_SPI_0, *_GPIO_3 etc... so definitely not something like HAS_SPI_1. DTS is the one that will generate the instances, Kconfig is the one that know about what general feature are enabled (so not per-instance).
Ideally, in a near future once dts/codegen will generate device instances properly, all instance specific options in Kconfig files could be removed. (and we could infer from that if spi, or i2c, else... are actually enabled: if there is no instance of any spi driver, no need to enable spi).

About this initialization stuff, that will require to adapt the device driver model. Currently, it's all initialized at boot time (following a certain level/prio). We could create a new level that would be UNINITIALIZED. (DTS could handle that relevantly as well) Then you would need to expose a function device_init(struct device *dev) for the app to initialize the device.

That said, is there really a relevant use-case for that kind of delayed initialization? 99% of device init is basically doing super trivial stuff like connecting it's irq line, maybe setting some data to 0 or else. The only complicated ones are, for instance, Ethernet or 15.4 drivers where they start an internal coop thread (usually for handling rx stuff).

[b0661]

I don't think we want to keep instance oriented Kconfig option. That is: anything like *_SPI_0, *_GPIO_3 etc...

You need a way for applications to select the instances to be created. By this you put all the burden to DTS. Is there a concept of a proj.dts to be created by e.g. menuconfig?

[SebastianBoe]

"proj.dts to be created by e.g. menuconfig?"

Wouldn't it be the other way around? Assuming DT is processed before Kconfig one could generate Kconfig from DT.

EDIT: Or are you proposing a GUI-frontend to DT?

[carlescufi]

@b0661 @SebastianBoe we discussed this yesterday with @erwango and @tbursztyka in the TSC call. I've also created #8499 to track this.

Nothing is decided but the current thinking seems to go towards:

1. DT runs first, exposing to Kconfig the available hardware devices including type (driver) and instances. Supposedly it does so by generating Kconfig variables that are injected to Kconfig (CONFIG_HAS_SPI)
2. Users select which features they want, such as CONFIG_SPI with Kconfig. In that case Kconfig would be able to use a dependency to enable that option (depends on HAS_SPI).

We still need discussion on the following I believe:
a) Who enables the actual driver build? probably Kconfig with CONFIG_SPI but which driver in particular to be built could be taken from DT and not Kconfig (through mapping the DT compatible label to a particular .c file).
b) How does the user select which instance(s) to use? I see 2 options here:
b.1) DT does this on its own by generating CONFIG_SPI_0 if the DT file has status = ok and not generating it if the DT file has status = disabled in it.
b.2) DT generates Kconfig options for all the currently enabled (i.e. status = ok in the DT file) instances, and then the user can enable them with Kconfig

[carlescufi]

@b0661 @SebastianBoe @erwango @tbursztyka, some more thoughts on the instancing in KConfig below.

Continuing with issue b) from my previous comment. Once Device Tree is processed before Kconfig, and regarding the way to enable and configure instances by the user, I think there are 2 possible approaches here (as long as we discard b.1 as a real option since in general people seem to be against enabling software features with DT):

1. DT generates "transient" Kconfig options that are not present in any Kconfig.* file but that are "injected" to Kconfig (we could have a Kconfig.dt and Kconfig.dt.defconfig generated in the build folder). Those options would be visible to the user, and in fact the "real" or "permanent" Kconfig options would use depends on with them. We could still maintain those "transient" Kconfig options documented by having the Kconfig documentation generator parse the DT files for each board and generate the help and reference for those.

2. Integrate DT parsing into menuconfig (or write a new dtconfig to configure DT). This is somehow blurrier on how it would look like, but in essence you would be able to visually configure the instancing without having to touch the Device Tree source files. I am not quite sure how this would solve the integration between DT and Kconfig under the hood, but it would certainly prevent the mandatory editing of DT files by users.

[tbursztyka]

I kind of like the proposal 1 which, in turn, would enable changing DT generated instances settings via menuconfig. User would have a unique tool to tweak things. Even non-enabled instance in DT should appear there I think. That would solve the issue of having the user going to modify .dts file in addition of using menuconfig.

Including them would not be too hard, as we could have something like that in, let's say, drivers/spi/Kconfig.stm32

if HAS_DT_SPI_STM32_INSTANCES
source drivers/spi/Kconfig.stm32.dt
endif # HAS_DT_SPI_STM32_INSTANCES

The only drawback is templating/generating these Kconfig.*.dt

[erwango]

I also agree on option1.
I'm still unclear about the point "Even non-enabled instance in DT should appear there I think", but this could be sorted out later.

For Kconfig.*.dt files generation, I think we should think of a new script using same database input as we think codegen should use.
This database would be a file with a structure easily parsable by python script(.pickle, json, ..) keeping most of information from device tree, augmented by yaml bindings.
Then we can even think of a common python extraction module that could used by codegen and the kconfiggen script.

[b0661]

My preferred way would be (I asume this is also option 1):

1) extract_dts_includes.py creates from status and label property (here "GPIOA") of an activated device node:

#define DTS_HAS_GPIO_0 1
#define DTS_HAS_GPIO_0_LABEL "GPIOA"

and

DTS_HAS_GPIO_0=1
DTS_HAS_GPIO_0_LABEL="GPIOA"

Index just follows the number of activated nodes of given driver type (here GPIO). No relation to label to keep it generic.

2) Kconfig reads in this information and creates appropriate configuration values:

config GPIO_GPIOA
   bool "Enable GPIO GPIOA"
   select GPIO_0
   default y

This way the user (and driver) gets the config options with the expected instance names that he knows and that are also specific to the SoC.

Kconfig only contains generic configurations:

menuconfig GPIO
    bool "Enable GPIO driver"
    depends on GPIO_0 || GPIO_1 || GPIO_2 ...

[ulfalizer]

Haven't followed the discussion closely, but as an FYI, a new Kconfig feature (that I had some input in) that might be relevant here has been merged into the C Kconfig tools now (in linux-next, which has stuff that will be in the next release).

It adds a simple preprocessor to Kconfig that (among other things) allows you to get input via shell commands. For example, this sets FOO to the output of the cmd --some --flags command:

config FOO
        string
        default "$(shell,cmd,--some,--flags)"

You can also check whether a command succeeds:

config FOO
        bool "foo"
        depends on $(success,cmd,--some,--flags)

I've been waiting for it to stabilize before I add it to Kconfiglib, but I think it should be pretty close now. If you think it'd be useful here, I could focus on getting it in.

[b0661]

After review of @erwango ´s proposal of a DTS database common to several tools (see #6762) I think the transfer from DTS to Kconfig should be done by a Kconfig.generated configuration file that is read by Kconfig.

This file should contain configuration data for all activated device. No more DTS_HAS_* configuration variables but instead directly creating something like the following:

config GPIO_GPIOA
   bool "Enable GPIO GPIOA"
   select GPIO_0
   default y

This is the general scheme from the review:

```
*.dts -> generate extended -> edtsdb.json --|---> extract kconfig --> kconfig.generated
dts database edtsdb.py |
|---> extract includes --> generated...includes.h
|
|---> Codegen --> generated..sources.c
````

• edtsdb.json: the extended DTS database
• edtsdb.py: A python class to access the database that is imported by all users.
• kconfig.generated: Kconfig configurations for active device instances to be read by Kconfig

[ulfalizer]

The preprocessor also supports macros btw, so could have something like depends on $(dts,foo), if the dts macro expands to a shell command that queries the DTS stuff somehow.

[galak]

We need a consistent view of the hardware, so we can't let Kconfig change hardware options from DT. For example, pins cfg can't be set by Kconfig, because other software like mcuboot, tfm, etc will not see a consistent view of the hardware.

[ulfalizer]

I've been working a bit on the new Kconfig preprocessor stuff (it's in linux-next now), and thought of something that might be useful here:

The new preprocessor lets you write stuff like depends on $(success,some-command), which checks the exit status of some-command and expands to either depends on y or depends on n.

success is actually a custom function (defined in the Kconfig files as well), which expands to a tricky $(shell,...) command (shell is builtin).

It wouldn't be too hard to allow custom functions to be defined in Python in Kconfiglib, and that might make querying DTS less awkard compared to running shell commands.

You could then have stuff like depends on $(dts,SOME_SETTING) = "foo", which would call e.g. a Python dts() function during parsing, which would return the value of SOME_SETTING (or whatever format makes sense for DTS).

Any thoughts?

[mnkp]

For example, pins cfg can't be set by Kconfig

I would like to extend a bit the statement from @galak. As we all know in a typical MCU pins are shared between multiple hardware modules thanks to the service provided by GPIO/pinmux block. However, on many architectures pin muxing is quite inflexible, e.g. on Atmel SAM E70 MCU enabling GMAC module renders USART2 useless.

All these intricate dependencies should be correctly handled by the configuration system, i.e. when configuring driver instances user should expect that enabling two driver instances which are mutually exclusive is not possible. This is not practically implementable in Kconfig, while - as far as I know - should be doable in DTS.

[erwango]

@galak

We need a consistent view of the hardware, so we can't let Kconfig change hardware options from DT. For example, pins cfg can't be set by Kconfig, because other software like mcuboot, tfm, etc will not see a consistent view of the hardware.

So from this I understand all available instances should be generated, then Kconfig can control what instances are compiled, is that correct?

[carlescufi]

@galak @erwango

We need a consistent view of the hardware, so we can't let Kconfig change hardware options from DT. For example, pins cfg can't be set by Kconfig, because other software like mcuboot, tfm, etc will not see a consistent view of the hardware.

Right. So if this is the case, how do we allow users to enable/disable hardware and set boot-time configuration in a friendly manner? Do we then need to create a menudts similar to menuconfig that allows users to set all of this?

So from this I understand all available instances should be generated, then Kconfig can control what instances are compiled, is that correct?

It's more than just the instances, which is fine I think from a DTS perspective. It's the pin mappings and default baudrates and the rest of boot-time config that worries me.

[carlescufi]

@galak @erwango @nashif @b0661 @SebastianBoe

So after a quick chat with @galak on IRC and based on all the info and PRs I've looked at, for me the only solution that maintains DT as the only absolute truth for hardware description and boot-time configuration is to go DT all the way for hardware and peripherals, and not rely on Kconfig at all for those.

So summarizing this new idea:

a) DT runs first, before Kconfig
b) All hardware related configuration is done exclusively on DT (using status and compatible).
c) Kconfig is fed a set of HAS_HW_* or similar options that can be then used to display or hide to the user software features that depend on a particular type of hardware being present

[galak]

Take a look at #9925 as a means to decouple dts from Kconfig.

[galak]

This should be resolved by PR #12108