Zephyr: cmake: Use plain CMake instead of Zephyr's custom interface

[tejlmand]

Zephyr are having their own wrapped versions of many CMake commands.
The commands looks similar, many of them are simply re-arranging words you need
to type.

Some drawbacks I see with the current approach are:

• Additional maintenance, both the commands themselves and the documentation
• Additional set of API. A user of Zephyr cannot just learn CMake commands and
  use CMake's online documentation as reference, the user also has to read/learn
  Zephyr additional set of API's.
• Slight re-arranging of words is not hiding complexity, it is just re-arranging
  it
  e.g. zephyr_sources(...) => target_sources(zephyr ...)
  The Zephyr just moves from being target name, to be the function name instead
• Risk associated with the use of macros (scoping)
• Hiding the library name is similar to hide a variable name in a programming
  language.

In CMake you will do:
add_library(MyLib ...)
target_sources(MyLib ....)
target_include_directories(MyLib ...)

This is with slight variation similar to an OO language, where one might have done:
MyLib = new library()
MyLib.sources(...)
MyLib.include_directories(...)
The object name in CMake is just placed as argument.

but in Zephyr, the object/variable name has disappeared:
zephyr_library()
zephyr_library_sources(...)
zephyr_library_include_directories(...)

Example of functions with the words re-arranged,
Instead of writing target_sources(zephyr PRIVATE ...), I have to write
zephyr_sources(), so target name zephyr is just part of the function name.

Instead of target_sources( PRIVATE ...), the target name is hidden and I
need to write zephyr_library_sources(...).

I still need to know the difference between zephyr_sources and
zephyr_library_sources, where CMake just has a single API.

One might argue, that you avoid to know the difference between PRIVATE, PUBLIC, and INTERFACE.
But those keywords are easy and useful to know:
PRIVATE : The content following is only used by the library.
PUBLIC : The content is used by the library and those who links to it.
INTERFACE: The content is only used by those who links to the library.

Zephyr's wrapped commands doesn't completely remove those keywords, it just
places them elsewhere, e.g.
zephyr_library_named() = add_library( STATIC "")
zephyr_interface_library_named() = add_library( INTERFACE)

The use of macros must be done with care:
As example, a user cannot use the variable: name,
as this name is used by the Zephyr macros,
Example, do the following in any CMakeLists.txt, and you'll see the macro
manipulates ${name}.
(CMake functions doesn't suffer from this problem):
set(name "MyName")
message("Value of name=${name}") # The value of name=MyName
zephyr_library()
message("Value of name=${name}") # Now the value of name is something else

I acknowledge that some commands are providing an easier out-of-the box
understanding by providing _ifdef versions, but I believe this can also be
achieved without the drawbacks mentioned above.

List of Zephyr commands that wraps CMake commands and their CMake equivalent:
zephyr_sources = target_sources(zephyr PRIVATE)
zephyr_include_directories = target_include_directories(zephyr_interface INTERFACE)
zephyr_system_include_directories = target_include_directories(zephyr_interface SYSTEM INTERFACE)
zephyr_compile_definitions = target_compile_definitions(zephyr_interface INTERFACE)
zephyr_compile_options = target_compile_options(zephyr_interface INTERFACE)
zephyr_link_libraries = target_link_libraries(zephyr_interface INTERFACE)

zephyr_library() = add_library( STATIC "") # With a library name based upon the folder
zephyr_library_named() = add_library( STATIC "")
zephyr_link_interface() = target_link_libraries( INTERFACE zephyr_interface)
zephyr_library_sources() = target_sources( PRIVATE )
zephyr_library_include_directories(

) = target_include_directories( PRIVATE )
zephyr_library_link_libraries() = target_link_libraries( )
zephyr_library_compile_definitions() = target_compile_definitions( PRIVATE )
zephyr_library_compile_options() = target_compile_options( INTERFACE )
zephyr_interface_library_named() = add_library( INTERFACE)

EDIT: This is part of umbrella issue: #8827

[tejlmand]

Issue #8614 further shows that wrapping of CMake commands limits the feature set of CMake.
And often when such issues arises, then additional wrapped commands gets introduced

[tejlmand]

Additionally, another issue with wrapped function, is that the behavior of the wrapped functions are not stable as their internals can change between commits.
Thus the behavior which were expected at the time of using the zephyr API, might unexpectedly change later.

This is seen in the suggested fix, referred to by this comment:
https://github.com/zephyrproject-rtos/zephyr/issues/8614#issuecomment-401090117

The CMake API is in contrast very stable, and behave consistently.

Also, wrapping if CMake API this way, limits the feature set of CMake's API, and thus, each time such blocked feature is needed, a new zephyr wrapper gets introduced.

[SebastianBoe]

I'm not sure what to make of this, could you change the title to make it clear whether this is a;

feature request,
question,
bugreport,
RFC.

[tejlmand]

Consider it a "reverse" feature request.

The request is to remove the mentioned wrapped functions, and instead stick to default CMake functions, which already provides what is needed.

Only create and maintain zephyr functions where a new functionality is really useful, e.g. like the zephyr check compiler flag functionality.

This will also ease the entry barrier for new comers who already know the CMake language.

By using plain cmake for target handling we avoid having to create new wrapper functions whenever a new CMake keyword / function argument needs to be supported, such as:

8614

And also we avoid e.g. PR's as:

9309

There are probably more out there.
See this as part of: #8827

[meuter]

As a newcomer to zephyr, I can attest that the whole wrapper macro thing is very frustrating.
CMake is arguably difficult to master, but once you do, it gives you a lot of flexibility. Having to learn an extra domain specific language on top of that feels redundant IMHO. I would suggest to stick with vanilla cmake and only supplement where it make sense.

[mped-oticon]

@meuter I agree, but CMake adheres to an opinionated I-will-deduce-everything-for-you mentality that collides with the nitty-gritty specifics required to build an OS. So I see the reason for why things are the way they are. Add to that, the need to support multiple wildly-different toolchains...

If you see improvements, feel free to send a Request-For-Comments pull request.

[meuter]

@mped-oticon I can very well understand the kind of details and complexity you're talking about and indeed it's bit of a culture clash. However, I think a more "transparent" approach would be easier to grasp. Currently the zephyr_xxx() and zephyr_library_xxx() macros are just trying to hide some of the complexity and are not really adding much to what cmake offers out of the box.

Another advantage I see in keeping with the more idiomatic cmake way of doing things is that it would make porting existing libraries to zephyr much easier (most are coming with CMakeLists.txt these days). You'd simply add_subdirectory(/path/to/external/library) and you should be mostly good to go.

In any case, once I have a bit more experience with zephyr (as I said I'm a newcomer), as you suggest I'll send a Request-For-Comments pull request.