Px4-autopilot: NuttX builds on MacOS and Windows always start from scratch

[MaEtUgR]

Describe the bug
When I successfully build e.g. make px4_fmu-v4_default using the Windows Cygwin toolchain (ninja build tool) and then repeat the same command without changing any file the build starts from scratch:

This makes any development on the hardware unbearable slow...

Expected behavior
It should use the incremental build and if I don't change anything say something like ninja: no work to do.

Log Files and Screenshots

MaEtUgR@Speedy ~/Firmware
$ make px4_fmu-v4_default
[1164/1164] Creating /cygdrive/c/PX4/home/Firmware/build/px4_fmu-v4_default/px4_fmu-v4_default.px4
MaEtUgR@Speedy ~/Firmware
$ make px4_fmu-v4_default
[1164/1164] Creating /cygdrive/c/PX4/home/Firmware/build/px4_fmu-v4_default/px4_fmu-v4_default.px4

Additional context

• make posix only rebuilds one command each time which is not expected but workable.

MaEtUgR@Speedy ~/Firmware
$ make posix
[1/1] Generating ../../logs

• @julianoes told me today that the problem also occurs on MacOS but not on linux.

[MaEtUgR]

I bisected it down to 41ff6c60ba1c54cbd6145d6ae6257b746f710d9f
Before that px4_fmu-v4_default builds 11 commands incrementally without any change after that 1163

[MaEtUgR]

I just saw I ended up with that commit because I was testing fmu-v4, #11672 is the PR with defconfig compression. It probably makes NuttX rebuild every time and because defined by the px4_... CMake macros everything depends on NuttX e.g. https://github.com/PX4/Firmware/blob/be8ad46fc966f526be3d164df9ecb257fc0ea06f/cmake/px4_add_library.cmake#L53 everything is rebuilt.

[dagar]

Can you run 'ninja -d explain' and catch the initial output?

[julianoes]

ninja-explain.txt

See attached log. It looks like everything is dirty somehow.

[MaEtUgR]

I ran ninja -d explain:

ninja explain: restat of output NuttX/nuttx/.config older than most recent input NuttX/nuttx/configs/dummy/Kconfig (1553761559390544300 vs 1553761563037797700)
ninja explain: NuttX/nuttx/.config is dirty
ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty
ninja explain: NuttX/nuttx/include/nuttx/version.h is dirty
ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty
ninja explain: NuttX/CMakeFiles/nuttx_context is dirty
ninja explain: NuttX/nuttx/include/nuttx/version.h is dirty
ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty
ninja explain: NuttX/nuttx/.config is dirty
ninja explain: src/lib/DriverFramework/framework/src/CMakeFiles/df_driver_framework.dir/DriverFramework_NuttX.cpp.obj is dirty
ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty
ninja explain: src/lib/DriverFramework/framework/src/CMakeFiles/df_driver_framework.dir/DevMgr_Nuttx.cpp.obj is dirty
ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty
ninja explain: src/lib/DriverFramework/framework/src/CMakeFiles/df_driver_framework.dir/DFList.cpp.obj is dirty
ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty
ninja explain: src/lib/DriverFramework/framework/src/CMakeFiles/df_driver_framework.dir/SyncObj.cpp.obj is dirty
ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty
ninja explain: src/lib/DriverFramework/framework/src/CMakeFiles/df_driver_framework.dir/Time.cpp.obj is dirty
ninja explain: src/lib/DriverFramework/framework/src/libdf_driver_framework.a is dirty
ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty
ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty
ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty
ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty
ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty
ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty
ninja explain: msg/CMakeFiles/uorb_msgs.dir/topics_sources/uORBTopics.cpp.obj is dirty
ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty
ninja explain: msg/CMakeFiles/uorb_msgs.dir/topics_sources/actuator_armed.cpp.obj is dirty
ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty
ninja explain: msg/CMakeFiles/uorb_msgs.dir/topics_sources/actuator_controls.cpp.obj is dirty
ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty
ninja explain: msg/CMakeFiles/uorb_msgs.dir/topics_sources/actuator_direct.cpp.obj is dirty
ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty
ninja explain: msg/CMakeFiles/uorb_msgs.dir/topics_sources/actuator_outputs.cpp.obj is dirty
ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty
...

I cut the output because it's huge. It lists what looks like all .cpp.obj files with ninja explain: NuttX/nuttx/include/nuttx/config.h is dirty being every second line. At the end there's a smaller normal looking linker dependency tree with .a files. So I need to check why the timestamp gets changed every time. Might be that on linux ninja has more sources to check if a file changed than the timestamp.

[dagar]

What matters is the first bit (.config dirty) that literally everything else is ultimately dependent on. I'll take a closer look when I'm back.

[MaEtUgR]

Fixed, see https://github.com/PX4/Firmware/pull/11777#issuecomment-480297948