Marlin: The makefile build system is deeply broken

[dgrat]

I tried yesterday to build with Arduino 1.6.10 and 1.6.11 under Linux and Windows with the makefile.
Build was not successful and not even close to that because of path mismatches.
Moreover the other build scripts seem to be broken as well. Maybe it would be smarter to either remove the makefiles entirely or to replace is with something better (cmake?), which solves the dependencies.

[Roxy-3D]

If you can help us get a cmake Makefile going... It will get used!

[thinkyhead]

This is an area that needs the expertise of those who understand each environment. Are you knowledgeable enough to help in this area, and would you like to?

[dgrat]

Okay, then I will contribute some cmake files. Nevertheless, it will probably take some time.
Currently, I am deeply busy with other stuff.

[thinkyhead]

Ain't we all!

[JustAnother1]

I have some experience with cmake (I prefer make for small project though). I'm rather busy right now myself. But if questions come up let me know.

[tohara]

Any specific functionality you are looking for with cmake other than being able to compile?

I do not have any experience with cmake, but I have been reading up on it lately and have set up this to work with marlin:

https://github.com/tohara/marlin-cmake

It is based on arduino-cmake with some modification.

To use it you place marlin-cmake in the root marlin folder, then copy the two files from CMakelists to the Marlin root folder and Marlin src folder.

I have only tested it under ubuntu.

[dgrat]

I thought about to focus on multi-platform build and the upload to the board.
Later also the selection of the correct configuration would be a nice to have.

[tohara]

The version I have set up should work on windows, linux and mac.
It will read the defined motherboard from Configuration.h, then request the board and mcu settings from a setting file that I have populated from boards.h. (still missing some entries)

This will again be used to get the correct settings from the aruino SDK.

Path to any 3rd party hardware or library has to be added to the CMakeList files. I have included a commented sample configuration for Sanguino hardware and U8glib library.

[thinkyhead]

Nice work. We have been looking (hoping, wishing) for a way to get Marlin to build on multiple platforms with make, but our expertise is not great in this area.

@rbartl Please give @tohara's solution a test and see how it does for you. We might decide to just go with that since it covers more platforms.

[Roxy-3D]

The version I have set up should work on windows, linux and mac.

I'm glad it works on Windows!

[rbartl]

@tohara that works quite nicely - build run through with nice progress bars ;).
but the generated hex file is smaller, elf file bigger than the old build.
and it will not boot up on my atmega2560 :(

cmake sizes ->
494844 Sep 4 10:26 ./Marlin/Marlin/Marlin.elf
148146 Sep 4 10:26 ./Marlin/Marlin/Marlin.hex
current Makefile sizes ->
208445 Sep 1 12:22 Marlin/applet/Marlin.elf
352593 Sep 1 12:22 ./Marlin/applet/Marlin.hex

i dont have enough experience with cmake to see what it doesnt like or what is missing.
would you have any tips ?

system is a debian linux and im using arduion 1.6.9

[JustAnother1]

@rbartl can you post the output? Did it compile all the files it needs to compile?

make VERBOSE=1 should also show the compile arguments used.

[rbartl]

@JustAnother1
amount of files seemed ok(?) ;)
liquid, wiring, etc is in the output,

full output is here ->
http://pastebin.com/C2dY11GR

[tohara]

From the output it looks like you are using arduino 1.6.5?

Have you tried setting the arduino sdk path manually in the root cmakelist to your 1.6.9 SDK?

And you have to remove all generated cmake files before you try again.

[tohara]

Are you building from within the root directory?

It is recommended with cmake to use a build folder in the same folder as the root cmakelist.

Or if you want to generate eclipse files in build folder as sibling to the root folder.

[rbartl]

@tohara
you're right - the directory i thought had 1.6.9 was a 1.6.5 version.
changed that (and recreated the build directory from scratch to have no old artifacts)
hex file was not much changed (few bytes difference).
need to try to flash it yet though.

i've started make and cmake from the build folder like this (after copying the CMakelist files).
sadly not enough knowledge about cmake yet, but i read that a build folder should be used.

if [ ! -d marlin-cmake ] ; then
  git clone https://github.com/tohara/marlin-cmake.git
fi


cmake . -DUPLOAD_PORT=/dev/ttyUSB0  -DARDUINO_SDK_PATH=/root/arduino/
make

[JustAnother1]

I looked at the file. I have the impression that it compiles some files twice. That is strange. It also generates a libMarlin, don't know if that is what it should do.

With too small and not functional files my first guess would be a linker problem. The linker is the last step. It generated the files and does this by pulling all needed parts together. Could it be that it thinks that the executable should be a library? Otherwise it would fail with an error if parts of the project have not been compiled.

Then Arduino has this "nice" feature that is adds header files. Missing header files can in rare cases cause trouble with linking. And then it ti mostly C++ nowadays. C++ has some additional complexities,...

[tohara]

@rbartl

This is how i set up my test starting from an empty directory:

git clone https://github.com/MarlinFirmware/Marlin.git
cd Marlin
git clone https://github.com/tohara/marlin-cmake.git
cp marlin-cmake/CMakelists/MarlinSrcDir/CMakeLists.txt Marlin/.
cp marlin-cmake/CMakelists/MarlinRootDir/CMakeLists.txt .
mkdir build && cd build
cmake .. -DUPLOAD_PORT=/dev/ttyUSB0  -DARDUINO_SDK_PATH=/root/arduino/
make

Note that the path passed to cmake is ".."

This will avoid any naming issues or duplicates, and is is very easy to clean the buildfolder with rm -rf *
CMake does not have any clean function.

[JustAnother1]

@tohara Is there a reason what you require at least cmake 3.0 ?

In my opinion the required version should be as low as possible, to cause as little trouble as possible for the users. So are there any features you need that have been introduced in version 3.0 of cmake?

[rbartl]

@tohara nope, that results also in the same files like before :(.
i'll try again with the newest version of arduino.

[tohara]

@rbartl

Did you try flashing after you changed to 1.6.9?

I get similar .hex and .elf size as you, and my atmega2560 booted just fine. I have tested with arduino 1.6.8 and 1.6.11.

[tohara]

@JustAnother1

I don't have any good reason for putting min 3.0 other than that I haven't looked into what the minimum version required should be based on the functions used.

That libmega_Marlin comes from functionality from arduino-cmake where folder in the root folder with a header file with the same name is detected as an arduino library. Not sure if this can cause trouble. It didn't for me.

[rbartl]

@tohara tried flashing it now (with stk500), but LCD doesnt look good - and no response to serial comm.
doesnt seem to work in my case.
i've compiled it on my octopi system (running on a normal PC though).
so its a debian jessie.

[tohara]

@rbartl
This is what I get when compiling RCBugFix:
-rwxrwxr-x 1 tom tom 303964 Sep 5 00:11 Marlin.elf
-rw-rw-r-- 1 tom tom 148117 Sep 5 00:11 Marlin.hex

and the size is only a few bytes different from what i get by running a verify in Arduino IDE.

LCD? the size of the .hex file does look too small if you have it set up with a LCD.

If I compile with LCD for my printer (using U8glib) I get:

-rwxrwxr-x 1 tom tom 676117 Sep 3 22:27 Marlin.elf
-rw-rw-r-- 1 tom tom 359077 Sep 3 22:27 Marlin.hex

could you also post the output from the cmake command ?

[rbartl]

attached is the log of the build.
how would i enable the LCD ?..
Some LCD files get compiled.
buildlog.txt

[Roxy-3D]

how would i enable the LCD ?

Towards the bottom of Configuration.h you will see the LCD options. You would un-comment the type of LCD Panel you have.

//
// LCD TYPE
//
// You may choose ULTRA_LCD if you have character based LCD with 16x2, 16x4, 20x2,
// 20x4 char/lines or DOGLCD for the full graphics display with 128x64 pixels
// (ST7565R family). (This option will be set automatically for certain displays.)
//

[rbartl]

@toharo
it works now.
problem was me being stupid and after starting from scratch not replacing the default Configuration.h ;).
Marlin.hex is now flashed and seems to work.
i've created a small branch that fetches the build script like in your example.
and i removed the default Configuration.h ;)
https://github.com/rbartl/Marlin/tree/cmake_build

Are the GPL from Marlin and MPL from the cmake scripts compatible without troubles ?
not sure after reading the FAQ -> https://www.mozilla.org/en-US/MPL/2.0/FAQ/

[Roxy-3D]

Are the GPL from Marlin and MPL from the cmake scripts compatible without troubles ?

My guess would be that they are. But if we are worried about it, can we just have the executable cmake files here? In my case, I most certainly don't want to build CMake. I'm on Windows. I just want to use it.

[thinkyhead]

I most certainly don't want to build CMake. I'm on Windows. I just want to use it.

https://cmake.org/download/

[tohara]

The way I read the FAQ it should be OK.
Only 3 files contains code copied from arduin-cmake:
Arduino.cmake
Arduino-SDK.cmake
ArduinoToolchain.cmake

These files are just used to compile the marlin code and are not itself compiled and distributed as an executable.
Even if it was it should be OK as long as the individual files containing MPL licensed code is still available under MPL.

[thinkyhead]

GPL is a-ok with any license that doesn't remove any freedoms guaranteed by GPL.

[tohara]

We could do like I have done here: https://github.com/tohara/Marlin/tree/RCBugFix

I have added only the two CMakeLists.txt files in the Marlin repo, and in the root CMakeLists.txt I have added links to download the script files from https://github.com/tohara/marlin-cmake on first run.

That way it will be kept as two individual software sources.

[thinkyhead]

That way it will be kept as two individual software sources.

Great! That's just how it should be.

[rbartl]

FYI:
works also on OSX. just tried (but without flashing it though)

osx Setup ->
brew install cmake & download arduino & unzip to ~/devel/
mkdir build ; cmake .. -DARDUINO_SDK_PATH=~/devel/Arduino.app/Contents/Java/

windows didnt like it though. some popup about "ld" not wanting to run. (but thats probably my funny VM ;) )

[tohara]

As I mentioned earlier it "should" work on windows, but I haven't tested it yet, it is on my ToDo list. ;)
Finally found a VM image that I can use to test/ debug.
I will let you know when I get it working.

[tohara]

@rbartl

I got the same thing. "ld.exe stopped working".
But this does not actually affect the generating of makefiles and compiling. The problem is that by default the cmake generator on windows will be Visual Studio or NMake. You need specify the generator when running cmake:

cmake .. -G"Unix Makefiles"

And you need a GNU make application registered in System Path.

For my test I downloaded make.exe from here and copied to the System32 folder:
http://www.equation.com/servlet/equation.cmd?fa=make

I am using Windows 7 x86

I trying to figure out a way to get around the ld.exe issue, it seems to be a know issue at least for Win7 x86 and x64.

[tohara]

Looks like the problem with "ld.exe stopped working" is in CMakeDetermineCompilerId.cmake if CMAKE_${lang}_FLAGS is not set. I have made workaround for this by setting this variable with a temporary flag until the Platform module is called.

[boelle]

@thinkyhead could that workaround work in the long run?

[thinkyhead]

@boelle F*** if I know. Windows is anathema in my universe.

[boelle]

yeah.... i never understood why the makefiles were there anyway...

but as the noob i am i guess its for those that do not use the arduino IDE

[tohara]

I think i found a cleaner way to solve this by setting the first test flag for the compiler ID function.

elseif(WIN32)
        #set flag -c for compiler ID test to avoid error message from linker.
        set(CMAKE_C_COMPILER_ID_TEST_FLAGS_FIRST -c CACHE INTERNAL "")
        set(CMAKE_CXX_COMPILER_ID_TEST_FLAGS_FIRST -c CACHE INTERNAL "")
        include(Platform/WindowsPaths) 
endif()

This will skip the linker stage for the test compile when cmake is trying to detect the compiler ID and version.

[boelle]

@dgrat any cmake files in the pipe?

[dgrat]

@boelle
I guess, I have some time in end of november to start/continue with this.
Atm I am unfortunately busy with lab-work.

[boelle]

np... i don't think @thinkyhead will run dry of work on this side of xmas anyway.

[mtowara]

for me (linux64) the Makefile works perfectly fine and I would regret seeing it go away (if there is a CMake version I'm of course ok with that).

The only thing which needs fixing in my setup is that cpp files need to be compiled with -std=gnu++11 (which the IDE seems to do by default but leads to errors with the makefile). Should I do a PR for that?

[Roxy-3D]

Should I do a PR for that?

Please do! It will be appreciated!

[thinkyhead]

I would regret seeing it go away

The Makefile is here to stay, as long as there are those willing to keep it up to date with the latest build tools and OS environments.

Thanks for your contribution, @mtowara