Px4-autopilot: separate build and execute commands

[dustingooding]

make posix_sitl_default gazebo not only compiles code, but also automatically executes Gazebo with a model loaded. This is not a good design. Build and execute should be different commands. For one, there's got to be a better way to launch Gazebo than with a Makefile (why not just use sitl_run.sh separately/directly), and two, this prevents any meaningful headless automated building things. I've never seen another project combine both build and execute steps, and especially in such a convoluted way.

[mhkabir]

Build and execute should be different commands

They are.

I've never seen another project combine both build and execute steps, and especially in such a convoluted way.

Maybe that's because you never bothered to search for the proper syntax?
Why don't you just try out make posix. That won't execute Gazebo, and just compile.

this prevents any meaningful headless automated building things

Again incorrect. Have you seen our Travis continuous integration builds that run on every commit?

[dustingooding]

Neither make posix nor the Travis CI builds make the Gazebo integration part. The only thing I've found that does is sitl_run.sh, and that both builds and runs gazebo. From the make front end, I see no way of _just_ building the standard posix build + the Gazebo integration. Likewise, I see no way of _just_ executing the PX4 console + the Gazebo sim (like sitl_run.sh does), without first attempting to build things. And even manually going in to Tools/sitl_gazebo and manually running a CMake build doesn't create the rootfs/rs/microsd directory or the rootfs/eeprom/parameters file. All that logic (how to build the Gazebo integration and how to make the things the Gazebo integration build misses and how to execute the PX4 + Gazebo sim is all wrapped up into a single shell script that's not a main entry point for users. That's what I'm getting at. Those things should be separated and made more easily executable. I shouldn't be expected to start PX4 + Gazebo from make. http://dev.px4.io/simulation-gazebo.html#running-the-simulation

[julianoes]

I see pros and cons for both variants:

Defacto with run commands wrapped by make:

• + You never forget to update the build, all changes are applied when you do a SITL test.
• + It is easy to use, you don't have to know the underlying commands.
• + The underlying commands can be changed under the hood without affecting what the user has to type.
• - It hides command line arguments or even makes them inaccessible (e.g. how to use gdb to start SITL?).

Variant with separate make and run commands:

• + No need to wait for make every time.
• + Allows custom commands, custom setups to run things like SITL.
• - Needs more documentation, commands may change often, and will depend on SITL application, and maybe even on the environment.

Personally, I'd suggest to keep the current way but maybe add more documentation on how to run custom setups.

[dustingooding]

Thanks for reopening @julianoes. My specific use case is ROS-related. I'm attempting to create a single roslaunch file that spins up the Gazebo simulator, the PX4 console, and my companion computer process. I've found a process by which I can get from source code, to built code, to on-demand execution. Basically, I 'make' PX4, then manually CMake the Gazebo sim integration, manually touch a few files/dirs, then (essentially) call 'mainapp' from roslaunch. (I actually had to create a wrapper script for mainapp, because mainapp apparently expects to be executed from a specific directory otherwise it can't find files it needs.) If you'd like specifics on what I did, so the can be included in the proposed new documentation, I'm happy to share.

I still think its weird to execute the software you've built in the same script used to build it, and both of which are executed from make. Like I mentioned previously, I've never seen that. For example, when you build Apache from scratch, it doesn't start the server. And when you want to start your Apache server, you don't need to rebuild it before it executes. Build and Execute are different things and should be separated.

[AndreasAntener]

After going through a similar experience like @dustingooding (for the automatic integration tests) I agree that it would make sense to take compilation of the gazebo/simulation part out of the sitl_run.sh script. I have absolutely nothing against the compile-and-run-it-all approach, very convenient. But the single tasks below that should be split appropriately.

My suggestion would be to move the compilation and generation parts from sitl_run.sh (e.g. https://github.com/PX4/Firmware/blob/master/Tools/sitl_run.sh#L80-L84, same for jMAVSim) to cmake targets and include them in the global build.

The interface would then maybe look like this:

# compile and run it all
make posix_sitl_default gazebo run
# compile it all
make posix_sitl_default gazebo
# compile gazebo parts
make gazebo
# compile px4 parts
make posix_sitl_default

[LorenzMeier]

This is on master now.

[julianoes]

@LorenzMeier is it in the docs?

[LorenzMeier]

Yep, in the Gazebo docs.

[JonasVautherin]

To me it seems like it is not completely solved. When writing scripts, I want to be able to just build everything (PX4 + Simulator) without running anything (and definitely not components relying on the X server).

As far as I understand, we can see e.g. here that gzserver is run no matter what (but note that there is a way to not run gzclient).

Would you guys reconsider changing the build this way? I like the proposition of @AndreasAntener, but it implies changing the way it works right now, which might be inconvenient.

So instead of @AndreasAntener proposition:

# compile and run it all
make posix_sitl_default gazebo run
# compile it all
make posix_sitl_default gazebo

We could also consider just adding a new one without changing the existing target:

# compile and run it all
make posix_sitl_default gazebo
# compile it all
make posix_sitl_default gazebo build

[julianoes]

I like @AndreasAntener's proposition better but @JonasVautherin's idea is a good compromise.

[JonasVautherin]

Me too, actually =). But it is a heavier change in developers' workflow.

In this case, for a transition period, commands such as make posix gazebo should show a warning explaining what to do to start the simulator.