Firejail: On code tests, why `make install`?

[Fred-Barclay]

I'm trying to learn more about code tests (if anyone's noticed the mess of commits that have been pushed to the gitlab_ci_tests branch :laughing: ) and one thing I don't understand is, why do we use make install -- i.e. the travis build script uses ./configure --prefix=/usr/ && make && make install && make test-travis.

Why not skip installing altogether, i.e. ./configure --prefix=/usr/ && make && make test-travis? This would break a few things like https://github.com/netblue30/firejail/blob/master/test/profiles/profiles.sh#L37
but that could be easily fixed by just using

PROFILES=`ls etc/*.profile`

since the tests would be run inside the firejail source folder anyhow, so all profiles are already reachable in etc/.

Cheers!
Fred

[reinerh]

I'm not exactly sure what the original reason here was.
But installing firejail sets the correct permissions, suid-flag etc.

So when the tests run you are testing an installed version like a user would also have it installed.
There might be differences to the real system when running a binary from the source directory.
(That's also what Debian's CI is doing, running the tests with a real installed package.)

Thanks for caring about tests. I think they are very important for finding regressions.
Unfortunately they are still a bit too flaky and the travis builds are passing, even when there were test errors. Would be awesome if that could be changed.

[chiraag-nataraj]

Seems like the question was answered, even if we're not entirely sure if our testing regime is good enough :stuck_out_tongue_winking_eye:

[reinerh]

@Fred-Barclay I couldn't find your email address (you seem to be using a protected one for commits), so I'll ping you here. :-)

I noticed that in current master there are test failures in your gitlab CI pipeline (while building for fedora).
I like that such things can be noticed, so I thought about creating a similar pipeline on Debian's Gitlab instance (salsa.debian.org), that runs additional Debian-specific tests.

Can you give me some pointers on how you integrated your Gitlab results into the Github repo here?
The .gitlab-ci.yml is probably not sufficient, is it? I would expect that it needs to be registered within Github somehow as well?

[reinerh]

Any maybe also how you set up the mirroring? :)

[Fred-Barclay]

@reinerh Sorry it's been a while but I'll try and remember. Also are you on GitLab? I can just add you to the repo there so you can poke around in the settings if you need.

The first method at https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/github_integration.html looks pretty much like what I remember. If this works for you, then GitLab will setup the mirror automatically.

I don't recall registering the gitlab config with github. IIRC gitlab just watches this repo and pulls changes as they're pushed, then runs the CI on the gitlab repo. So it's not that complicated to do, really. :smile:

If you want a badge, go to the General settings and expand the Badge option.
It will ask you for a link url and a badge image url. These would probably work for you (but you'll probably have to change gitlab.com to salsa.debian.org):

• link: https://gitlab.com/%{project_path}/pipelines/
  
  
  
  • badge image: https://gitlab.com/%{project_path}/badges/%{default_branch}/pipeline.svg
  
  

Sorry I don't remember much more! If you run into a specific issue feel free to ping me here/email me at [email protected]

Cheers!
Fred

EDIT: found and added you on GitLab :wink: