Singularity: More thorough instance testing

[GodloveD]

Version of Singularity:

current master

Expected behavior

I think that the instance tests should do the following:

1. Run all of the instance tests in a loop with the various flags that can be used with instances as input.
2. Check for the presence and absence of the temp files that are created by the instance command at the appropriate times (check for cleanup after instance stop)
3. Look at the std out of the instance list command and make sure it returns what is expected.
4. call instance list when no instances are running and verify that the std out returns no running instances.

Actual behavior

Looks like the tests just look at error codes of functions.

[phpHavok]

@GodloveD I'll take this one if the other guys haven't started already.

[jscook2345]

@phpHavok Go for it. Let me know if you have questions. Either here or in Slack.

[phpHavok]

@GodloveD @jscook2345 I've been working on these, and I have a new framework and set of basic tests built for instances. They are tabular now, so easy to add more later like the other test files. Furthermore, I'm testing some actual functionality now like starting a basic echo server and verifying you can contact it over the network. I've also mocked out all of the flags, but I haven't tested most of them yet.

Can you look over what I've done and let me know what you think and which particular start flags you'd like tested? Also, are the temp files you were talking about stuff in /tmp/With(out)Privilege or something else?

My updates are available here: https://github.com/phpHavok/singularity/blob/2129/better_instance_tests/cmd/singularity/instance_test.go

Thanks!

[phpHavok]

@jscook2345 I've added quite a few more changes since my last message, and am now testing several flags. A PR is attached now for review.

[GodloveD]

Thanks for all the hard work @phpHavok !

[jscook2345]

Closed by pull #2416