Robottelo: Where to put testcases for HotBackup

[pgagne]

I am working on created the stubs for the 6.3 HotBackup feature and am not sure where the best place for the testsuite module is.

The feature revolves around running the katello-backup script. It might also involve doing other things to the satellite to populate it. Although its only stubs for now, I imagine we will have to automate it at some point.

Originally I thought to put the module underrobotello/tests/foreman/cli but the test in there seem to be for testing the hammer cli.

Do we have a place for tests like this? If not should we create one?

[rochacbruno]

robottelo/tests/foreman/sys is my suggestion

[rplevka]

+1 To sys.
It will be the same place as the one for storing stuff like foreman-debug-related tests, etc.

[sghai]

Since HotBackup actually using katello-backup utility/tool. So let's have a name like:

• sys as @rochacbruno suggested
• utils
• tools
  I'm fine with any of these.

[omaciel]

My vote is for robottelo/tests/foreman/backup then inside we can have modules for each area, for example:

• robottelo/tests/foreman/backup/test_hot.py
• robottelo/tests/foreman/backup/test_backup.py
• robottelo/tests/foreman/backup/test_restore.py

[rochacbruno]

@omaciel doesn't it break the semantics?

For example, if in future they add a "backup" button in UI and also a backup endpoint in API and a backup command in Hammer, that tests should be part of specific /ui/api/cli or the /backup path?

As [ui, api, cli] stands for the interface being used to interact with the system, can't we just put that on /cli folder as it can also mean command line interface in general and not only hammer?

[pgagne]

@omaciel @rochacbruno also consider this scenario. For IPv6 I think there will be CLI/UI/API testcases like normal, but there also might be some testcases that fall into the system test scenerio. Not sure how we handle this

[rochacbruno]

foreman/{sys|tools|utils}

[omaciel]

@rochacbruno so your idea is to create a new sys directory and then have api/cli/ui entries in there?

[pgagne]

I am just curious is there a particular reason we group testcases by method of execution (api/cli/ui) and not feature (Host, Host Group, etc). so instead of:

test/foreman/{cli|api|ui}/test_hostcollection.py 

You would have:

test/foreman/host_collection/test_host_collection_{api|ui|cli|etc}.py

This would seem more flexible to me and allow use to easily add testcases that fail outside of the api/cli/ui paradigm

I am not saying we should re-org the entire structure, I was just curious what the background on this is. Does something the the test framework make it so it has to be done this way

[rochacbruno]

@omaciel no, I am suggesting only adding foreman/tests/sys and then put system related tests there.

└── tests
    ├── foreman
    │   ├── api
    │   ├── cli
    │   ├── data
    │   ├── endtoend
    │   ├── installer
    │   ├── longrun
    │   ├── performance
    │   ├── rhai
    │   ├── rhci
    │   ├── sys
    │   │     ├─── test_hot_backup.py
    │   │     ├─── test_ipv6.py
    │   │     ├─── test_foreman_debug.py
    │   ├── ui

[omaciel]

@pgagne early on we used to run our automation based on its end point, and then we switched to using tiers... Your suggestion definitely makes sense to me but I don't think we can afford to change it now :/

@rochacbruno if @pgagne agrees with your suggestion I don't have anything against it.

[pgagne]

@omaciel @rochacbruno the sys directory seems like a good place to me.

[sghai]

I like the layout @rochacbruno suggested. But I feel that we should keep sys directory a place for all foreman tools/utils. like katello-backup, katello-restore, foreman-debug etc.

Also, changes around IPV6 includes as below, so test should fall like:

• subnet page - tests can be covered under test/foreman/{cli|api|ui}I/test_subnet.py
• host page - test can be covered under test/foreman/{cli|api|ui}I/test_host.py

And when it will be supported by satellite-installer then we can plan out something like we did for sys.

└── tests
    ├── foreman
    │   ├── api
    │   ├── cli
    │   ├── installer
    │   │     ├─── test_ipv6.py
    │   │     ├─── test_infoblox.py

Just a thought..

[sghai]

@pgagne : Can we closed this now as stubs are already merged in master ?