Appcenter-cli: Further test-parameter specification for xcuitest

[bm-dschloesser]

Hi,

I was searching for the possibility to run specific uitests only. Is there a possibility in appcenter to specify this in the test-parameter of the appcenter test run xcuitest? There is no documentation for possible values.
For example with the xcodebuild command I can specify -only-testing:<suite> to run only a specific test class specified by <suite>. I tried to run the appcenter command using -p "only-testing:=<suite>" and also with -p "only-testing=<suite>", but appearently all tests were executed in the appcenter.

Is there maybe a different way to trigger only a subset of the defined tests? I also tried run a single test by adding only one test in the scheme that is used in the build, but in the appcenter all tests were executed.

[amchew]

Hi @bm-dschloesser, yes, I'll write the documentation and get back again. Sorry for the delay.

[daniel-varela]

Hi, is there any news on this? I'm having the same problem.
I also tried to exclude some tests in the app scheme, but app center seems not to ignore the excluded tests and run them no matter what. I hoped that the CLI would be able to set a parameter, but as @bm-dschloesser noticed too, the lack of documentation is not helpful :/

[amchew]

Hi @daniel-varela, this is on me, I've placed a to-do for myself to work with @owenniblock and write the documentation by tomorrow. Will update again.

[owenniblock]

While we're working on this - it's worth noting that the --test-parameter values are generally undocumented because they're either experimental or still under test (or internal commands which we use over here). They don't equate to parameters sent to the test runner.

I've asked my colleagues for advise on ways to exclude tests when uploading via the CLI but I don't think it will be possible using this cli parameter.

[owenniblock]

@bm-dschloesser @daniel-varela - I've been discussing this over here and it looks like there's no way to do what you want currently.

I would advise reaching out to us on support via Intercom (the little widget on the bottom right when you're logged in to App Center). There you'll be able to request this as a feature (if you both do this it will carry more weight when we come to triage feature requests).

I believe it's possible to restrict tests being run by removing classes from the test target - if you reach out to us on the support channel - someone should be able to help you (please feel free to send them a link to this issue to help speed up the discussion).

[amchew]

Thanks @owenniblock for the updates! @bm-dschloesser and @daniel-varela, when you reach out to us via support, please mention to be transferred to Amanda Chew, and I'll triage it to the right team.

[GHoward90]

Hello

Just wondered if there's been any update/movement on this feature?

This was a great feature on the Xamarin Test Cloud solution, it would awesome to get native support for it.

Thanks

Greg