Fastlane-plugin-test_center: If use test plans all tests in its class will be retried when only one test in class failed

[vdavydovHH]

New Issue Checklist

• [x] Updated fastlane-plugin-test_center to the latest version
• [x] I read the README.md
• [x] I reviewed the example(s) for the action(s) I am using
• [x] I have removed any sensitive data such as passwords, authentication tokens, or anything else I do not want to world to see

If you love this fastlane plugin, consider sponsoring it or asking your company to sponsor it. I would really appreciate any
gesture: https://github.com/sponsors/lyndsey-ferguson. 😍

Issue Description

I started to use test plans and on my project logs show when one test failed all tests in its class were retried. Could you research that?

It is my multi_scan parameters:

multi_scan(
    clean: true,
    workspace: Project::Path::WORKSPACE_FILE_NAME,
    scheme: "SomeScheme",
    testplan: "SomeTestplan",
    derived_data_path: Project::Path::DERIVED_DATA_FOLDER,
    device: Project::Environment::SIMULATOR,
    parallel_testrun_count: 3,
    try_count: 5,
    code_coverage: false,
    collate_reports: false,
    result_bundle: true,
    output_types: 'junit',
    output_directory: Project::Path::BUILD_FOLDER,
    buildlog_path: "#{Project::Path::BUILD_FOLDER}/Log"
  )

If you need something else, please, let me know

Thank you in advance!

[lyndsey-ferguson]

@vdavydovHH Good morning (that's the time here)! As my time is very limited currently, I won't be able to spend a lot of time investigating. You can help to speed things up by providing some logs, and if I cannot figure out the problem from the logs, I'll need a sample project.

Can you attach the logs by running your lane with the --verbose flag and attach the resulting output here? You can feel free to remove file paths, test names, and any output generated by the compilation phase.

[lyndsey-ferguson]

@vdavydovHH bump ⬆️

[vdavydovHH]

Sorry for the delay, I tried to figure out what the problem is so that you have more information

I suppose that the problem occurs if you disable some tests in the test plan. I have attached logs where you will see how testLaunchPerfomance_4 is called twice.
I also attached a test app where you can reproduce this behavior

logs.zip
MultiScanTestPlans 3.zip

[lyndsey-ferguson]

Ok, thank you!

[lyndsey-ferguson]

@vdavydovHH, can you modify your Pluginfile per my instructions below, run bundle install, and then run your fastlane again (with the --verbose flag)?

Pluginfile:

gem 'fastlane-plugin-test_center', :git => "https://github.com/lyndsey-ferguson/fastlane-plugin-test_center.git", :branch => "issue-237-testplans-retry-passed-tests"

If there are still problems, please let me know and attach the console output as a text file to this issue (makes it easier for me to review). If it works, please let me know.

[vdavydovHH]

Okay
Looks like a fix 👍

[lyndsey-ferguson]

Fixed in v3.11.4