Fastlane-plugin-test_center: Parallelization Bug: only the last completed test run block's tests are reported .test_result bundle

[ishammoho]

New Issue Checklist

• [ ] Updated fastlane-plugin-test_center to the latest version
• [ ] I read the README.md
• [ ] I reviewed the example(s) for the action(s) I am using
• [ ] I have removed any sensitive data such as passwords, authentication tokens, or anything else I do not want to world to see

Issue Description

Only the last completed test run block's tests are reported .test_result bundle when running tests using parallelization**

[INSERT OUTPUT HERE]

Environment

[INSERT OUTPUT HERE]

[lyndsey-ferguson]

@ishammoho please test Add support to not collate report files to confirm that this allows you to use the intermediate test_bundles.

[lyndsey-ferguson]

@szymon-kazmierczak, you too please ^

[lyndsey-ferguson]

Fixed in v3.8.0.parallelizing.beta.4

[ishammoho]

when I ask multi_scan to collate the results this is still happening. However when I set collate: false to generate report using each batch reporting part is fine. However if there are re-tried tests the build shows as failed.

[lyndsey-ferguson]

@ishammoho thank you for the information. Yes, I can imagine that the build will show as failed as the uncollated test results will have failed tests in them. Collating will remove failed tests if those same tests passed in a later run.

With collation, I imagine that the way that they are being collated is not compatible with xchtmlreport. If you can create a separate ticket for that issue, I can look at it at a later date.

[jineshqa]

@lyndsey-ferguson as you know xchtmlreport makes report easily consumable with screenshot of failure on the side. Could you look into fixing this issue or suggest a workaround?

[lyndsey-ferguson]

Can you create a ticket with a sample project, Fastfile, what you're expecting in the collated report and what you're getting instead? Thanks!

[jineshqa]

I can create a ticket but not a sample project where I can reproduce the exact error. I was running tests in parallel, 4 sims. Report showed 9 failures in total but xchtmlreport shows only 3.

Should I create a ticket with the info and attach my result bundle?

[lyndsey-ferguson]

Yes please. The main thing is that I need to know what you're expecting and how the collation is not meeting those expectations.