Fastlane-plugin-test_center: only_testing flag results in undefined method `sort' error
New Issue Checklist
- [x] Updated
fastlane-plugin-test_centerto the latest version - [x] I read the README.md
- [x] I reviewed the example(s) for the action(s) I am using
- [x] I have removed any sensitive data such as passwords, authentication tokens, or anything else I do not want to world to see
Issue Description
I am running multi_scan on my project with the same arguments I was using in scan and I am getting a ruby error that looks related to the only_testing argument I am passing in.
/Users/myuser/.rvm/gems/ruby-2.5.1/gems/fastlane-plugin-test_center-3.5.7/lib/fastlane/plugin/test_center/helper/test_collector.rb:45:in `only_testing_to_testables_tests': [!] undefined method `sort' for "UITestTarget/UITestClass":String (NoMethodError)
[08:48:57]: ------------------------
[08:48:57]: --- Step: multi_scan ---
[08:48:57]: ------------------------
+----------------------+----------------------+
| Summary for multi_scan (test_center v3.5.7) |
+----------------------+----------------------+
| try_count | 3 |
| fail_build | false |
+----------------------+----------------------+
[08:48:57]: $ xcodebuild -showBuildSettings -workspace /Users/me/git/repo/MyWorkspace.xcworkspace -scheme MyTestScheme -configuration DEBUG
[08:49:00]: Command timed out after 3 seconds on try 1 of 4, trying again with a 6 second timeout...
[08:49:02]: $ xcodebuild -showBuildSettings -workspace /Users/me/git/repo/MyWorkspace.xcworkspace -scheme MyTestScheme -configuration DEBUG
+----------------------------+-------------------------------------------------------------------------------+
| Summary for scan 2.105.2 |
+----------------------------+-------------------------------------------------------------------------------+
| clean | true |
| workspace | /Users/me/git/repo/MyWorkspace.xcworkspace |
| scheme | MyTestScheme |
| device | iPhone 8 |
| output_directory | /Users/me/artifactsZ/artifacts/tests |
| only_testing | ["MyTestScheme/MyTestClass"] |
| include_simulator_logs | true |
| result_bundle | true |
| derived_data_path | /Users/me/artifactsZ/DerivedData |
| configuration | DEBUG |
| xcargs | ORG_NAME='feedsdk-cs4b' |
| fail_build | false |
| build_for_testing | true |
| skip_detect_devices | false |
| skip_build | false |
| output_types | html,junit |
| buildlog_path | ~/Library/Logs/scan |
| disable_concurrent_testing | false |
| should_zip_build_products | false |
| open_report | false |
| skip_slack | false |
| slack_only_on_failure | false |
| use_clang_report_name | false |
| xcode_path | /Applications/Xcode/10.0/Xcode.app |
+----------------------------+-------------------------------------------------------------------------------+
[08:49:04]: $ set -o pipefail && env NSUnbufferedIO=YES xcodebuild -workspace /Users/me/git/repo/MyWorkspace.xcworkspace -scheme MyTestScheme -configuration DEBUG -destination 'platform=iOS Simulator,id=685E7851-3B75-416E-B8D8-8906797C5B0D' -derivedDataPath '/Users/me/artifactsZ/DerivedData' -resultBundlePath '/Users/me/artifactsZ/artifacts/tests/MyTestScheme.test_result' ORG_NAME='feedsdk-cs4b' -only-testing:MyTestScheme/MyTestClass clean build-for-testing | tee '/Users/me/Library/Logs/scan/FeedSDKSampleApp-MyTestScheme.log' | xcpretty --report html --output '/Users/me/artifactsZ/artifacts/tests/report.html' --report junit --output '/Users/me/artifactsZ/artifacts/tests/report.junit' --report junit --output '/var/folders/t4/j7prc3m93276n3bd2vbxf4fs8k4yv1/T/junit_report20181002-67829-13wwgxx'
[08:49:04]: ▸ Loading...
[08:49:06]: ▸ Clean Succeeded
[08:49:09]: ▸ Processing Info.plist
... Lots of compile commands in between from xcodebuild
[09:12:08]: ▸ Test build Succeeded
+--------------------+---+
| Test Results |
+--------------------+---+
| Number of tests | 0 |
| Number of failures | 0 |
+--------------------+---+
[09:12:08]: ------------------------------
[09:12:08]: --- Collecting system logs ---
[09:12:08]: ------------------------------
[09:12:08]: $ xcrun simctl spawn 685E7851-3B75-416E-B8D8-8906797C5B0D log collect --output /Users/me/artifactsZ/artifacts/tests/system_logs-iPhone\ 8_iOS_12.0.logarchive 2>/dev/null
[09:12:09]: Exit status: 74
+----------------------------+------------------------------------------+
| Lane Context |
+----------------------------+------------------------------------------+
| DEFAULT_PLATFORM | ios |
| PLATFORM_NAME | |
| LANE_NAME | sweep |
| SCAN_DERIVED_DATA_PATH | /Users/me/artifactsZ/DerivedData |
| SCAN_GENERATED_PLIST_FILES | [] |
| SCAN_GENERATED_PLIST_FILE | |
+----------------------------+------------------------------------------+
+------+---------------------------------------------------------------+-------------+
| fastlane summary |
+------+---------------------------------------------------------------+-------------+
| Step | Action | Time (in s) |
+------+---------------------------------------------------------------+-------------+
| 1 | Verifying fastlane version | 0 |
| 2 | default_platform | 0 |
| 3 | rm -rf /Users/me/artifactsZ/DerivedData || echo "Dele | 1 |
| 4 | rm -rf /Users/me/artifactsZ/artifacts/tests || echo " | 0 |
| 💥 | multi_scan | 1391 |
+------+---------------------------------------------------------------+-------------+
[09:12:09]: fastlane finished with errors
Looking for related GitHub issues on fastlane/fastlane...
Found no similar issues. To create a new issue, please visit:
https://github.com/fastlane/fastlane/issues/new
Run `fastlane env` to append the fastlane environment to your issue
Traceback (most recent call last):
31: from /Users/me/.rvm/gems/ruby-2.5.1/bin/ruby_executable_hooks:24:in `'
30: from /Users/me/.rvm/gems/ruby-2.5.1/bin/ruby_executable_hooks:24:in `eval'
29: from /Users/me/.rvm/gems/ruby-2.5.1/bin/fastlane:23:in `'
28: from /Users/me/.rvm/gems/ruby-2.5.1/bin/fastlane:23:in `load'
27: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/bin/fastlane:20:in `'
26: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/fastlane/lib/fastlane/cli_tools_distributor.rb:108:in `take_off'
25: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/fastlane/lib/fastlane/commands_generator.rb:41:in `start'
24: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/fastlane/lib/fastlane/commands_generator.rb:333:in `run'
23: from /Users/me/.rvm/gems/ruby-2.5.1/gems/commander-fastlane-4.4.6/lib/commander/delegates.rb:15:in `run!'
22: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/fastlane_core/lib/fastlane_core/ui/fastlane_runner.rb:76:in `run!'
21: from /Users/me/.rvm/gems/ruby-2.5.1/gems/commander-fastlane-4.4.6/lib/commander/runner.rb:476:in `run_active_command'
20: from /Users/me/.rvm/gems/ruby-2.5.1/gems/commander-fastlane-4.4.6/lib/commander/command.rb:153:in `run'
19: from /Users/me/.rvm/gems/ruby-2.5.1/gems/commander-fastlane-4.4.6/lib/commander/command.rb:178:in `call'
18: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/fastlane/lib/fastlane/commands_generator.rb:108:in `block (2 levels) in run'
17: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/fastlane/lib/fastlane/command_line_handler.rb:36:in `handle'
16: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/fastlane/lib/fastlane/lane_manager.rb:56:in `cruise_lane'
15: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/fastlane/lib/fastlane/runner.rb:45:in `execute'
14: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/fastlane/lib/fastlane/runner.rb:45:in `chdir'
13: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/fastlane/lib/fastlane/runner.rb:49:in `block in execute'
12: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/fastlane/lib/fastlane/lane.rb:33:in `call'
11: from Fastfile:294:in `block in parsing_binding'
10: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/fastlane/lib/fastlane/fast_file.rb:149:in `method_missing'
9: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/fastlane/lib/fastlane/runner.rb:157:in `trigger_action_by_name'
8: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/fastlane/lib/fastlane/runner.rb:227:in `execute_action'
7: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/fastlane/lib/fastlane/runner.rb:227:in `chdir'
6: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/fastlane/lib/fastlane/runner.rb:232:in `block in execute_action'
5: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/fastlane/lib/fastlane/actions/actions_helper.rb:50:in `execute_action'
4: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-2.105.2/fastlane/lib/fastlane/runner.rb:253:in `block (2 levels) in execute_action'
3: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-plugin-test_center-3.5.7/lib/fastlane/plugin/test_center/actions/multi_scan.rb:23:in `run'
2: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-plugin-test_center-3.5.7/lib/fastlane/plugin/test_center/helper/correcting_scan_helper.rb:39:in `scan'
1: from /Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-plugin-test_center-3.5.7/lib/fastlane/plugin/test_center/helper/test_collector.rb:35:in `testables'
/Users/me/.rvm/gems/ruby-2.5.1/gems/fastlane-plugin-test_center-3.5.7/lib/fastlane/plugin/test_center/helper/test_collector.rb:45:in `only_testing_to_testables_tests': [!] undefined method `sort' for "MyTestScheme/MyTestClass":String (NoMethodError)
Environment
Stack
| Key | Value |
| --------------------------- | ------------------------------------------------------ |
| OS | 10.13.6 |
| Ruby | 2.5.1 |
| Bundler? | true |
| Git | git version 2.17.1 (Apple Git-112) |
| Installation Source | ~/.rvm/gems/ruby-2.5.1/bin/fastlane |
| Host | Mac OS X 10.13.6 (17G65) |
| Ruby Lib Dir | ~/.rvm/rubies/ruby-2.5.1/lib |
| OpenSSL Version | OpenSSL 1.1.1 11 Sep 2018 |
| Is contained | false |
| Is homebrew | false |
| Is installed via Fabric.app | false |
| Xcode Path | /Applications/Xcode/10.0/Xcode.app/Contents/Developer/ |
| Xcode Version | 10.0 |
System Locale
| Variable | Value | |
| -------- | ----------- | - |
| LANG | en_US.UTF-8 | ✅ |
| LC_ALL | | |
| LANGUAGE | | |
fastlane files:
./fastlane/Fastfile
lane :sweep do |options|
scheme_to_test = options[:scheme_to_test]
UI.important("scheme_to_test: #{scheme_to_test}")
base_derived_data_path = options[:derived_data_path]
UI.important("base_derived_data_path: #{base_derived_data_path}")
delete_dir(base_derived_data_path)
output_directory = options[:output_directory]
UI.important("output_directory: #{output_directory}")
delete_dir(output_directory)
test_to_run = options[:test_to_run]
UI.important("test_to_run: #{test_to_run}")
device_to_test = options[:device]
UI.important("device: #{device_to_test}")
workspace_to_test = options[:workspace]
UI.important("workspace: #{workspace_to_test}")
test_run_block = lambda do |testrun_info|
failed_test_count = testrun_info[:failed].size
if failed_test_count > 0
UI.important('The run of tests would finish with failures due to fragile tests here.')
try_attempt = testrun_info[:try_count]
if try_attempt < 3
UI.header('Since we are using :multi_scan, we can re-run just those failing tests!')
end
end
end
result = multi_scan(
clean: true,
workspace: File.absolute_path("../" + workspace_to_test),
scheme: scheme_to_test,
device: device_to_test,
output_types: "junit",
output_files: "report.junit",
output_directory: output_directory,
only_testing: test_to_run,
include_simulator_logs: true,
result_bundle: true,
derived_data_path: base_derived_data_path,
configuration:"Debug",
xcargs: "ORG_NAME='myCustomVar'",
try_count: 3,
fail_build: false,
testrun_completed_block: test_run_block
)
unless result[:failed_testcount].zero?
UI.message("There are #{result[:failed_testcount]} legitimate failing tests")
end
end
Loaded gems
| Gem | Version | | ---------------------------------------------- | ------------ | | did_you_mean | 1.2.0 | | executable-hooks | 1.5.0 | | bundler-unload | 1.0.2 | | rubygems-bundler | 1.4.5 | | bundler | 1.16.5 | | CFPropertyList | 3.0.0 | | public_suffix | 2.0.5 | | addressable | 2.5.2 | | atomos | 0.1.3 | | babosa | 1.0.2 | | claide | 1.0.2 | | colored2 | 3.1.2 | | cork | 0.3.0 | | nap | 1.1.0 | | open4 | 1.3.4 | | claide-plugins | 0.9.2 | | colored | 1.2 | | highline | 1.7.10 | | commander-fastlane | 4.4.6 | | multipart-post | 2.0.0 | | faraday | 0.15.3 | | faraday-http-cache | 1.3.1 | | git | 1.5.0 | | kramdown | 1.17.0 | | no_proxy_fix | 0.1.2 | | sawyer | 0.8.1 | | octokit | 4.12.0 | | unicode-display_width | 1.4.0 | | terminal-table | 1.8.0 | | danger | 5.6.7 | | danger-plugin-api | 1.0.0 | | danger-xcode_summary | 0.5.0 | | dotenv | 2.5.0 | | emoji_regex | 0.1.1 | | excon | 0.62.0 | | unf_ext | 0.0.7.5 | | unf | 0.1.4 | | domain_name | 0.5.20180417 | | http-cookie | 1.0.3 | | faraday-cookie_jar | 0.0.6 | | faraday_middleware | 0.12.2 | | fastimage | 2.1.4 | | gh_inspector | 1.1.3 | | jwt | 2.1.0 | | memoist | 0.16.0 | | multi_json | 1.13.1 | | os | 1.0.0 | | signet | 0.10.0 | | googleauth | 0.6.6 | | httpclient | 2.8.3 | | mime-types-data | 3.2018.0812 | | mime-types | 3.2.2 | | declarative | 0.0.10 | | declarative-option | 0.1.0 | | uber | 0.1.0 | | representable | 3.0.4 | | retriable | 3.1.2 | | google-api-client | 0.23.9 | | json | 2.1.0 | | mini_magick | 4.5.1 | | multi_xml | 0.6.0 | | plist | 3.4.0 | | rubyzip | 1.2.2 | | security | 0.1.3 | | naturally | 2.2.0 | | simctl | 1.6.5 | | slack-notifier | 2.3.2 | | terminal-notifier | 1.8.0 | | tty-screen | 0.6.5 | | tty-cursor | 0.6.0 | | tty-spinner | 0.8.0 | | word_wrap | 1.0.0 | | nanaimo | 0.2.6 | | xcodeproj | 1.6.0 | | rouge | 2.0.7 | | xcpretty | 0.3.0 | | xcpretty-travis-formatter | 1.0.0 | | xcov | 1.5.0 | | danger-xcov | 0.4.1 | | fastlane-plugin-merge_junit_report | 0.5.0 | | fastlane-plugin-setup_fragile_tests_for_rescan | 2.1.1 | | xctest_list | 1.1.4 | | fastlane-plugin-test_center | 3.5.7 | | xcpretty-json-formatter | 0.1.1 |
ericwengelking
All 10 comments
Okay. The error is that I am expecting an array of strings and you are passing in a string for the :only_testing option. scan coerces that to an array, and multi_scan does not yet.
Can you confirm by changing:
only_testing: test_to_run,
to
only_testing: [test_to_run],
When calling multi_scan?
lyndsey-ferguson
on 2 Oct 2018
👍1
I'm sure that is it. I'll close this issue for now and re-open if it is still a problem. Not sure how I missed that in the docs, thank you!
ericwengelking
on 2 Oct 2018
Oh, no, I want this open. It is a real problem: I just gave you a work-around until I fix it.
lyndsey-ferguson
on 2 Oct 2018
The workaround did resolve my issue. Thank you!
ericwengelking
on 2 Oct 2018
👍1
If I run from the command line like the following
> bundle exec fastlane run multi_scan device:"iPhone X" only_testing:"UI Tests/SmokeUITests/testFeature1" scheme:"UI Tests" workspace:"TEST.xcworkspace"
I still got the error mentioned above. May I ask is there any workaround for this? Thanks.
quintuschen
on 19 Feb 2019
Hm. I do not run it from the command line, but would this work?
only_testing:"[UI Tests/SmokeUITests/testFeature1]"
Further research shows that this may not work. You could set up a Scanfile that has the tests (you can create that file programmatically).
OR you could open up a PR to fix this. I can provide guidance, I do not think it would be too complicated.
lyndsey-ferguson
on 19 Feb 2019
I would like to open a PR to fix this. :) Some hints would be very appreciated!
quintuschen
on 19 Feb 2019
Here is how scan does the coercion. You would need to do something similar here. Please add tests too.
lyndsey-ferguson
on 19 Feb 2019
Fixed in 3.8.0.parallelizing.beta.8. I will close this issue once that is official released.
lyndsey-ferguson
on 25 Jul 2019
Fixed in v3.8.0
lyndsey-ferguson
on 7 Aug 2019
Was this page helpful?
0 / 5 - 0 ratings
Related issues
Ravi61
·
8Comments
b3hni4
·
7Comments
bpoplauschi
·
9Comments
lucasecf
·
11Comments
lyndsey-ferguson
·
5Comments