Fastlane-plugin-test_center: when XCTestCase is configured to continue after failures, failing testcases are reported multiple times and are not collated correctly

Created on 1 Nov 2018 · 42Comments · Source: lyndsey-ferguson/fastlane-plugin-test_center

New Issue Checklist

[X] Updated fastlane-plugin-test_center to the latest version
[X] I read the README.md
[X] I reviewed the example(s) for the action(s) I am using
[X] I have removed any sensitive data such as passwords, authentication tokens, or anything else I do not want to world to see

Issue Description

Test Cases are marked as failed even when they pass on the retry. For ex: With a retry_count of 2, say test cases A, B and C fail in the first attempt. They are picked up in the second attempt and they all PASS. But the multi_scan results does not update the failed_testcount and marks all the tests A, B and C as failed in the final result and marks the build as failed. This is happening consistently enough to file this bug report. Please look into this.

Complete output when running fastlane, including the stack trace and command used

[INSERT OUTPUT HERE]

```INFO [2018-11-01 03:06:19.26]: -----------------------------------------------------
INFO [2018-11-01 03:06:19.26]: Starting test run on testable 'ToolkitUITests'
INFO [2018-11-01 03:06:19.26]: -----------------------------------------------------
INFO [2018-11-01 03:06:22.82]: $ xcodebuild -showBuildSettings -workspace /Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/Toolkit.xcworkspace -scheme ToolkitUITests
DEBUG [2018-11-01 03:06:27.43]: Fetching available simulator devices
INFO [2018-11-01 03:06:28.11]: $ xcodebuild -showBuildSettings -workspace /Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/Toolkit.xcworkspace -scheme ToolkitUITests
DEBUG [2018-11-01 03:06:31.96]: Fetching available simulator devices

+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Summary for scan 2.106.0 |
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| workspace | /Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/Toolkit.xcworkspace |
| device | iPhone X |
| scheme | ToolkitUITests |
| include_simulator_logs | false |
| code_coverage | true |
| test_without_building | true |
| derived_data_path | /Users/mobileteam/Library/Developer/Xcode/DerivedData/Toolkit-deqclbvztwxpkyfaluiupxvfpiej |
| skip_detect_devices | false |
| skip_build | false |
| buildlog_path | ~/Library/Logs/scan |
| disable_concurrent_testing | false |
| should_zip_build_products | false |
| result_bundle | false |
| open_report | false |
| skip_slack | false |
| slack_only_on_failure | false |
| use_clang_report_name | false |
| clean | false |
| only_testing | ["ToolkitUITests/LoginPageTests/testEmptyUserName", "ToolkitUITests/LoginPageTests/testForgotPassword", "ToolkitUITests/LoginPageTests/testOnlySpacePassword", "ToolkitUITests/LoginPageTests/testSpaceCharInPassword", "ToolkitUITests/LoginPageTests/testForSwitchBetweenUsers", "ToolkitUITests/LoginPageTests/testTrailingSpaceUserNameSuccess", "ToolkitUITests/AddContactTests/testVerifyAddingValidContact", "ToolkitUITests/AddContactTests/testVerifyErrorForNoEmailAddress", "ToolkitUITests/AddContactTests/testVerifyErrorForInvalidEmailAddress", "ToolkitUITests/AddContactTests/testVerifyErrorForNoEmailListSelection", "ToolkitUITests/AddContactTests/testVerifyErrorForSavingWithoutAnyEntries", "ToolkitUITests/LibraryPageTest/testSortLibraryImagesByName", "ToolkitUITests/LibraryPageTest/testSortLibraryImagesByMostRecent", "ToolkitUITests/LibraryPageTest/testEmptyFolder", "ToolkitUITests/LibraryPageTest/testBrowseAllFolder", "ToolkitUITests/LibraryPageTest/testBrowseAllImages", "ToolkitUITests/LibraryPageTest/testSwitchToListView", "ToolkitUITests/LibraryPageTest/testLastSelectedFolder", "ToolkitUITests/EditContactTests/testDeleteContact", "ToolkitUITests/EditContactTests/testEditExistingEmailAddress", "ToolkitUITests/EditContactTests/testEditExistingContact", "ToolkitUITests/SentDashboardTests/testViewSentDetailsByTappingBarChart", "ToolkitUITests/SentDashboardTests/testViewSentDetailsByTappingSentOpensClicks", "ToolkitUITests/SentDashboardTests/testTapCampaignName", "ToolkitUITests/SentDashboardTests/testTappingLeftRightArrows", "ToolkitUITests/ActiveContactsTests/testSortingContacts", "ToolkitUITests/ActiveContactsTests/testSortByDateAdded", "ToolkitUITests/ActiveContactsTests/testSortByFirstName"] |
| output_directory | reports/ui-tests |
| output_types | html,junit |
| output_files | report-2.html,report-2.junit |
| fail_build | true |
| xcode_path | /Applications/Xcode.app |
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

INFO [2018-11-01 03:06:32.46]: $ set -o pipefail && env NSUnbufferedIO=YES xcodebuild -workspace /Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/Toolkit.xcworkspace -scheme ToolkitUITests -destination 'platform=iOS Simulator,id=599DE921-5C65-43FD-A852-E3D28791505D' -derivedDataPath '/Users/mobileteam/Library/Developer/Xcode/DerivedData/Toolkit-deqclbvztwxpkyfaluiupxvfpiej' -enableCodeCoverage YES -only-testing:ToolkitUITests/LoginPageTests/testEmptyUserName -only-testing:ToolkitUITests/LoginPageTests/testForgotPassword -only-testing:ToolkitUITests/LoginPageTests/testOnlySpacePassword -only-testing:ToolkitUITests/LoginPageTests/testSpaceCharInPassword -only-testing:ToolkitUITests/LoginPageTests/testForSwitchBetweenUsers -only-testing:ToolkitUITests/LoginPageTests/testTrailingSpaceUserNameSuccess -only-testing:ToolkitUITests/AddContactTests/testVerifyAddingValidContact -only-testing:ToolkitUITests/AddContactTests/testVerifyErrorForNoEmailAddress -only-testing:ToolkitUITests/AddContactTests/testVerifyErrorForInvalidEmailAddress -only-testing:ToolkitUITests/AddContactTests/testVerifyErrorForNoEmailListSelection -only-testing:ToolkitUITests/AddContactTests/testVerifyErrorForSavingWithoutAnyEntries -only-testing:ToolkitUITests/LibraryPageTest/testSortLibraryImagesByName -only-testing:ToolkitUITests/LibraryPageTest/testSortLibraryImagesByMostRecent -only-testing:ToolkitUITests/LibraryPageTest/testEmptyFolder -only-testing:ToolkitUITests/LibraryPageTest/testBrowseAllFolder -only-testing:ToolkitUITests/LibraryPageTest/testBrowseAllImages -only-testing:ToolkitUITests/LibraryPageTest/testSwitchToListView -only-testing:ToolkitUITests/LibraryPageTest/testLastSelectedFolder -only-testing:ToolkitUITests/EditContactTests/testDeleteContact -only-testing:ToolkitUITests/EditContactTests/testEditExistingEmailAddress -only-testing:ToolkitUITests/EditContactTests/testEditExistingContact -only-testing:ToolkitUITests/SentDashboardTests/testViewSentDetailsByTappingBarChart -only-testing:ToolkitUITests/SentDashboardTests/testViewSentDetailsByTappingSentOpensClicks -only-testing:ToolkitUITests/SentDashboardTests/testTapCampaignName -only-testing:ToolkitUITests/SentDashboardTests/testTappingLeftRightArrows -only-testing:ToolkitUITests/ActiveContactsTests/testSortingContacts -only-testing:ToolkitUITests/ActiveContactsTests/testSortByDateAdded -only-testing:ToolkitUITests/ActiveContactsTests/testSortByFirstName test-without-building | tee '/Users/mobileteam/Library/Logs/scan/Toolkit.Dev-ToolkitUITests.log' | xcpretty --report html --output '/Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/reports/ui-tests/report-2.html' --report junit --output '/Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/reports/ui-tests/report-2.junit' --report junit --output '/var/folders/dk/jfnrj9z13wdckb084w7xxnbc0000gq/T/junit_report20181101-86392-18c5bqf'
INFO [2018-11-01 03:06:32.46]: ▸ Loading...
INFO [2018-11-01 03:06:35.08]: ▸ 2018-11-01 03:06:35.083 xcodebuild[11576:19231582] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
INFO [2018-11-01 03:06:35.20]: ▸ No devices are booted.
INFO [2018-11-01 03:06:35.21]: ▸ 2018-11-01 03:06:35.217 xcodebuild[11538:19231430] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
INFO [2018-11-01 03:06:35.21]: ▸ /Users/mobileteam/Library/Developer/Xcode/DerivedData/Toolkit-deqclbvztwxpkyfaluiupxvfpiej/Logs/Test/Test-ToolkitUITests-2018.11.01_03-06-34--0400.xcresult/1_Test/Diagnostics/ToolkitUITests-0CDFC2BC-6EC2-4228-A7C3-4DA05CBCF919/ToolkitUITests-846ED78E-DB13-47C1-B6CB-29BF1925F417/Session-ToolkitUITests-2018-11-01_030635-SNyUlp.log
INFO [2018-11-01 03:06:35.21]: ▸ 2018-11-01 03:06:35.218 xcodebuild[11538:19231426] [MT] IDETestOperationsObserverDebug: (DED222EE-8A11-40EA-A245-80385761425F) Beginning test session ToolkitUITests-DED222EE-8A11-40EA-A245-80385761425F at 2018-11-01 03:06:35.218 with Xcode 10A255 on target {
INFO [2018-11-01 03:06:35.21]: ▸ SimDevice: iPhone X (599DE921-5C65-43FD-A852-E3D28791505D, iOS 12.0, Shutdown)
INFO [2018-11-01 03:06:35.21]: ▸ } (12.0 (16A366))
INFO [2018-11-01 03:06:51.07]: ▸ Selected tests
INFO [2018-11-01 03:06:51.07]: ▸ Test Suite ToolkitUITests.xctest started
INFO [2018-11-01 03:06:51.08]: ▸ ActiveContactsTests
INFO [2018-11-01 03:07:40.57]: ▸ ✓ testSortByDateAdded (49.491 seconds)
INFO [2018-11-01 03:08:29.72]: ▸ ✓ testSortByFirstName (49.148 seconds)
INFO [2018-11-01 03:09:29.03]: ▸ ✓ testSortingContacts (59.318 seconds)
INFO [2018-11-01 03:09:29.04]: ▸ AddContactTests
INFO [2018-11-01 03:10:27.82]: ▸ ✓ testVerifyAddingValidContact (58.788 seconds)
INFO [2018-11-01 03:11:29.66]: ▸ ✓ testVerifyErrorForInvalidEmailAddress (61.834 seconds)
INFO [2018-11-01 03:12:33.73]: ▸ ✓ testVerifyErrorForNoEmailAddress (64.072 seconds)
INFO [2018-11-01 03:13:35.00]: ▸ ✓ testVerifyErrorForNoEmailListSelection (61.274 seconds)
INFO [2018-11-01 03:14:35.67]: ▸ ✓ testVerifyErrorForSavingWithoutAnyEntries (60.662 seconds)
INFO [2018-11-01 03:14:35.67]: ▸ EditContactTests
INFO [2018-11-01 03:15:45.30]: ▸ ✗ testDeleteContact, Failed to find "Contacts" Other after 5.0 seconds
INFO [2018-11-01 03:15:45.34]: ▸ ✗ testDeleteContact, Asynchronous wait failed: Exceeded timeout of 5 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Contacts" Other".
INFO [2018-11-01 03:16:25.13]: ▸ ✗ testEditExistingContact, Failed to find "Log In" Button after 30.0 seconds
INFO [2018-11-01 03:16:25.18]: ▸ ✗ testEditExistingContact, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
INFO [2018-11-01 03:17:05.18]: ▸ ✗ testEditExistingEmailAddress, Failed to find "Log In" Button after 30.0 seconds
INFO [2018-11-01 03:17:05.22]: ▸ ✗ testEditExistingEmailAddress, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
INFO [2018-11-01 03:17:07.86]: ▸ LibraryPageTest
INFO [2018-11-01 03:17:44.96]: ▸ ✗ testBrowseAllFolder, Failed to find "Log In" Button after 30.0 seconds
INFO [2018-11-01 03:17:45.00]: ▸ ✗ testBrowseAllFolder, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
INFO [2018-11-01 03:18:24.90]: ▸ ✗ testBrowseAllImages, Failed to find "Log In" Button after 30.0 seconds
INFO [2018-11-01 03:18:24.94]: ▸ ✗ testBrowseAllImages, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
INFO [2018-11-01 03:19:05.07]: ▸ ✗ testEmptyFolder, Failed to find "Log In" Button after 30.0 seconds
INFO [2018-11-01 03:19:05.11]: ▸ ✗ testEmptyFolder, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
INFO [2018-11-01 03:19:55.22]: ▸ ✓ testLastSelectedFolder (42.562 seconds)
INFO [2018-11-01 03:20:32.15]: ▸ ✓ testSortLibraryImagesByMostRecent (36.937 seconds)
INFO [2018-11-01 03:21:09.23]: ▸ ✓ testSortLibraryImagesByName (37.081 seconds)
INFO [2018-11-01 03:29:37.02]: ▸ ✓ testSwitchToListView (507.788 seconds)
INFO [2018-11-01 03:29:37.03]: ▸ LoginPageTests
INFO [2018-11-01 03:29:58.40]: ▸ ✓ testEmptyUserName (21.374 seconds)
INFO [2018-11-01 03:30:14.89]: ▸ ✓ testForgotPassword (16.486 seconds)
INFO [2018-11-01 03:31:27.32]: ▸ ✓ testForSwitchBetweenUsers (72.434 seconds)
INFO [2018-11-01 03:31:49.10]: ▸ ✓ testOnlySpacePassword (21.785 seconds)
INFO [2018-11-01 03:32:07.02]: ▸ ✓ testSpaceCharInPassword (17.919 seconds)
INFO [2018-11-01 03:32:36.88]: ▸ ✓ testTrailingSpaceUserNameSuccess (29.854 seconds)
INFO [2018-11-01 03:32:36.90]: ▸ SentDashboardTests
INFO [2018-11-01 03:33:17.20]: ▸ ✓ testTapCampaignName (40.317 seconds)
INFO [2018-11-01 03:33:54.16]: ▸ ✓ testTappingLeftRightArrows (36.962 seconds)
INFO [2018-11-01 03:34:33.32]: ▸ ✓ testViewSentDetailsByTappingBarChart (39.164 seconds)
INFO [2018-11-01 03:35:15.33]: ▸ ✓ testViewSentDetailsByTappingSentOpensClicks (42.005 seconds)
INFO [2018-11-01 03:35:15.35]: ▸ ToolkitUITests.EditContactTests
INFO [2018-11-01 03:35:15.35]: ▸ testDeleteContact, Failed to find "Contacts" Other after 5.0 seconds
INFO [2018-11-01 03:35:15.35]: ▸ ContactsDashboardPage.swift:22
INFO [2018-11-01 03:35:15.35]: ▸ testDeleteContact, Asynchronous wait failed: Exceeded timeout of 5 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Contacts" Other".
INFO [2018-11-01 03:35:15.36]: ▸ HelperExtensions.swift:32
INFO [2018-11-01 03:35:15.36]: ▸ testEditExistingContact, Failed to find "Log In" Button after 30.0 seconds
INFO [2018-11-01 03:35:15.36]: ▸ LandingPage.swift:25
INFO [2018-11-01 03:35:15.36]: ▸ testEditExistingContact, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
INFO [2018-11-01 03:35:15.36]: ▸ HelperExtensions.swift:32
INFO [2018-11-01 03:35:15.36]: ▸ testEditExistingEmailAddress, Failed to find "Log In" Button after 30.0 seconds
INFO [2018-11-01 03:35:15.36]: ▸ LandingPage.swift:25
INFO [2018-11-01 03:35:15.36]: ▸ testEditExistingEmailAddress, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
INFO [2018-11-01 03:35:15.36]: ▸ HelperExtensions.swift:32
INFO [2018-11-01 03:35:15.36]: ▸ ToolkitUITests.LibraryPageTest
INFO [2018-11-01 03:35:15.36]: ▸ testBrowseAllFolder, Failed to find "Log In" Button after 30.0 seconds
INFO [2018-11-01 03:35:15.36]: ▸ LandingPage.swift:25
INFO [2018-11-01 03:35:15.36]: ▸ testBrowseAllFolder, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
INFO [2018-11-01 03:35:15.36]: ▸ HelperExtensions.swift:32
INFO [2018-11-01 03:35:15.36]: ▸ testBrowseAllImages, Failed to find "Log In" Button after 30.0 seconds
INFO [2018-11-01 03:35:15.36]: ▸ LandingPage.swift:25
INFO [2018-11-01 03:35:15.36]: ▸ testBrowseAllImages, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
INFO [2018-11-01 03:35:15.36]: ▸ HelperExtensions.swift:32
INFO [2018-11-01 03:35:15.36]: ▸ testEmptyFolder, Failed to find "Log In" Button after 30.0 seconds
INFO [2018-11-01 03:35:15.36]: ▸ LandingPage.swift:25
INFO [2018-11-01 03:35:15.36]: ▸ testEmptyFolder, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
INFO [2018-11-01 03:35:15.36]: ▸ HelperExtensions.swift:32
INFO [2018-11-01 03:35:15.36]: ▸ Executed 28 tests, with 12 failures (0 unexpected) in 1704.234 (1704.255) seconds
INFO [2018-11-01 03:35:15.36]: ▸
INFO [2018-11-01 03:35:35.01]: ▸ 2018-11-01 03:35:35.013 xcodebuild[11538:19231426] [MT] IDETestOperationsObserverDebug: 1739.802 elapsed -- Testing started completed.
INFO [2018-11-01 03:35:35.01]: ▸ 2018-11-01 03:35:35.013 xcodebuild[11538:19231426] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
INFO [2018-11-01 03:35:35.01]: ▸ 2018-11-01 03:35:35.013 xcodebuild[11538:19231426] [MT] IDETestOperationsObserverDebug: 1739.802 sec, +1739.802 sec -- end
INFO [2018-11-01 03:35:35.39]: ▸ 2018-11-01 03:35:35.393 xcodebuild[29339:19300882] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
INFO [2018-11-01 03:35:37.33]: ▸ An error was encountered processing the command (domain=com.apple.CoreSimulator.SimError, code=164):
INFO [2018-11-01 03:35:37.33]: ▸ Invalid device state
INFO [2018-11-01 03:35:37.33]: ▸ Underlying error (domain=NSMachErrorDomain, code=-308):
INFO [2018-11-01 03:35:37.35]: ▸ The operation couldn’t be completed. (Mach error -308 - (ipc/mig) server died)
INFO [2018-11-01 03:35:39.69]: ▸ Failing tests:
INFO [2018-11-01 03:35:39.69]: ▸ EditContactTests.testDeleteContact()
INFO [2018-11-01 03:35:39.69]: ▸ EditContactTests.testDeleteContact()
INFO [2018-11-01 03:35:39.70]: ▸ EditContactTests.testEditExistingContact()
INFO [2018-11-01 03:35:39.70]: ▸ EditContactTests.testEditExistingContact()
INFO [2018-11-01 03:35:39.70]: ▸ EditContactTests.testEditExistingEmailAddress()
INFO [2018-11-01 03:35:39.70]: ▸ EditContactTests.testEditExistingEmailAddress()
INFO [2018-11-01 03:35:39.70]: ▸ LibraryPageTest.testBrowseAllFolder()
INFO [2018-11-01 03:35:39.70]: ▸ LibraryPageTest.testBrowseAllFolder()
INFO [2018-11-01 03:35:39.70]: ▸ LibraryPageTest.testBrowseAllImages()
INFO [2018-11-01 03:35:39.70]: ▸ LibraryPageTest.testBrowseAllImages()
INFO [2018-11-01 03:35:39.70]: ▸ LibraryPageTest.testEmptyFolder()
INFO [2018-11-01 03:35:39.70]: ▸ LibraryPageTest.testEmptyFolder()
INFO [2018-11-01 03:35:39.70]: ▸ * TEST EXECUTE FAILED *
2018-11-01 03:06:35.083 xcodebuild[11576:19231582] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
No devices are booted.
2018-11-01 03:06:35.217 xcodebuild[11538:19231430] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
/Users/mobileteam/Library/Developer/Xcode/DerivedData/Toolkit-deqclbvztwxpkyfaluiupxvfpiej/Logs/Test/Test-ToolkitUITests-2018.11.01_03-06-34--0400.xcresult/1_Test/Diagnostics/ToolkitUITests-0CDFC2BC-6EC2-4228-A7C3-4DA05CBCF919/ToolkitUITests-846ED78E-DB13-47C1-B6CB-29BF1925F417/Session-ToolkitUITests-2018-11-01_030635-SNyUlp.log
2018-11-01 03:06:35.218 xcodebuild[11538:19231426] [MT] IDETestOperationsObserverDebug: (DED222EE-8A11-40EA-A245-80385761425F) Beginning test session ToolkitUITests-DED222EE-8A11-40EA-A245-80385761425F at 2018-11-01 03:06:35.218 with Xcode 10A255 on target {
SimDevice: iPhone X (599DE921-5C65-43FD-A852-E3D28791505D, iOS 12.0, Shutdown)
} (12.0 (16A366))
Selected tests
Test Suite ToolkitUITests.xctest started
ActiveContactsTests
✓ testSortByDateAdded (49.491 seconds)
✓ testSortByFirstName (49.148 seconds)
✓ testSortingContacts (59.318 seconds)
AddContactTests
✓ testVerifyAddingValidContact (58.788 seconds)
✓ testVerifyErrorForInvalidEmailAddress (61.834 seconds)
✓ testVerifyErrorForNoEmailAddress (64.072 seconds)
✓ testVerifyErrorForNoEmailListSelection (61.274 seconds)
✓ testVerifyErrorForSavingWithoutAnyEntries (60.662 seconds)
EditContactTests
✗ testDeleteContact, Failed to find "Contacts" Other after 5.0 seconds
✗ testDeleteContact, Asynchronous wait failed: Exceeded timeout of 5 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Contacts" Other".
✗ testEditExistingContact, Failed to find "Log In" Button after 30.0 seconds
✗ testEditExistingContact, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
✗ testEditExistingEmailAddress, Failed to find "Log In" Button after 30.0 seconds
✗ testEditExistingEmailAddress, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
LibraryPageTest
✗ testBrowseAllFolder, Failed to find "Log In" Button after 30.0 seconds
✗ testBrowseAllFolder, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
✗ testBrowseAllImages, Failed to find "Log In" Button after 30.0 seconds
✗ testBrowseAllImages, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
✗ testEmptyFolder, Failed to find "Log In" Button after 30.0 seconds
✗ testEmptyFolder, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
✓ testLastSelectedFolder (42.562 seconds)
✓ testSortLibraryImagesByMostRecent (36.937 seconds)
✓ testSortLibraryImagesByName (37.081 seconds)
✓ testSwitchToListView (507.788 seconds)
LoginPageTests
✓ testEmptyUserName (21.374 seconds)
✓ testForgotPassword (16.486 seconds)
✓ testForSwitchBetweenUsers (72.434 seconds)
✓ testOnlySpacePassword (21.785 seconds)
✓ testSpaceCharInPassword (17.919 seconds)
✓ testTrailingSpaceUserNameSuccess (29.854 seconds)
SentDashboardTests
✓ testTapCampaignName (40.317 seconds)
✓ testTappingLeftRightArrows (36.962 seconds)
✓ testViewSentDetailsByTappingBarChart (39.164 seconds)
✓ testViewSentDetailsByTappingSentOpensClicks (42.005 seconds)

ToolkitUITests.EditContactTests
testDeleteContact, Failed to find "Contacts" Other after 5.0 seconds
ContactsDashboardPage.swift:22

testDeleteContact, Asynchronous wait failed: Exceeded timeout of 5 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Contacts" Other".
HelperExtensions.swift:32

testEditExistingContact, Failed to find "Log In" Button after 30.0 seconds
LandingPage.swift:25

testEditExistingContact, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
HelperExtensions.swift:32

testEditExistingEmailAddress, Failed to find "Log In" Button after 30.0 seconds
LandingPage.swift:25

testEditExistingEmailAddress, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
HelperExtensions.swift:32

ToolkitUITests.LibraryPageTest
testBrowseAllFolder, Failed to find "Log In" Button after 30.0 seconds
LandingPage.swift:25

testBrowseAllFolder, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
HelperExtensions.swift:32

testBrowseAllImages, Failed to find "Log In" Button after 30.0 seconds
LandingPage.swift:25

testBrowseAllImages, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
HelperExtensions.swift:32

testEmptyFolder, Failed to find "Log In" Button after 30.0 seconds
LandingPage.swift:25

testEmptyFolder, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
HelperExtensions.swift:32

 Executed 28 tests, with 12 failures (0 unexpected) in 1704.234 (1704.255) seconds

2018-11-01 03:35:35.013 xcodebuild[11538:19231426] [MT] IDETestOperationsObserverDebug: 1739.802 elapsed -- Testing started completed.
2018-11-01 03:35:35.013 xcodebuild[11538:19231426] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
2018-11-01 03:35:35.013 xcodebuild[11538:19231426] [MT] IDETestOperationsObserverDebug: 1739.802 sec, +1739.802 sec -- end
2018-11-01 03:35:35.393 xcodebuild[29339:19300882] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
An error was encountered processing the command (domain=com.apple.CoreSimulator.SimError, code=164):
Invalid device state
Underlying error (domain=NSMachErrorDomain, code=-308):
The operation couldn’t be completed. (Mach error -308 - (ipc/mig) server died)
Failing tests:
EditContactTests.testDeleteContact()
EditContactTests.testDeleteContact()
EditContactTests.testEditExistingContact()
EditContactTests.testEditExistingContact()
EditContactTests.testEditExistingEmailAddress()
EditContactTests.testEditExistingEmailAddress()
LibraryPageTest.testBrowseAllFolder()
LibraryPageTest.testBrowseAllFolder()
LibraryPageTest.testBrowseAllImages()
LibraryPageTest.testBrowseAllImages()
LibraryPageTest.testEmptyFolder()
LibraryPageTest.testEmptyFolder()
* TEST EXECUTE FAILED *
ERROR [2018-11-01 03:35:39.73]: Exit status: 65
+--------------------+----+
| Test Results |
+--------------------+----+
| Number of tests | 34 |
| Number of failures | 12 |
+--------------------+----+

DEBUG [2018-11-01 03:35:40.23]: Scan failed with Tests have failed
INFO [2018-11-01 03:35:40.32]: Re-running scan on only failed tests
INFO [2018-11-01 03:35:43.82]: $ xcodebuild -showBuildSettings -workspace /Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/Toolkit.xcworkspace -scheme ToolkitUITests
DEBUG [2018-11-01 03:35:48.59]: Fetching available simulator devices
INFO [2018-11-01 03:35:49.28]: $ xcodebuild -showBuildSettings -workspace /Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/Toolkit.xcworkspace -scheme ToolkitUITests
DEBUG [2018-11-01 03:35:53.14]: Fetching available simulator devices

INFO [2018-11-01 03:35:53.65]: $ set -o pipefail && env NSUnbufferedIO=YES xcodebuild -workspace /Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/Toolkit.xcworkspace -scheme ToolkitUITests -destination 'platform=iOS Simulator,id=599DE921-5C65-43FD-A852-E3D28791505D' -derivedDataPath '/Users/mobileteam/Library/Developer/Xcode/DerivedData/Toolkit-deqclbvztwxpkyfaluiupxvfpiej' -only-testing:ToolkitUITests/EditContactTests/testDeleteContact -only-testing:ToolkitUITests/EditContactTests/testDeleteContact -only-testing:ToolkitUITests/EditContactTests/testEditExistingContact -only-testing:ToolkitUITests/EditContactTests/testEditExistingContact -only-testing:ToolkitUITests/EditContactTests/testEditExistingEmailAddress -only-testing:ToolkitUITests/EditContactTests/testEditExistingEmailAddress -only-testing:ToolkitUITests/LibraryPageTest/testBrowseAllFolder -only-testing:ToolkitUITests/LibraryPageTest/testBrowseAllFolder -only-testing:ToolkitUITests/LibraryPageTest/testBrowseAllImages -only-testing:ToolkitUITests/LibraryPageTest/testBrowseAllImages -only-testing:ToolkitUITests/LibraryPageTest/testEmptyFolder -only-testing:ToolkitUITests/LibraryPageTest/testEmptyFolder test-without-building | tee '/Users/mobileteam/Library/Logs/scan/Toolkit.Dev-ToolkitUITests.log' | xcpretty --report html --output '/Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/reports/ui-tests/report-3.html' --report junit --output '/Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/reports/ui-tests/report-3.junit' --report junit --output '/var/folders/dk/jfnrj9z13wdckb084w7xxnbc0000gq/T/junit_report20181101-86392-10zvtn'
INFO [2018-11-01 03:35:53.65]: ▸ Loading...
INFO [2018-11-01 03:35:56.18]: ▸ 2018-11-01 03:35:56.180 xcodebuild[29637:19302332] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
INFO [2018-11-01 03:35:56.30]: ▸ No devices are booted.
INFO [2018-11-01 03:35:56.31]: ▸ 2018-11-01 03:35:56.314 xcodebuild[29604:19302249] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
INFO [2018-11-01 03:35:56.31]: ▸ /Users/mobileteam/Library/Developer/Xcode/DerivedData/Toolkit-deqclbvztwxpkyfaluiupxvfpiej/Logs/Test/Test-ToolkitUITests-2018.11.01_03-35-55--0400.xcresult/1_Test/Diagnostics/ToolkitUITests-C364D1E7-924D-45AF-A095-28B846DF17FA/ToolkitUITests-84402E7D-62A2-432A-BFCB-F2EDE5D8511F/Session-ToolkitUITests-2018-11-01_033556-MrMoNQ.log
INFO [2018-11-01 03:35:56.31]: ▸ 2018-11-01 03:35:56.315 xcodebuild[29604:19302181] [MT] IDETestOperationsObserverDebug: (0F520EF5-BE73-48C0-960E-9BBA8BFADE6A) Beginning test session ToolkitUITests-0F520EF5-BE73-48C0-960E-9BBA8BFADE6A at 2018-11-01 03:35:56.315 with Xcode 10A255 on target {
INFO [2018-11-01 03:35:56.31]: ▸ SimDevice: iPhone X (599DE921-5C65-43FD-A852-E3D28791505D, iOS 12.0, Shutdown)
INFO [2018-11-01 03:35:56.31]: ▸ } (12.0 (16A366))
INFO [2018-11-01 03:36:10.47]: ▸ Selected tests
INFO [2018-11-01 03:36:10.47]: ▸ Test Suite ToolkitUITests.xctest started
INFO [2018-11-01 03:36:10.47]: ▸ EditContactTests
INFO [2018-11-01 03:37:35.86]: ▸ ✓ testDeleteContact (85.381 seconds)
INFO [2018-11-01 03:38:44.82]: ▸ ✓ testEditExistingContact (68.959 seconds)
INFO [2018-11-01 03:39:48.39]: ▸ ✓ testEditExistingEmailAddress (63.576 seconds)
INFO [2018-11-01 03:39:48.44]: ▸ LibraryPageTest
INFO [2018-11-01 03:41:42.76]: ▸ ✓ testBrowseAllFolder (114.368 seconds)
INFO [2018-11-01 03:43:35.81]: ▸ ✓ testBrowseAllImages (113.044 seconds)
INFO [2018-11-01 03:44:13.44]: ▸ ✓ testEmptyFolder (37.636 seconds)
INFO [2018-11-01 03:44:13.48]: ▸ Executed 6 tests, with 0 failures (0 unexpected) in 482.965 (482.970) seconds
INFO [2018-11-01 03:44:13.48]: ▸
INFO [2018-11-01 03:44:13.81]: ▸ 2018-11-01 03:44:13.819 xcodebuild[29604:19302181] [MT] IDETestOperationsObserverDebug: 497.510 elapsed -- Testing started completed.
INFO [2018-11-01 03:44:13.81]: ▸ 2018-11-01 03:44:13.819 xcodebuild[29604:19302181] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
INFO [2018-11-01 03:44:13.81]: ▸ 2018-11-01 03:44:13.819 xcodebuild[29604:19302181] [MT] IDETestOperationsObserverDebug: 497.510 sec, +497.510 sec -- end
INFO [2018-11-01 03:44:14.00]: ▸ 2018-11-01 03:44:14.007 xcodebuild[34672:19322409] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
INFO [2018-11-01 03:44:16.67]: ▸ Test execute Succeeded
+--------------------+---+
| Test Results |
+--------------------+---+
| Number of tests | 6 |
| Number of failures | 0 |
+--------------------+---+

INFO [2018-11-01 03:44:16.91]: -----------------------------------------------------
INFO [2018-11-01 03:44:16.91]: Starting test run on testable 'ToolkitUITests'
INFO [2018-11-01 03:44:16.91]: -----------------------------------------------------
INFO [2018-11-01 03:44:22.76]: $ xcodebuild -showBuildSettings -workspace /Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/Toolkit.xcworkspace -scheme ToolkitUITests
DEBUG [2018-11-01 03:44:27.07]: Fetching available simulator devices
INFO [2018-11-01 03:44:27.74]: $ xcodebuild -showBuildSettings -workspace /Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/Toolkit.xcworkspace -scheme ToolkitUITests
DEBUG [2018-11-01 03:44:31.60]: Fetching available simulator devices

+----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Summary for scan 2.106.0 |
+----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| workspace | /Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/Toolkit.xcworkspace |
| device | iPhone X |
| scheme | ToolkitUITests |
| include_simulator_logs | false |
| code_coverage | true |
| test_without_building | true |
| derived_data_path | /Users/mobileteam/Library/Developer/Xcode/DerivedData/Toolkit-deqclbvztwxpkyfaluiupxvfpiej |
| skip_detect_devices | false |
| skip_build | false |
| buildlog_path | ~/Library/Logs/scan |
| disable_concurrent_testing | false |
| should_zip_build_products | false |
| result_bundle | false |
| open_report | false |
| skip_slack | false |
| slack_only_on_failure | false |
| use_clang_report_name | false |
| clean | false |
| only_testing | ["ToolkitUITests/ActiveContactsTests/testSortByCompanyName", "ToolkitUITests/ActiveContactsTests/testSortByEmailAddress", "ToolkitUITests/ActiveContactsTests/testViewContactDetails", "ToolkitUITests/ContactDetailsTests/testViewEmailListsDetails", "ToolkitUITests/ContactDetailsTests/testEditContactFromContactDetailsPage", "ToolkitUITests/ContactDetailsTests/testViewContactDetails", "ToolkitUITests/ContactDetailsTests/testVerifyModifyingEmailLists", "ToolkitUITests/ContactDetailsTests/testVerifyAtleastOneEmailListError", "ToolkitUITests/DraftDashboardTests/testViewDraftCampaignDetails", "ToolkitUITests/DraftDashboardTests/testSendTestCampaign", "ToolkitUITests/DraftDashboardTests/testRemoveCampaignDetails", "ToolkitUITests/SearchContactsTests/testVerifyZeroSearchResults", "ToolkitUITests/SearchContactsTests/testValidContactSearch", "ToolkitUITests/SearchContactsTests/testVerifySortInContactsSearch", "ToolkitUITests/SearchContactsTests/testViewContactDetailsInContactsSearch", "ToolkitUITests/SearchContactsTests/testTapSearch", "ToolkitUITests/CampaignMetricsTests/testResendCampaignFlow", "ToolkitUITests/ScheduleDashboardTests/testScheduleDashboard", "ToolkitUITests/ScheduledCampaignTests/testScheduleCampaignTitleTextBlocks", "ToolkitUITests/ScheduledCampaignTests/testScheduleCampaignButtonLinkImgBlocks", "ToolkitUITests/ScheduledCampaignTests/testUnschueduleCampaign", "ToolkitUITests/ScheduledCampaignTests/testScheduleDraftCampaign", "ToolkitUITests/ScheduledCampaignTests/testScheduleNoSendToListError", "ToolkitUITests/ContactsAcceptanceTests/testContactsUserScenario1", "ToolkitUITests/ContactsAcceptanceTests/testContactsUserScenario2", "ToolkitUITests/ContactsAcceptanceTests/testContactsUserScenario3", "ToolkitUITests/ContactsAcceptanceTests/testContactsUserScenario4", "ToolkitUITests/ContactsAcceptanceTests/testContactsUserScenario5"] |
| output_directory | reports/ui-tests |
| output_types | html,junit |
| output_files | report-4.html,report-4.junit |
| fail_build | true |
| xcode_path | /Applications/Xcode.app |
+----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

INFO [2018-11-01 03:44:32.10]: $ set -o pipefail && env NSUnbufferedIO=YES xcodebuild -workspace /Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/Toolkit.xcworkspace -scheme ToolkitUITests -destination 'platform=iOS Simulator,id=599DE921-5C65-43FD-A852-E3D28791505D' -derivedDataPath '/Users/mobileteam/Library/Developer/Xcode/DerivedData/Toolkit-deqclbvztwxpkyfaluiupxvfpiej' -enableCodeCoverage YES -only-testing:ToolkitUITests/ActiveContactsTests/testSortByCompanyName -only-testing:ToolkitUITests/ActiveContactsTests/testSortByEmailAddress -only-testing:ToolkitUITests/ActiveContactsTests/testViewContactDetails -only-testing:ToolkitUITests/ContactDetailsTests/testViewEmailListsDetails -only-testing:ToolkitUITests/ContactDetailsTests/testEditContactFromContactDetailsPage -only-testing:ToolkitUITests/ContactDetailsTests/testViewContactDetails -only-testing:ToolkitUITests/ContactDetailsTests/testVerifyModifyingEmailLists -only-testing:ToolkitUITests/ContactDetailsTests/testVerifyAtleastOneEmailListError -only-testing:ToolkitUITests/DraftDashboardTests/testViewDraftCampaignDetails -only-testing:ToolkitUITests/DraftDashboardTests/testSendTestCampaign -only-testing:ToolkitUITests/DraftDashboardTests/testRemoveCampaignDetails -only-testing:ToolkitUITests/SearchContactsTests/testVerifyZeroSearchResults -only-testing:ToolkitUITests/SearchContactsTests/testValidContactSearch -only-testing:ToolkitUITests/SearchContactsTests/testVerifySortInContactsSearch -only-testing:ToolkitUITests/SearchContactsTests/testViewContactDetailsInContactsSearch -only-testing:ToolkitUITests/SearchContactsTests/testTapSearch -only-testing:ToolkitUITests/CampaignMetricsTests/testResendCampaignFlow -only-testing:ToolkitUITests/ScheduleDashboardTests/testScheduleDashboard -only-testing:ToolkitUITests/ScheduledCampaignTests/testScheduleCampaignTitleTextBlocks -only-testing:ToolkitUITests/ScheduledCampaignTests/testScheduleCampaignButtonLinkImgBlocks -only-testing:ToolkitUITests/ScheduledCampaignTests/testUnschueduleCampaign -only-testing:ToolkitUITests/ScheduledCampaignTests/testScheduleDraftCampaign -only-testing:ToolkitUITests/ScheduledCampaignTests/testScheduleNoSendToListError -only-testing:ToolkitUITests/ContactsAcceptanceTests/testContactsUserScenario1 -only-testing:ToolkitUITests/ContactsAcceptanceTests/testContactsUserScenario2 -only-testing:ToolkitUITests/ContactsAcceptanceTests/testContactsUserScenario3 -only-testing:ToolkitUITests/ContactsAcceptanceTests/testContactsUserScenario4 -only-testing:ToolkitUITests/ContactsAcceptanceTests/testContactsUserScenario5 test-without-building | tee '/Users/mobileteam/Library/Logs/scan/Toolkit.Dev-ToolkitUITests.log' | xcpretty --report html --output '/Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/reports/ui-tests/report-4.html' --report junit --output '/Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/reports/ui-tests/report-4.junit' --report junit --output '/var/folders/dk/jfnrj9z13wdckb084w7xxnbc0000gq/T/junit_report20181101-86392-1uv7ftw'
INFO [2018-11-01 03:44:32.10]: ▸ Loading...
INFO [2018-11-01 03:44:34.64]: ▸ 2018-11-01 03:44:34.646 xcodebuild[34972:19323662] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
INFO [2018-11-01 03:44:34.77]: ▸ No devices are booted.
INFO [2018-11-01 03:44:34.78]: ▸ 2018-11-01 03:44:34.780 xcodebuild[34938:19323531] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
INFO [2018-11-01 03:44:34.78]: ▸ /Users/mobileteam/Library/Developer/Xcode/DerivedData/Toolkit-deqclbvztwxpkyfaluiupxvfpiej/Logs/Test/Test-ToolkitUITests-2018.11.01_03-44-34--0400.xcresult/1_Test/Diagnostics/ToolkitUITests-F72979CE-93C7-43B5-832E-1B2FEBCF19C7/ToolkitUITests-B2037FD3-6574-4D1C-AD69-6F90164A86C3/Session-ToolkitUITests-2018-11-01_034434-fXaAPg.log
INFO [2018-11-01 03:44:34.78]: ▸ 2018-11-01 03:44:34.780 xcodebuild[34938:19323519] [MT] IDETestOperationsObserverDebug: (49E90D64-B4C5-4B7B-BDBB-6705D9129211) Beginning test session ToolkitUITests-49E90D64-B4C5-4B7B-BDBB-6705D9129211 at 2018-11-01 03:44:34.781 with Xcode 10A255 on target {
INFO [2018-11-01 03:44:34.78]: ▸ SimDevice: iPhone X (599DE921-5C65-43FD-A852-E3D28791505D, iOS 12.0, Shutdown)
INFO [2018-11-01 03:44:34.78]: ▸ } (12.0 (16A366))
INFO [2018-11-01 03:44:46.44]: ▸ Selected tests
INFO [2018-11-01 03:44:46.44]: ▸ Test Suite ToolkitUITests.xctest started
INFO [2018-11-01 03:44:46.44]: ▸ ActiveContactsTests
INFO [2018-11-01 03:45:36.12]: ▸ ✓ testSortByCompanyName (49.671 seconds)
INFO [2018-11-01 03:46:25.57]: ▸ ✓ testSortByEmailAddress (49.454 seconds)
INFO [2018-11-01 03:47:16.49]: ▸ ✓ testViewContactDetails (50.919 seconds)
INFO [2018-11-01 03:47:16.53]: ▸ CampaignMetricsTests
INFO [2018-11-01 03:48:15.49]: ▸ ✓ testResendCampaignFlow (58.998 seconds)
INFO [2018-11-01 03:48:15.53]: ▸ ContactDetailsTests
INFO [2018-11-01 03:49:13.68]: ▸ ✓ testEditContactFromContactDetailsPage (58.190 seconds)
INFO [2018-11-01 03:50:13.22]: ▸ ✓ testVerifyAtleastOneEmailListError (59.538 seconds)
INFO [2018-11-01 03:51:22.68]: ▸ ✓ testVerifyModifyingEmailLists (69.455 seconds)
INFO [2018-11-01 03:52:13.87]: ▸ ✓ testViewContactDetails (51.192 seconds)
INFO [2018-11-01 03:53:05.18]: ▸ ✓ testViewEmailListsDetails (51.311 seconds)
INFO [2018-11-01 03:53:05.22]: ▸ ContactsAcceptanceTests
INFO [2018-11-01 03:54:38.70]: ▸ ✓ testContactsUserScenario1 (93.513 seconds)
INFO [2018-11-01 03:55:48.32]: ▸ ✓ testContactsUserScenario2 (69.621 seconds)
INFO [2018-11-01 03:56:59.98]: ▸ ✓ testContactsUserScenario3 (71.663 seconds)
INFO [2018-11-01 03:58:27.51]: ▸ ✓ testContactsUserScenario4 (87.529 seconds)
INFO [2018-11-01 03:59:51.60]: ▸ ✓ testContactsUserScenario5 (84.084 seconds)
INFO [2018-11-01 03:59:51.65]: ▸ DraftDashboardTests
INFO [2018-11-01 04:01:21.20]: ▸ ✓ testRemoveCampaignDetails (89.599 seconds)
INFO [2018-11-01 04:02:38.69]: ▸ ✗ testSendTestCampaign, Application state is still unknown for com.constantcontact.toolkit.dev (pid:46250)
INFO [2018-11-01 04:02:43.79]: ▸ ✗ testViewDraftCampaignDetails, Application state is still unknown for com.constantcontact.toolkit.dev (pid:46250)
INFO [2018-11-01 04:02:48.85]: ▸ ✗ testViewDraftCampaignDetails, Application state is still unknown for com.constantcontact.toolkit.dev (pid:46250)
INFO [2018-11-01 04:02:48.85]: ▸ ScheduledCampaignTests
INFO [2018-11-01 04:02:51.48]: ▸ ✗ testScheduleCampaignButtonLinkImgBlocks, Unexpected test interruption without any failures reported
INFO [2018-11-01 04:02:56.21]: ▸ 2018-11-01 04:02:56.218 xcodebuild[34938:19369686] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
INFO [2018-11-01 04:02:56.21]: ▸ /Users/mobileteam/Library/Developer/Xcode/DerivedData/Toolkit-deqclbvztwxpkyfaluiupxvfpiej/Logs/Test/Test-ToolkitUITests-2018.11.01_03-44-34--0400.xcresult/1_Test/Diagnostics/ToolkitUITests-F72979CE-93C7-43B5-832E-1B2FEBCF19C7/ToolkitUITests-B2037FD3-6574-4D1C-AD69-6F90164A86C3/Session-ToolkitUITests-2018-11-01_040256-70IpzL.log
INFO [2018-11-01 04:02:56.21]: ▸ 2018-11-01 04:02:56.219 xcodebuild[34938:19323519] [MT] IDETestOperationsObserverDebug: (EA876BB3-C635-43B3-9345-02F9EC9F3B52) Beginning test session ToolkitUITests-EA876BB3-C635-43B3-9345-02F9EC9F3B52 at 2018-11-01 04:02:56.219 with Xcode 10A255 on target {
INFO [2018-11-01 04:02:56.22]: ▸ SimDevice: iPhone X (599DE921-5C65-43FD-A852-E3D28791505D, iOS 12.0, Booted)
INFO [2018-11-01 04:02:56.22]: ▸ } (12.0 (16A366))
INFO [2018-11-01 04:03:00.89]: ▸ ✗ testScheduleCampaignButtonLinkImgBlocks, Test crashed
INFO [2018-11-01 04:03:00.90]: ▸ Selected tests
INFO [2018-11-01 04:03:00.90]: ▸ Test Suite ToolkitUITests.xctest started
INFO [2018-11-01 04:03:00.90]: ▸ ScheduledCampaignTests
INFO [2018-11-01 04:04:09.37]: ▸ ✗ testScheduleCampaignTitleTextBlocks, Failed to find "Drafts" Other after 30.0 seconds
INFO [2018-11-01 04:04:09.42]: ▸ ✗ testScheduleCampaignTitleTextBlocks, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Drafts" Other".
INFO [2018-11-01 04:05:58.18]: ▸ ✓ testScheduleDraftCampaign (106.103 seconds)
INFO [2018-11-01 04:07:55.15]: ▸ ✓ testScheduleNoSendToListError (116.967 seconds)
INFO [2018-11-01 04:09:19.48]: ▸ ✗ testUnschueduleCampaign, Failed to find "Edit Email" Other after 10.0 seconds
INFO [2018-11-01 04:09:19.54]: ▸ ✗ testUnschueduleCampaign, Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Edit Email" Other".
INFO [2018-11-01 04:09:27.11]: ▸ SearchContactsTests
INFO [2018-11-01 04:10:18.50]: ▸ ✓ testTapSearch (51.388 seconds)
INFO [2018-11-01 04:11:09.28]: ▸ ✓ testValidContactSearch (50.776 seconds)
INFO [2018-11-01 04:12:17.50]: ▸ ✓ testVerifySortInContactsSearch (68.217 seconds)
INFO [2018-11-01 04:13:08.63]: ▸ ✓ testVerifyZeroSearchResults (51.134 seconds)
INFO [2018-11-01 04:14:09.00]: ▸ ✓ testViewContactDetailsInContactsSearch (60.368 seconds)
INFO [2018-11-01 04:14:09.03]: ▸ ToolkitUITests.ScheduledCampaignTests
INFO [2018-11-01 04:14:09.03]: ▸ testScheduleCampaignTitleTextBlocks, Failed to find "Drafts" Other after 30.0 seconds
INFO [2018-11-01 04:14:09.03]: ▸ DraftDetailsPage.swift:28
INFO [2018-11-01 04:14:09.03]: ▸ testScheduleCampaignTitleTextBlocks, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Drafts" Other".
INFO [2018-11-01 04:14:09.03]: ▸ HelperExtensions.swift:32
INFO [2018-11-01 04:14:09.03]: ▸ testUnschueduleCampaign, Failed to find "Edit Email" Other after 10.0 seconds
INFO [2018-11-01 04:14:09.03]: ▸ EditDraftPage.swift:20
INFO [2018-11-01 04:14:09.03]: ▸ testUnschueduleCampaign, Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Edit Email" Other".
INFO [2018-11-01 04:14:09.03]: ▸ HelperExtensions.swift:32
INFO [2018-11-01 04:14:09.04]: ▸ Executed 27 tests, with 8 failures (0 unexpected) in 668.096 (668.108) seconds
INFO [2018-11-01 04:14:09.04]: ▸
INFO [2018-11-01 04:14:20.80]: ▸ 2018-11-01 04:14:20.803 xcodebuild[34938:19323519] [MT] IDETestOperationsObserverDebug: 1786.029 elapsed -- Testing started completed.
INFO [2018-11-01 04:14:20.80]: ▸ 2018-11-01 04:14:20.803 xcodebuild[34938:19323519] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
INFO [2018-11-01 04:14:20.80]: ▸ 2018-11-01 04:14:20.803 xcodebuild[34938:19323519] [MT] IDETestOperationsObserverDebug: 1786.029 sec, +1786.029 sec -- end
INFO [2018-11-01 04:14:21.22]: ▸ 2018-11-01 04:14:21.220 xcodebuild[54070:19402755] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
INFO [2018-11-01 04:14:23.10]: ▸ An error was encountered processing the command (domain=com.apple.CoreSimulator.SimError, code=164):
INFO [2018-11-01 04:14:23.10]: ▸ Invalid device state
INFO [2018-11-01 04:14:23.10]: ▸ Underlying error (domain=NSMachErrorDomain, code=-308):
INFO [2018-11-01 04:14:23.11]: ▸ The operation couldn’t be completed. (Mach error -308 - (ipc/mig) server died)
INFO [2018-11-01 04:14:25.13]: ▸ Failing tests:
INFO [2018-11-01 04:14:25.13]: ▸ DraftDashboardTests.testSendTestCampaign()
INFO [2018-11-01 04:14:25.13]: ▸ DraftDashboardTests.testViewDraftCampaignDetails()
INFO [2018-11-01 04:14:25.13]: ▸ DraftDashboardTests.testViewDraftCampaignDetails()
INFO [2018-11-01 04:14:25.13]: ▸ ScheduledCampaignTests.testScheduleCampaignButtonLinkImgBlocks()
INFO [2018-11-01 04:14:25.13]: ▸ ScheduledCampaignTests.testScheduleCampaignTitleTextBlocks()
INFO [2018-11-01 04:14:25.13]: ▸ ScheduledCampaignTests.testScheduleCampaignTitleTextBlocks()
INFO [2018-11-01 04:14:25.13]: ▸ ScheduledCampaignTests.testUnschueduleCampaign()
INFO [2018-11-01 04:14:25.13]: ▸ ScheduledCampaignTests.testUnschueduleCampaign()
INFO [2018-11-01 04:14:25.13]: ▸ * TEST EXECUTE FAILED *
2018-11-01 03:44:34.646 xcodebuild[34972:19323662] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
No devices are booted.
2018-11-01 03:44:34.780 xcodebuild[34938:19323531] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
/Users/mobileteam/Library/Developer/Xcode/DerivedData/Toolkit-deqclbvztwxpkyfaluiupxvfpiej/Logs/Test/Test-ToolkitUITests-2018.11.01_03-44-34--0400.xcresult/1_Test/Diagnostics/ToolkitUITests-F72979CE-93C7-43B5-832E-1B2FEBCF19C7/ToolkitUITests-B2037FD3-6574-4D1C-AD69-6F90164A86C3/Session-ToolkitUITests-2018-11-01_034434-fXaAPg.log
2018-11-01 03:44:34.780 xcodebuild[34938:19323519] [MT] IDETestOperationsObserverDebug: (49E90D64-B4C5-4B7B-BDBB-6705D9129211) Beginning test session ToolkitUITests-49E90D64-B4C5-4B7B-BDBB-6705D9129211 at 2018-11-01 03:44:34.781 with Xcode 10A255 on target {
SimDevice: iPhone X (599DE921-5C65-43FD-A852-E3D28791505D, iOS 12.0, Shutdown)
} (12.0 (16A366))
Selected tests
Test Suite ToolkitUITests.xctest started
ActiveContactsTests
✓ testSortByCompanyName (49.671 seconds)
✓ testSortByEmailAddress (49.454 seconds)
✓ testViewContactDetails (50.919 seconds)
CampaignMetricsTests
✓ testResendCampaignFlow (58.998 seconds)
ContactDetailsTests
✓ testEditContactFromContactDetailsPage (58.190 seconds)
✓ testVerifyAtleastOneEmailListError (59.538 seconds)
✓ testVerifyModifyingEmailLists (69.455 seconds)
✓ testViewContactDetails (51.192 seconds)
✓ testViewEmailListsDetails (51.311 seconds)
ContactsAcceptanceTests
✓ testContactsUserScenario1 (93.513 seconds)
✓ testContactsUserScenario2 (69.621 seconds)
✓ testContactsUserScenario3 (71.663 seconds)
✓ testContactsUserScenario4 (87.529 seconds)
✓ testContactsUserScenario5 (84.084 seconds)
DraftDashboardTests
✓ testRemoveCampaignDetails (89.599 seconds)
✗ testSendTestCampaign, Application state is still unknown for com.constantcontact.toolkit.dev (pid:46250)
✗ testViewDraftCampaignDetails, Application state is still unknown for com.constantcontact.toolkit.dev (pid:46250)
✗ testViewDraftCampaignDetails, Application state is still unknown for com.constantcontact.toolkit.dev (pid:46250)
ScheduledCampaignTests
✗ testScheduleCampaignButtonLinkImgBlocks, Unexpected test interruption without any failures reported
2018-11-01 04:02:56.218 xcodebuild[34938:19369686] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
/Users/mobileteam/Library/Developer/Xcode/DerivedData/Toolkit-deqclbvztwxpkyfaluiupxvfpiej/Logs/Test/Test-ToolkitUITests-2018.11.01_03-44-34--0400.xcresult/1_Test/Diagnostics/ToolkitUITests-F72979CE-93C7-43B5-832E-1B2FEBCF19C7/ToolkitUITests-B2037FD3-6574-4D1C-AD69-6F90164A86C3/Session-ToolkitUITests-2018-11-01_040256-70IpzL.log
2018-11-01 04:02:56.219 xcodebuild[34938:19323519] [MT] IDETestOperationsObserverDebug: (EA876BB3-C635-43B3-9345-02F9EC9F3B52) Beginning test session ToolkitUITests-EA876BB3-C635-43B3-9345-02F9EC9F3B52 at 2018-11-01 04:02:56.219 with Xcode 10A255 on target {
SimDevice: iPhone X (599DE921-5C65-43FD-A852-E3D28791505D, iOS 12.0, Booted)
} (12.0 (16A366))
✗ testScheduleCampaignButtonLinkImgBlocks, Test crashed
Selected tests
Test Suite ToolkitUITests.xctest started
ScheduledCampaignTests
✗ testScheduleCampaignTitleTextBlocks, Failed to find "Drafts" Other after 30.0 seconds
✗ testScheduleCampaignTitleTextBlocks, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Drafts" Other".
✓ testScheduleDraftCampaign (106.103 seconds)
✓ testScheduleNoSendToListError (116.967 seconds)
✗ testUnschueduleCampaign, Failed to find "Edit Email" Other after 10.0 seconds
✗ testUnschueduleCampaign, Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Edit Email" Other".
SearchContactsTests
✓ testTapSearch (51.388 seconds)
✓ testValidContactSearch (50.776 seconds)
✓ testVerifySortInContactsSearch (68.217 seconds)
✓ testVerifyZeroSearchResults (51.134 seconds)
✓ testViewContactDetailsInContactsSearch (60.368 seconds)

ToolkitUITests.ScheduledCampaignTests
testScheduleCampaignTitleTextBlocks, Failed to find "Drafts" Other after 30.0 seconds
DraftDetailsPage.swift:28

testScheduleCampaignTitleTextBlocks, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Drafts" Other".
HelperExtensions.swift:32

testUnschueduleCampaign, Failed to find "Edit Email" Other after 10.0 seconds
EditDraftPage.swift:20

testUnschueduleCampaign, Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Edit Email" Other".
HelperExtensions.swift:32

 Executed 27 tests, with 8 failures (0 unexpected) in 668.096 (668.108) seconds

2018-11-01 04:14:20.803 xcodebuild[34938:19323519] [MT] IDETestOperationsObserverDebug: 1786.029 elapsed -- Testing started completed.
2018-11-01 04:14:20.803 xcodebuild[34938:19323519] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
2018-11-01 04:14:20.803 xcodebuild[34938:19323519] [MT] IDETestOperationsObserverDebug: 1786.029 sec, +1786.029 sec -- end
2018-11-01 04:14:21.220 xcodebuild[54070:19402755] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
An error was encountered processing the command (domain=com.apple.CoreSimulator.SimError, code=164):
Invalid device state
Underlying error (domain=NSMachErrorDomain, code=-308):
The operation couldn’t be completed. (Mach error -308 - (ipc/mig) server died)
Failing tests:
DraftDashboardTests.testSendTestCampaign()
DraftDashboardTests.testViewDraftCampaignDetails()
DraftDashboardTests.testViewDraftCampaignDetails()
ScheduledCampaignTests.testScheduleCampaignButtonLinkImgBlocks()
ScheduledCampaignTests.testScheduleCampaignTitleTextBlocks()
ScheduledCampaignTests.testScheduleCampaignTitleTextBlocks()
ScheduledCampaignTests.testUnschueduleCampaign()
ScheduledCampaignTests.testUnschueduleCampaign()
* TEST EXECUTE FAILED *
ERROR [2018-11-01 04:14:25.15]: Exit status: 65
+--------------------+----+
| Test Results |
+--------------------+----+
| Number of tests | 31 |
| Number of failures | 9 |
+--------------------+----+

DEBUG [2018-11-01 04:14:25.66]: Scan failed with Tests have failed
INFO [2018-11-01 04:14:25.68]: Re-running scan on only failed tests
INFO [2018-11-01 04:14:29.22]: $ xcodebuild -showBuildSettings -workspace /Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/Toolkit.xcworkspace -scheme ToolkitUITests
DEBUG [2018-11-01 04:14:34.20]: Fetching available simulator devices
INFO [2018-11-01 04:14:34.89]: $ xcodebuild -showBuildSettings -workspace /Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/Toolkit.xcworkspace -scheme ToolkitUITests
DEBUG [2018-11-01 04:14:38.75]: Fetching available simulator devices

INFO [2018-11-01 04:14:39.25]: $ set -o pipefail && env NSUnbufferedIO=YES xcodebuild -workspace /Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/Toolkit.xcworkspace -scheme ToolkitUITests -destination 'platform=iOS Simulator,id=599DE921-5C65-43FD-A852-E3D28791505D' -derivedDataPath '/Users/mobileteam/Library/Developer/Xcode/DerivedData/Toolkit-deqclbvztwxpkyfaluiupxvfpiej' -only-testing:ToolkitUITests/DraftDashboardTests/testSendTestCampaign -only-testing:ToolkitUITests/DraftDashboardTests/testViewDraftCampaignDetails -only-testing:ToolkitUITests/DraftDashboardTests/testViewDraftCampaignDetails -only-testing:ToolkitUITests/ScheduledCampaignTests/testScheduleCampaignButtonLinkImgBlocks -only-testing:ToolkitUITests/ScheduledCampaignTests/testScheduleCampaignButtonLinkImgBlocks -only-testing:ToolkitUITests/ScheduledCampaignTests/testScheduleCampaignTitleTextBlocks -only-testing:ToolkitUITests/ScheduledCampaignTests/testScheduleCampaignTitleTextBlocks -only-testing:ToolkitUITests/ScheduledCampaignTests/testUnschueduleCampaign -only-testing:ToolkitUITests/ScheduledCampaignTests/testUnschueduleCampaign test-without-building | tee '/Users/mobileteam/Library/Logs/scan/Toolkit.Dev-ToolkitUITests.log' | xcpretty --report html --output '/Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/reports/ui-tests/report-5.html' --report junit --output '/Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/reports/ui-tests/report-5.junit' --report junit --output '/var/folders/dk/jfnrj9z13wdckb084w7xxnbc0000gq/T/junit_report20181101-86392-1ku65tn'
INFO [2018-11-01 04:14:39.25]: ▸ Loading...
INFO [2018-11-01 04:14:41.78]: ▸ 2018-11-01 04:14:41.789 xcodebuild[54361:19404052] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
INFO [2018-11-01 04:14:41.91]: ▸ No devices are booted.
INFO [2018-11-01 04:14:41.92]: ▸ 2018-11-01 04:14:41.923 xcodebuild[54327:19403913] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
INFO [2018-11-01 04:14:41.92]: ▸ /Users/mobileteam/Library/Developer/Xcode/DerivedData/Toolkit-deqclbvztwxpkyfaluiupxvfpiej/Logs/Test/Test-ToolkitUITests-2018.11.01_04-14-41--0400.xcresult/1_Test/Diagnostics/ToolkitUITests-380A74E3-C0F8-420C-837D-F40560E071AD/ToolkitUITests-3135564F-8541-49E6-B1B1-35ED66BA1A50/Session-ToolkitUITests-2018-11-01_041441-nyLa33.log
INFO [2018-11-01 04:14:41.92]: ▸ 2018-11-01 04:14:41.924 xcodebuild[54327:19403909] [MT] IDETestOperationsObserverDebug: (231530AB-9BC2-4672-8FDF-656169BDC5A4) Beginning test session ToolkitUITests-231530AB-9BC2-4672-8FDF-656169BDC5A4 at 2018-11-01 04:14:41.924 with Xcode 10A255 on target {
INFO [2018-11-01 04:14:41.92]: ▸ SimDevice: iPhone X (599DE921-5C65-43FD-A852-E3D28791505D, iOS 12.0, Shutdown)
INFO [2018-11-01 04:14:41.92]: ▸ } (12.0 (16A366))
INFO [2018-11-01 04:14:55.01]: ▸ Selected tests
INFO [2018-11-01 04:14:55.01]: ▸ Test Suite ToolkitUITests.xctest started
INFO [2018-11-01 04:14:55.01]: ▸ DraftDashboardTests
INFO [2018-11-01 04:16:10.63]: ▸ ✓ testSendTestCampaign (75.612 seconds)
INFO [2018-11-01 04:17:18.75]: ▸ ✓ testViewDraftCampaignDetails (68.127 seconds)
INFO [2018-11-01 04:17:18.79]: ▸ ScheduledCampaignTests
INFO [2018-11-01 04:20:12.89]: ▸ ✓ testScheduleCampaignButtonLinkImgBlocks (174.136 seconds)
INFO [2018-11-01 04:22:50.20]: ▸ ✓ testScheduleCampaignTitleTextBlocks (157.309 seconds)
INFO [2018-11-01 04:25:39.98]: ▸ ✓ testUnschueduleCampaign (169.783 seconds)
INFO [2018-11-01 04:25:40.04]: ▸ Executed 5 tests, with 0 failures (0 unexpected) in 644.967 (644.972) seconds
INFO [2018-11-01 04:25:40.04]: ▸
INFO [2018-11-01 04:25:40.40]: ▸ 2018-11-01 04:25:40.406 xcodebuild[54327:19403909] [MT] IDETestOperationsObserverDebug: 658.489 elapsed -- Testing started completed.
INFO [2018-11-01 04:25:40.40]: ▸ 2018-11-01 04:25:40.407 xcodebuild[54327:19403909] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
INFO [2018-11-01 04:25:40.40]: ▸ 2018-11-01 04:25:40.407 xcodebuild[54327:19403909] [MT] IDETestOperationsObserverDebug: 658.489 sec, +658.489 sec -- end
INFO [2018-11-01 04:25:40.73]: ▸ 2018-11-01 04:25:40.739 xcodebuild[61108:19430711] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
INFO [2018-11-01 04:25:42.35]: ▸ An error was encountered processing the command (domain=com.apple.CoreSimulator.SimError, code=164):
INFO [2018-11-01 04:25:42.35]: ▸ Invalid device state
INFO [2018-11-01 04:25:42.35]: ▸ Underlying error (domain=NSMachErrorDomain, code=-308):
INFO [2018-11-01 04:25:42.36]: ▸ The operation couldn’t be completed. (Mach error -308 - (ipc/mig) server died)
INFO [2018-11-01 04:25:43.88]: ▸ Test execute Succeeded
+--------------------+---+
| Test Results |
+--------------------+---+
| Number of tests | 5 |
| Number of failures | 0 |
+--------------------+---+

+--------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| multi_scan results |
+--------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| result | true |
| total_tests | 87 |
| passing_testcount | 77 |
| failed_testcount | 10 |
| failed_tests | ["ToolkitUITests/EditContactTests/testDeleteContact", "ToolkitUITests/EditContactTests/testEditExistingContact", "ToolkitUITests/EditContactTests/testEditExistingEmailAddress", "ToolkitUITests/LibraryPageTest/testBrowseAllFolder", "ToolkitUITests/LibraryPageTest/testBrowseAllImages", "ToolkitUITests/LibraryPageTest/testEmptyFolder", "ToolkitUITests/DraftDashboardTests/testViewDraftCampaignDetails", "ToolkitUITests/ScheduledCampaignTests/testScheduleCampaignButtonLinkImgBlocks", "ToolkitUITests/ScheduledCampaignTests/testScheduleCampaignTitleTextBlocks", "ToolkitUITests/ScheduledCampaignTests/testUnschueduleCampaign"] |
| failure_details.ToolkitUITests/EditContactTests/testDeleteContact.message | Asynchronous wait failed: Exceeded timeout of 5 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Contacts" Other". |
| failure_details.ToolkitUITests/EditContactTests/testDeleteContact.location | |
| | HelperExtensions.swift:32 |
| | |
| failure_details.ToolkitUITests/EditContactTests/testEditExistingContact.message | Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button". |
| failure_details.ToolkitUITests/EditContactTests/testEditExistingContact.location | |
| | HelperExtensions.swift:32 |
| | |
| failure_details.ToolkitUITests/EditContactTests/testEditExistingEmailAddress.message | Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button". |
| failure_details.ToolkitUITests/EditContactTests/testEditExistingEmailAddress.location | |
| | HelperExtensions.swift:32 |
| | |
| failure_details.ToolkitUITests/LibraryPageTest/testBrowseAllFolder.message | Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button". |
| failure_details.ToolkitUITests/LibraryPageTest/testBrowseAllFolder.location | |
| | HelperExtensions.swift:32 |
| | |
| failure_details.ToolkitUITests/LibraryPageTest/testBrowseAllImages.message | Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button". |
| failure_details.ToolkitUITests/LibraryPageTest/testBrowseAllImages.location | |
| | HelperExtensions.swift:32 |
| | |
| failure_details.ToolkitUITests/LibraryPageTest/testEmptyFolder.message | Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button". |
| failure_details.ToolkitUITests/LibraryPageTest/testEmptyFolder.location | |
| | HelperExtensions.swift:32 |
| | |
| failure_details.ToolkitUITests/DraftDashboardTests/testViewDraftCampaignDetails.message | Application state is still unknown for com.constantcontact.toolkit.dev (pid:46250) |
| failure_details.ToolkitUITests/DraftDashboardTests/testViewDraftCampaignDetails.location | |
| | DraftDashboardTests.swift:30 |
| | |
| failure_details.ToolkitUITests/ScheduledCampaignTests/testScheduleCampaignButtonLinkImgBlocks.message | Test crashed |
| failure_details.ToolkitUITests/ScheduledCampaignTests/testScheduleCampaignButtonLinkImgBlocks.location | |
| | n/a |
| | |
| failure_details.ToolkitUITests/ScheduledCampaignTests/testScheduleCampaignTitleTextBlocks.message | Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Drafts" Other". |
| failure_details.ToolkitUITests/ScheduledCampaignTests/testScheduleCampaignTitleTextBlocks.location | |
| | HelperExtensions.swift:32 |
| | |
| failure_details.ToolkitUITests/ScheduledCampaignTests/testUnschueduleCampaign.message | Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Edit Email" Other". |
| failure_details.ToolkitUITests/ScheduledCampaignTests/testUnschueduleCampaign.location | |
| | HelperExtensions.swift:32 |
| | |
| total_retry_count | 2 |
| report_files | ["/Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/reports/ui-tests/report.junit", "/Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/reports/ui-tests/report.html"] |
+--------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

INFO [2018-11-01 04:25:45.52]: ---------------------
INFO [2018-11-01 04:25:45.52]: --- Step: slather ---
INFO [2018-11-01 04:25:45.52]: ---------------------
INFO [2018-11-01 04:25:45.52]: gem 'slather' is already installed
INFO [2018-11-01 04:25:45.52]: $ slather coverage --jenkins --cobertura-xml --output-directory reports/uitest-coverage --scheme ToolkitUITests --workspace Toolkit.xcworkspace --binary-basename Toolkit.Dev --source-files EMM/ToolkitSwift/**/*.swift Toolkit.xcodeproj
INFO [2018-11-01 04:25:46.74]: ▸ Slathering...
INFO [2018-11-01 04:25:52.87]: ▸ Slathered
INFO [2018-11-01 04:25:52.88]: ------------------
INFO [2018-11-01 04:25:52.88]: --- Step: xcov ---
INFO [2018-11-01 04:25:52.88]: ------------------
INFO [2018-11-01 04:25:52.98]: gem 'xcov' is already installed
INFO [2018-11-01 04:25:53.05]: Skipping file blacklisting as no ignore file was found at path ./fastlane/.xcovignore

Environment

[INSERT OUTPUT HERE]

🐞bug 🔷collate_junit_reports

Source

Sandhu412-zz

All 42 comments

multi_scan parses the final junit test results found at /Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/reports/ui-tests/report.junit. Can you review it and let me know if it has those tests marked as failing?

My concern is this block of output:

INFO [2018-11-01 04:25:42.35]: ▸ An error was encountered processing the command (domain=com.apple.CoreSimulator.SimError, code=164):
INFO [2018-11-01 04:25:42.35]: ▸ Invalid device state
INFO [2018-11-01 04:25:42.35]: ▸ Underlying error (domain=NSMachErrorDomain, code=-308):
INFO [2018-11-01 04:25:42.36]: ▸ The operation couldn’t be completed. (Mach error -308 - (ipc/mig) server died)
INFO [2018-11-01 04:25:43.88]: ▸ Test execute Succeeded

To me, it could indicate that the final junit file, no. 5, was unable to be written, and so multi_scan could not gather that information to merge into the other junit result files.

I see an issue around XCODE_DEVELOPER_DIR_PATH, perhaps that is the problem. It wants a full path. Have you set that yourself?

Let's print it out in a callback that:

test_run_block = lambda do |testrun_info|
  puts "XCODE_DEVELOPER_DIR_PATH : '#{ENV['XCODE_DEVELOPER_DIR_PATH']}'"
end

lyndsey-ferguson on 1 Nov 2018

@lyndsey-ferguson -- yes, the Junit report lists those tests as failed. I use a publish Junit report plugin in the jenkins job and it does list all the 10 tests as failed.

multi_scan parses the final junit test results found at /Users/Shared/Jenkins/Home/workspace/toolkit-ios-beta-tests/reports/ui-tests/report.junit. Can you review it and let me know if it has those tests marked as failing?

No, I dont set the path.

I see an issue around XCODE_DEVELOPER_DIR_PATH, perhaps that is the problem. It wants a full path. Have you set that yourself?

Where should I place the below snippet of code? In my ui_tests lane in Fastfile?

Let's print it out in a callback that:

test_run_block = lambda do |testrun_info|
puts "XCODE_DEVELOPER_DIR_PATH : '#{ENV['XCODE_DEVELOPER_DIR_PATH']}'"
end

Sandhu412-zz on 1 Nov 2018

Sorry, you should pass that test_run_block to multi_scan. The key for that value is testrun_completed_block.

lyndsey-ferguson on 2 Nov 2018

@lyndsey-ferguson -- I added the call back and passed it to multi_scan as shown below. Please let me know if the below is correct? When I run this lane, the output for "XCODE_DEVELOPER_DIR_PATH :" is blank. But what does that indicate and how is it related to multi_scan not consolidating the junit test results and marking the passed ones as failed? Please clarify.

```desc "Runs all the TK UI Tests"
lane :toolkit_ui_tests do
ENV['FASTLANE_XCODE_LIST_TIMEOUT'] = '120'
clear_derived_data
sh('mkdir -p ../reports/uitest-coverage/')
sh('mkdir -p ../reports/uitest-xcov-coverage/')
cocoapods(repo_update: false, silent: true) # CP repo update moved to a Jenkins task that runs once a day
test_run_block = lambda do |testrun_info|
puts "XCODE_DEVELOPER_DIR_PATH : '#{ENV['XCODE_DEVELOPER_DIR_PATH']}'"
end
multi_scan(
workspace: File.absolute_path('../Toolkit.xcworkspace'),
device: "iPhone 8",
scheme: 'ToolkitUITests',
output_directory: "reports/ui-tests",
include_simulator_logs: false,
try_count: 2, # runs only the failed tests again without building to the set count
batch_count: 3, # batches the total number of tests into set count to reduce the load on simulator
fail_build: true,
code_coverage: true,
testrun_completed_block: test_run_block
)
slather(
scheme: "ToolkitUITests",
proj: "Toolkit.xcodeproj",
workspace: "Toolkit.xcworkspace",
output_directory: "reports/uitest-coverage",
binary_basename: "Toolkit.Dev",
source_files: "EMM/ToolkitSwift/*/.swift",
cobertura_xml: true,
jenkins: true
)
end

Sandhu412-zz on 6 Nov 2018

XCODE_DEVELOPER_DIR_PATH is used to declare what the path is to the version of Xcode that is running. I saw the error in your logs and thought that perhaps you had set it to something. It looks like you may not have set it. Can you run:

printenv

from the command line to see if that environment variable has been set?

Also, run this command from the command line:

xcode-select -p

and provide me the output.

lyndsey-ferguson on 6 Nov 2018

output for printenv is below:

```TERM_PROGRAM=Apple_Terminal
ANDROID_HOME=/Users/mobileteam/Library/Android/sdk/
SHELL=/bin/bash
TERM=xterm-256color
CLICOLOR=1
TMPDIR=/var/folders/dk/jfnrj9z13wdckb084w7xxnbc0000gq/T/
Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.9TNCm0D5Rf/Render
TERM_PROGRAM_VERSION=404
OLDPWD=/Users/Shared/Jenkins/Home/workspace
TERM_SESSION_ID=BD4BEF8F-D6B3-4555-B0F0-8B5536967407
USER=mobileteam
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.suKVmwE0Fp/Listeners
LSCOLORS=GxFxCxDxBxegedabagaced
PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin:/Users/mobileteam/Library/Android/sdk/platform-tools/:/Users/mobileteam/Library/Android/sdk/tools:/Users/mobileteam/Library/Android/sdk/tools/bin
PWD=/Users/mobileteam
LANG=en_US.UTF-8
XPC_FLAGS=0x0
ITERM_ORIG_PS1=md5-909a6e7e9bd9cf38f0b2a5684acd14adh:W u$
PS1=md5-909a6e7e9bd9cf38f0b2a5684acd14adh:W u$
XPC_SERVICE_NAME=0
SHLVL=1
HOME=/Users/mobileteam
ITERM_PREV_PS1=[]md5-909a6e7e9bd9cf38f0b2a5684acd14adh:W u$ []
LOGNAME=mobileteam
SECURITYSESSIONID=186a8
_=/usr/bin/printenv

Sandhu412-zz on 7 Nov 2018

output for xcode-select -p

```MOBILEDEVMAC1:~ mobileteam$ xcode-select -p
/Applications/Xcode.app/Contents/Developer

Sandhu412-zz on 7 Nov 2018

Do you specify fail_build: false when you call multi_scan?

lyndsey-ferguson on 7 Nov 2018

Do you specify fail_build: false when you call multi_scan?

No, I specify fail_build: true in multi_scan

Sandhu412-zz on 7 Nov 2018

Ok, let's update your test_run_block to copy the test reports to the desktop to see if we get report-5.

test_run_block = lambda do |testrun_info|
  src_report_filepath = testrun_info[:report_filepath]
  dest_report_filepath = File.expand_path("~/Desktop/#{File.basename(src_report_filepath)}")
  FileUtils.cp(src_report_filepath, dest_report_filepath)
end

Run your fastlane, and tell me if it copied report-5 to the desktop. If so, what are the contents?

lyndsey-ferguson on 7 Nov 2018

@lyndsey-ferguson -- Below is the fastlane output from the ui tests with the above test_run_block. All the 5 reports are copied to the desktop and the 5th report contains the list of 27 passed tests that were "run" in that batch. I can't attach the reports, but I can email them to you if you would like to see.

As per my analysis, the issue is that the junit reports are generated and are accurate but when multi_scan generates the test results, it is for some reason not consolidating the failed runs and counting the failed tests even if they pass on the next try.

Also, at the end after all batches are run, I see that sometimes the log shows multi_scan results section but sometimes it does not (as in the case with the below log). Any reason why? This "multi_scan results" section consolidates all the test results.

```INFO [2018-11-07 13:48:29.76]: ▸ Linking ToolkitUITests
INFO [2018-11-07 13:48:30.26]: ▸ Copying OverlayConfig.plist
INFO [2018-11-07 13:48:30.26]: ▸ Copying Localizable.strings
INFO [2018-11-07 13:48:30.27]: ▸ Running script '[CP] Embed Pods Frameworks'
INFO [2018-11-07 13:48:55.53]: ▸ Running script '[CP] Copy Pods Resources'
INFO [2018-11-07 13:49:04.29]: ▸ Copying /Applications/Xcode_10.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/Library/PrivateFrameworks/XCTAutomationSupport.framework
INFO [2018-11-07 13:49:04.29]: ▸ Copying /Applications/Xcode_10.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/Library/Frameworks/XCTest.framework
INFO [2018-11-07 13:49:06.42]: Running Tests: ▸ Touching ToolkitUITests.xctest (in target: ToolkitUITests)
INFO [2018-11-07 13:49:06.52]: ▸ Test build Succeeded
+--------------------+---+
| Test Results |
+--------------------+---+
| Number of tests | 0 |
| Number of failures | 0 |
+--------------------+---+

INFO [2018-11-07 13:49:07.56]: ------------------------------------------------------
INFO [2018-11-07 13:49:07.56]: --- Starting test run on testable 'ToolkitUITests' ---
INFO [2018-11-07 13:49:07.56]: ------------------------------------------------------
INFO [2018-11-07 13:49:08.69]: $ xcodebuild -showBuildSettings -workspace /Users/sanjaneya/Documents/toolkit-ios/Toolkit.xcworkspace -scheme ToolkitUITests
WARN [2018-11-07 13:49:11.70]: Command timed out after 3 seconds on try 1 of 4, trying again with a 6 second timeout...
DEBUG [2018-11-07 13:49:15.44]: Fetching available simulator devices
INFO [2018-11-07 13:49:16.21]: $ xcodebuild -showBuildSettings -workspace /Users/sanjaneya/Documents/toolkit-ios/Toolkit.xcworkspace -scheme ToolkitUITests
WARN [2018-11-07 13:49:19.22]: Command timed out after 3 seconds on try 1 of 4, trying again with a 6 second timeout...
DEBUG [2018-11-07 13:49:23.21]: Fetching available simulator devices

INFO [2018-11-07 13:49:23.85]: $ set -o pipefail && env NSUnbufferedIO=YES xcodebuild -workspace /Users/sanjaneya/Documents/toolkit-ios/Toolkit.xcworkspace -scheme ToolkitUITests -destination 'platform=iOS Simulator,id=65B5C280-4F76-43A8-BE58-D9CC4EE7AAAA' -derivedDataPath '/Users/sanjaneya/Library/Developer/Xcode/DerivedData/Toolkit-cbjnzqipycngqehjhegjjguoghlk' -enableCodeCoverage YES -only-testing:ToolkitUITests/CCTClearcutLogEvent/testCodes -only-testing:ToolkitUITests/CCTClearcutLogEvent/testID -only-testing:ToolkitUITests/FIRInstanceIDTokenOperation/testBlock -only-testing:ToolkitUITests/PHTPhenotypeDataFlag/testOverride -only-testing:ToolkitUITests/PHTPhenotypeNumberFlag/testOverride -only-testing:ToolkitUITests/PHTPhenotypeStringFlag/testOverride -only-testing:ToolkitUITests/OverlayTests/testOverlayCTA -only-testing:ToolkitUITests/OverlayTests/testSwipeOverlays -only-testing:ToolkitUITests/SideMenuTests/testSideMenuLinks -only-testing:ToolkitUITests/EditDraftTests/testEditDraftsLink -only-testing:ToolkitUITests/EditDraftTests/testEditDraftsText -only-testing:ToolkitUITests/EditDraftTests/testEditDraftsImage -only-testing:ToolkitUITests/EditDraftTests/testEditDraftsTitle -only-testing:ToolkitUITests/EditDraftTests/testEditDraftsButton -only-testing:ToolkitUITests/EditDraftTests/testEditDraftsAddImageLink -only-testing:ToolkitUITests/EditDraftTests/testEditDraftsAddImageCamera -only-testing:ToolkitUITests/EditDraftTests/testEditDraftsAddImageLibrary -only-testing:ToolkitUITests/EditDraftTests/testEditDraftsAddImageLinkError -only-testing:ToolkitUITests/EmailListTests/testExistingEmailListNameError -only-testing:ToolkitUITests/EmailListTests/testCreateNewEmailList -only-testing:ToolkitUITests/EmailListTests/testNavigateToEmailListDetails -only-testing:ToolkitUITests/LoginPageTests/testEncodedCharLoginError -only-testing:ToolkitUITests/LoginPageTests/testForMultiUserLogin -only-testing:ToolkitUITests/LoginPageTests/testEmptyUserNameEmptyPassword -only-testing:ToolkitUITests/LoginPageTests/testInValidPasswordLogin -only-testing:ToolkitUITests/LoginPageTests/testInValidUserNameLogin -only-testing:ToolkitUITests/LoginPageTests/testValidLogin -only-testing:ToolkitUITests/LoginPageTests/testEmptyPassword test-without-building | tee '/Users/sanjaneya/Library/Logs/scan/Toolkit.Dev-ToolkitUITests.log' | xcpretty --report html --output '/Users/sanjaneya/Documents/toolkit-ios/reports/ui-tests/report.html' --report junit --output '/Users/sanjaneya/Documents/toolkit-ios/reports/ui-tests/report.junit' --report junit --output '/var/folders/nc/x2sh_3t17692n9jf3pybxt2n8c1sfm/T/junit_report20181107-2669-1uxk9df'
INFO [2018-11-07 13:49:23.85]: ▸ Loading...
INFO [2018-11-07 13:49:26.38]: ▸ 2018-11-07 13:49:26.389 xcodebuild[8805:3308347] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
INFO [2018-11-07 13:49:26.52]: ▸ No devices are booted.
INFO [2018-11-07 13:49:26.55]: ▸ 2018-11-07 13:49:26.548 xcodebuild[8796:3308271] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
INFO [2018-11-07 13:49:26.55]: ▸ /Users/sanjaneya/Library/Developer/Xcode/DerivedData/Toolkit-cbjnzqipycngqehjhegjjguoghlk/Logs/Test/Test-ToolkitUITests-2018.11.07_13-49-26--0500.xcresult/1_Test/Diagnostics/ToolkitUITests-7A57AE88-F03A-4259-B8AA-7A92E1EB8F29/ToolkitUITests-F7708721-EE60-4FC5-90D7-1C381493DDD5/Session-ToolkitUITests-2018-11-07_134926-DzN7Ix.log
INFO [2018-11-07 13:49:26.55]: ▸ 2018-11-07 13:49:26.549 xcodebuild[8796:3308261] [MT] IDETestOperationsObserverDebug: (AA49B10C-9021-47CF-90A9-B66464A9021D) Beginning test session ToolkitUITests-AA49B10C-9021-47CF-90A9-B66464A9021D at 2018-11-07 13:49:26.549 with Xcode 10A255 on target {
INFO [2018-11-07 13:49:26.55]: ▸ SimDevice: iPhone 8 (65B5C280-4F76-43A8-BE58-D9CC4EE7AAAA, iOS 12.0, Shutdown)
INFO [2018-11-07 13:49:26.55]: ▸ } (12.0 (16A366))
INFO [2018-11-07 13:49:52.14]: ▸ Selected tests
INFO [2018-11-07 13:49:52.14]: ▸ Test Suite ToolkitUITests.xctest started
INFO [2018-11-07 13:49:52.14]: ▸ EditDraftTests
INFO [2018-11-07 13:50:36.86]: ▸ ✗ testEditDraftsAddImageCamera, Failed to find "Log In" Button after 30.0 seconds
INFO [2018-11-07 13:50:36.87]: ▸ ✗ testEditDraftsAddImageCamera, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
INFO [2018-11-07 13:51:16.49]: ▸ ✗ testEditDraftsAddImageLibrary, Failed to find "Log In" Button after 30.0 seconds
INFO [2018-11-07 13:51:16.51]: ▸ ✗ testEditDraftsAddImageLibrary, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
INFO [2018-11-07 13:52:43.39]: ▸ ✗ testEditDraftsAddImageLink, Failed to find "Link" NavigationBar after 10.0 seconds
INFO [2018-11-07 13:52:43.41]: ▸ ✗ testEditDraftsAddImageLink, Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Link" NavigationBar".
INFO [2018-11-07 13:55:18.16]: ▸ ✓ testEditDraftsAddImageLinkError (147.510 seconds)
INFO [2018-11-07 13:56:54.26]: ▸ ✓ testEditDraftsButton (96.098 seconds)
INFO [2018-11-07 13:58:23.96]: ▸ ✓ testEditDraftsImage (89.704 seconds)
INFO [2018-11-07 13:59:54.43]: ▸ ✓ testEditDraftsLink (90.469 seconds)
INFO [2018-11-07 14:01:26.82]: ▸ ✓ testEditDraftsText (92.384 seconds)
INFO [2018-11-07 14:02:59.76]: ▸ ✓ testEditDraftsTitle (92.941 seconds)
INFO [2018-11-07 14:02:59.81]: ▸ EmailListTests
INFO [2018-11-07 14:03:51.41]: ▸ ✓ testCreateNewEmailList (51.652 seconds)
INFO [2018-11-07 14:04:47.46]: ▸ ✓ testExistingEmailListNameError (56.045 seconds)
INFO [2018-11-07 14:05:30.33]: ▸ ✓ testNavigateToEmailListDetails (42.869 seconds)
INFO [2018-11-07 14:05:30.35]: ▸ LoginPageTests
INFO [2018-11-07 14:05:53.93]: ▸ ✓ testEmptyPassword (23.600 seconds)
INFO [2018-11-07 14:06:13.07]: ▸ ✓ testEmptyUserNameEmptyPassword (19.144 seconds)
INFO [2018-11-07 14:06:32.25]: ▸ ✓ testEncodedCharLoginError (19.179 seconds)
INFO [2018-11-07 14:07:25.49]: ▸ ✓ testForMultiUserLogin (53.237 seconds)
INFO [2018-11-07 14:07:45.89]: ▸ ✓ testInValidPasswordLogin (20.399 seconds)
INFO [2018-11-07 14:08:05.23]: ▸ ✓ testInValidUserNameLogin (19.341 seconds)
INFO [2018-11-07 14:08:37.35]: ▸ ✓ testValidLogin (32.121 seconds)
INFO [2018-11-07 14:08:37.37]: ▸ OverlayTests
INFO [2018-11-07 14:09:09.03]: ▸ ✓ testOverlayCTA (31.677 seconds)
INFO [2018-11-07 14:09:40.86]: ▸ ✓ testSwipeOverlays (31.833 seconds)
INFO [2018-11-07 14:09:40.88]: ▸ SideMenuTests
INFO [2018-11-07 14:10:22.79]: ▸ ✓ testSideMenuLinks (41.932 seconds)
INFO [2018-11-07 14:10:22.81]: ▸ ToolkitUITests.EditDraftTests
INFO [2018-11-07 14:10:22.81]: ▸ testEditDraftsAddImageCamera, Failed to find "Log In" Button after 30.0 seconds
INFO [2018-11-07 14:10:22.81]: ▸ LandingPage.swift:25
INFO [2018-11-07 14:10:22.81]: ▸ testEditDraftsAddImageCamera, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
INFO [2018-11-07 14:10:22.82]: ▸ HelperExtensions.swift:32
INFO [2018-11-07 14:10:22.82]: ▸ testEditDraftsAddImageLibrary, Failed to find "Log In" Button after 30.0 seconds
INFO [2018-11-07 14:10:22.82]: ▸ LandingPage.swift:25
INFO [2018-11-07 14:10:22.82]: ▸ testEditDraftsAddImageLibrary, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
INFO [2018-11-07 14:10:22.82]: ▸ HelperExtensions.swift:32
INFO [2018-11-07 14:10:22.82]: ▸ testEditDraftsAddImageLink, Failed to find "Link" NavigationBar after 10.0 seconds
INFO [2018-11-07 14:10:22.82]: ▸ EditImageLinkPage.swift:20
INFO [2018-11-07 14:10:22.82]: ▸ testEditDraftsAddImageLink, Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Link" NavigationBar".
INFO [2018-11-07 14:10:22.82]: ▸ HelperExtensions.swift:32
INFO [2018-11-07 14:10:22.82]: ▸ Executed 22 tests, with 6 failures (0 unexpected) in 1230.636 (1230.655) seconds
INFO [2018-11-07 14:10:22.82]: ▸
INFO [2018-11-07 14:10:47.08]: ▸ 2018-11-07 14:10:47.089 xcodebuild[8796:3308261] [MT] IDETestOperationsObserverDebug: 1280.560 elapsed -- Testing started completed.
INFO [2018-11-07 14:10:47.09]: ▸ 2018-11-07 14:10:47.089 xcodebuild[8796:3308261] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
INFO [2018-11-07 14:10:47.09]: ▸ 2018-11-07 14:10:47.089 xcodebuild[8796:3308261] [MT] IDETestOperationsObserverDebug: 1280.560 sec, +1280.560 sec -- end
INFO [2018-11-07 14:10:47.62]: ▸ 2018-11-07 14:10:47.626 xcodebuild[9693:3372546] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
INFO [2018-11-07 14:10:49.42]: ▸ An error was encountered processing the command (domain=NSMachErrorDomain, code=-308):
INFO [2018-11-07 14:10:49.43]: ▸ The operation couldn’t be completed. (Mach error -308 - (ipc/mig) server died)
INFO [2018-11-07 14:10:52.74]: ▸ Failing tests:
INFO [2018-11-07 14:10:52.74]: ▸ EditDraftTests.testEditDraftsAddImageCamera()
INFO [2018-11-07 14:10:52.74]: ▸ EditDraftTests.testEditDraftsAddImageCamera()
INFO [2018-11-07 14:10:52.74]: ▸ EditDraftTests.testEditDraftsAddImageLibrary()
INFO [2018-11-07 14:10:52.74]: ▸ EditDraftTests.testEditDraftsAddImageLibrary()
INFO [2018-11-07 14:10:52.74]: ▸ EditDraftTests.testEditDraftsAddImageLink()
INFO [2018-11-07 14:10:52.74]: ▸ EditDraftTests.testEditDraftsAddImageLink()
INFO [2018-11-07 14:10:52.74]: ▸ * TEST EXECUTE FAILED *
2018-11-07 13:49:26.389 xcodebuild[8805:3308347] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
No devices are booted.
2018-11-07 13:49:26.548 xcodebuild[8796:3308271] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
/Users/sanjaneya/Library/Developer/Xcode/DerivedData/Toolkit-cbjnzqipycngqehjhegjjguoghlk/Logs/Test/Test-ToolkitUITests-2018.11.07_13-49-26--0500.xcresult/1_Test/Diagnostics/ToolkitUITests-7A57AE88-F03A-4259-B8AA-7A92E1EB8F29/ToolkitUITests-F7708721-EE60-4FC5-90D7-1C381493DDD5/Session-ToolkitUITests-2018-11-07_134926-DzN7Ix.log
2018-11-07 13:49:26.549 xcodebuild[8796:3308261] [MT] IDETestOperationsObserverDebug: (AA49B10C-9021-47CF-90A9-B66464A9021D) Beginning test session ToolkitUITests-AA49B10C-9021-47CF-90A9-B66464A9021D at 2018-11-07 13:49:26.549 with Xcode 10A255 on target {
SimDevice: iPhone 8 (65B5C280-4F76-43A8-BE58-D9CC4EE7AAAA, iOS 12.0, Shutdown)
} (12.0 (16A366))
Selected tests
Test Suite ToolkitUITests.xctest started
EditDraftTests
✗ testEditDraftsAddImageCamera, Failed to find "Log In" Button after 30.0 seconds
✗ testEditDraftsAddImageCamera, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
✗ testEditDraftsAddImageLibrary, Failed to find "Log In" Button after 30.0 seconds
✗ testEditDraftsAddImageLibrary, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
✗ testEditDraftsAddImageLink, Failed to find "Link" NavigationBar after 10.0 seconds
✗ testEditDraftsAddImageLink, Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Link" NavigationBar".
✓ testEditDraftsAddImageLinkError (147.510 seconds)
✓ testEditDraftsButton (96.098 seconds)
✓ testEditDraftsImage (89.704 seconds)
✓ testEditDraftsLink (90.469 seconds)
✓ testEditDraftsText (92.384 seconds)
✓ testEditDraftsTitle (92.941 seconds)
EmailListTests
✓ testCreateNewEmailList (51.652 seconds)
✓ testExistingEmailListNameError (56.045 seconds)
✓ testNavigateToEmailListDetails (42.869 seconds)
LoginPageTests
✓ testEmptyPassword (23.600 seconds)
✓ testEmptyUserNameEmptyPassword (19.144 seconds)
✓ testEncodedCharLoginError (19.179 seconds)
✓ testForMultiUserLogin (53.237 seconds)
✓ testInValidPasswordLogin (20.399 seconds)
✓ testInValidUserNameLogin (19.341 seconds)
✓ testValidLogin (32.121 seconds)
OverlayTests
✓ testOverlayCTA (31.677 seconds)
✓ testSwipeOverlays (31.833 seconds)
SideMenuTests
✓ testSideMenuLinks (41.932 seconds)

ToolkitUITests.EditDraftTests
testEditDraftsAddImageCamera, Failed to find "Log In" Button after 30.0 seconds
LandingPage.swift:25

testEditDraftsAddImageCamera, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
HelperExtensions.swift:32

testEditDraftsAddImageLibrary, Failed to find "Log In" Button after 30.0 seconds
LandingPage.swift:25

testEditDraftsAddImageLibrary, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Log In" Button".
HelperExtensions.swift:32

testEditDraftsAddImageLink, Failed to find "Link" NavigationBar after 10.0 seconds
EditImageLinkPage.swift:20

testEditDraftsAddImageLink, Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Link" NavigationBar".
HelperExtensions.swift:32

 Executed 22 tests, with 6 failures (0 unexpected) in 1230.636 (1230.655) seconds

2018-11-07 14:10:47.089 xcodebuild[8796:3308261] [MT] IDETestOperationsObserverDebug: 1280.560 elapsed -- Testing started completed.
2018-11-07 14:10:47.089 xcodebuild[8796:3308261] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
2018-11-07 14:10:47.089 xcodebuild[8796:3308261] [MT] IDETestOperationsObserverDebug: 1280.560 sec, +1280.560 sec -- end
2018-11-07 14:10:47.626 xcodebuild[9693:3372546] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
An error was encountered processing the command (domain=NSMachErrorDomain, code=-308):
The operation couldn’t be completed. (Mach error -308 - (ipc/mig) server died)
Failing tests:
EditDraftTests.testEditDraftsAddImageCamera()
EditDraftTests.testEditDraftsAddImageCamera()
EditDraftTests.testEditDraftsAddImageLibrary()
EditDraftTests.testEditDraftsAddImageLibrary()
EditDraftTests.testEditDraftsAddImageLink()
EditDraftTests.testEditDraftsAddImageLink()
* TEST EXECUTE FAILED *
ERROR [2018-11-07 14:10:52.79]: Exit status: 65
+--------------------+----+
| Test Results |
+--------------------+----+
| Number of tests | 25 |
| Number of failures | 6 |
+--------------------+----+

DEBUG [2018-11-07 14:10:53.17]: Scan failed with Tests have failed
INFO [2018-11-07 14:10:53.29]: Re-running scan on only failed tests
INFO [2018-11-07 14:10:55.47]: $ xcodebuild -showBuildSettings -workspace /Users/sanjaneya/Documents/toolkit-ios/Toolkit.xcworkspace -scheme ToolkitUITests
WARN [2018-11-07 14:10:58.49]: Command timed out after 3 seconds on try 1 of 4, trying again with a 6 second timeout...
DEBUG [2018-11-07 14:11:01.98]: Fetching available simulator devices
INFO [2018-11-07 14:11:02.70]: $ xcodebuild -showBuildSettings -workspace /Users/sanjaneya/Documents/toolkit-ios/Toolkit.xcworkspace -scheme ToolkitUITests
WARN [2018-11-07 14:11:05.71]: Command timed out after 3 seconds on try 1 of 4, trying again with a 6 second timeout...
DEBUG [2018-11-07 14:11:08.89]: Fetching available simulator devices

INFO [2018-11-07 14:11:09.41]: $ set -o pipefail && env NSUnbufferedIO=YES xcodebuild -workspace /Users/sanjaneya/Documents/toolkit-ios/Toolkit.xcworkspace -scheme ToolkitUITests -destination 'platform=iOS Simulator,id=65B5C280-4F76-43A8-BE58-D9CC4EE7AAAA' -derivedDataPath '/Users/sanjaneya/Library/Developer/Xcode/DerivedData/Toolkit-cbjnzqipycngqehjhegjjguoghlk' -only-testing:ToolkitUITests/EditDraftTests/testEditDraftsAddImageCamera -only-testing:ToolkitUITests/EditDraftTests/testEditDraftsAddImageCamera -only-testing:ToolkitUITests/EditDraftTests/testEditDraftsAddImageLibrary -only-testing:ToolkitUITests/EditDraftTests/testEditDraftsAddImageLibrary -only-testing:ToolkitUITests/EditDraftTests/testEditDraftsAddImageLink -only-testing:ToolkitUITests/EditDraftTests/testEditDraftsAddImageLink test-without-building | tee '/Users/sanjaneya/Library/Logs/scan/Toolkit.Dev-ToolkitUITests.log' | xcpretty --report html --output '/Users/sanjaneya/Documents/toolkit-ios/reports/ui-tests/report-2.html' --report junit --output '/Users/sanjaneya/Documents/toolkit-ios/reports/ui-tests/report-2.junit' --report junit --output '/var/folders/nc/x2sh_3t17692n9jf3pybxt2n8c1sfm/T/junit_report20181107-2669-9oqogd'
INFO [2018-11-07 14:11:09.41]: ▸ Loading...
INFO [2018-11-07 14:11:11.44]: ▸ 2018-11-07 14:11:11.443 xcodebuild[9789:3373931] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
INFO [2018-11-07 14:11:11.56]: ▸ No devices are booted.
INFO [2018-11-07 14:11:11.57]: ▸ 2018-11-07 14:11:11.578 xcodebuild[9782:3373900] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
INFO [2018-11-07 14:11:11.57]: ▸ /Users/sanjaneya/Library/Developer/Xcode/DerivedData/Toolkit-cbjnzqipycngqehjhegjjguoghlk/Logs/Test/Test-ToolkitUITests-2018.11.07_14-11-11--0500.xcresult/1_Test/Diagnostics/ToolkitUITests-215A022E-EF31-4B85-AEBD-9397361F2F39/ToolkitUITests-4439C977-55DE-43F9-945B-DB90C43AEB70/Session-ToolkitUITests-2018-11-07_141111-dIpxEc.log
INFO [2018-11-07 14:11:11.57]: ▸ 2018-11-07 14:11:11.578 xcodebuild[9782:3373861] [MT] IDETestOperationsObserverDebug: (5D929A4A-857A-450E-B447-664E34FC24AE) Beginning test session ToolkitUITests-5D929A4A-857A-450E-B447-664E34FC24AE at 2018-11-07 14:11:11.578 with Xcode 10A255 on target {
INFO [2018-11-07 14:11:11.57]: ▸ SimDevice: iPhone 8 (65B5C280-4F76-43A8-BE58-D9CC4EE7AAAA, iOS 12.0, Shutdown)
INFO [2018-11-07 14:11:11.57]: ▸ } (12.0 (16A366))
INFO [2018-11-07 14:11:29.85]: ▸ Selected tests
INFO [2018-11-07 14:11:29.85]: ▸ Test Suite ToolkitUITests.xctest started
INFO [2018-11-07 14:11:29.85]: ▸ EditDraftTests
INFO [2018-11-07 14:13:11.45]: ▸ ✓ testEditDraftsAddImageCamera (101.599 seconds)
INFO [2018-11-07 14:14:45.03]: ▸ ✓ testEditDraftsAddImageLibrary (93.577 seconds)
INFO [2018-11-07 14:16:01.43]: ▸ ✗ testEditDraftsAddImageLink, Failed to find "All" Button after 10.0 seconds
INFO [2018-11-07 14:16:01.45]: ▸ ✗ testEditDraftsAddImageLink, Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "All" Button".
INFO [2018-11-07 14:16:07.63]: ▸ ToolkitUITests.EditDraftTests
INFO [2018-11-07 14:16:07.63]: ▸ testEditDraftsAddImageLink, Failed to find "All" Button after 10.0 seconds
INFO [2018-11-07 14:16:07.63]: ▸ EditDraftPage.swift:196
INFO [2018-11-07 14:16:07.63]: ▸ testEditDraftsAddImageLink, Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "All" Button".
INFO [2018-11-07 14:16:07.63]: ▸ HelperExtensions.swift:32
INFO [2018-11-07 14:16:07.63]: ▸ Executed 3 tests, with 2 failures (0 unexpected) in 277.767 (277.773) seconds
INFO [2018-11-07 14:16:07.63]: ▸
INFO [2018-11-07 14:16:21.60]: ▸ 2018-11-07 14:16:21.600 xcodebuild[9782:3373861] [MT] IDETestOperationsObserverDebug: 310.032 elapsed -- Testing started completed.
INFO [2018-11-07 14:16:21.60]: ▸ 2018-11-07 14:16:21.600 xcodebuild[9782:3373861] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
INFO [2018-11-07 14:16:21.60]: ▸ 2018-11-07 14:16:21.600 xcodebuild[9782:3373861] [MT] IDETestOperationsObserverDebug: 310.032 sec, +310.032 sec -- end
INFO [2018-11-07 14:16:21.89]: ▸ 2018-11-07 14:16:21.893 xcodebuild[10098:3383304] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
INFO [2018-11-07 14:16:25.20]: ▸ Failing tests:
INFO [2018-11-07 14:16:25.20]: ▸ EditDraftTests.testEditDraftsAddImageLink()
INFO [2018-11-07 14:16:25.20]: ▸ EditDraftTests.testEditDraftsAddImageLink()
INFO [2018-11-07 14:16:25.20]: ▸ * TEST EXECUTE FAILED *
2018-11-07 14:11:11.443 xcodebuild[9789:3373931] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
No devices are booted.
2018-11-07 14:11:11.578 xcodebuild[9782:3373900] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
/Users/sanjaneya/Library/Developer/Xcode/DerivedData/Toolkit-cbjnzqipycngqehjhegjjguoghlk/Logs/Test/Test-ToolkitUITests-2018.11.07_14-11-11--0500.xcresult/1_Test/Diagnostics/ToolkitUITests-215A022E-EF31-4B85-AEBD-9397361F2F39/ToolkitUITests-4439C977-55DE-43F9-945B-DB90C43AEB70/Session-ToolkitUITests-2018-11-07_141111-dIpxEc.log
2018-11-07 14:11:11.578 xcodebuild[9782:3373861] [MT] IDETestOperationsObserverDebug: (5D929A4A-857A-450E-B447-664E34FC24AE) Beginning test session ToolkitUITests-5D929A4A-857A-450E-B447-664E34FC24AE at 2018-11-07 14:11:11.578 with Xcode 10A255 on target {
SimDevice: iPhone 8 (65B5C280-4F76-43A8-BE58-D9CC4EE7AAAA, iOS 12.0, Shutdown)
} (12.0 (16A366))
Selected tests
Test Suite ToolkitUITests.xctest started
EditDraftTests
✓ testEditDraftsAddImageCamera (101.599 seconds)
✓ testEditDraftsAddImageLibrary (93.577 seconds)
✗ testEditDraftsAddImageLink, Failed to find "All" Button after 10.0 seconds
✗ testEditDraftsAddImageLink, Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "All" Button".

ToolkitUITests.EditDraftTests
testEditDraftsAddImageLink, Failed to find "All" Button after 10.0 seconds
EditDraftPage.swift:196

 Executed 3 tests, with 2 failures (0 unexpected) in 277.767 (277.773) seconds

2018-11-07 14:16:21.600 xcodebuild[9782:3373861] [MT] IDETestOperationsObserverDebug: 310.032 elapsed -- Testing started completed.
2018-11-07 14:16:21.600 xcodebuild[9782:3373861] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
2018-11-07 14:16:21.600 xcodebuild[9782:3373861] [MT] IDETestOperationsObserverDebug: 310.032 sec, +310.032 sec -- end
2018-11-07 14:16:21.893 xcodebuild[10098:3383304] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
Failing tests:
EditDraftTests.testEditDraftsAddImageLink()
EditDraftTests.testEditDraftsAddImageLink()
* TEST EXECUTE FAILED *
ERROR [2018-11-07 14:16:25.23]: Exit status: 65
+--------------------+---+
| Test Results |
+--------------------+---+
| Number of tests | 4 |
| Number of failures | 2 |
+--------------------+---+

DEBUG [2018-11-07 14:16:25.63]: Scan failed with Tests have failed
INFO [2018-11-07 14:16:25.63]: ------------------------------------------------------
INFO [2018-11-07 14:16:25.63]: --- Starting test run on testable 'ToolkitUITests' ---
INFO [2018-11-07 14:16:25.63]: ------------------------------------------------------
INFO [2018-11-07 14:16:29.85]: $ xcodebuild -showBuildSettings -workspace /Users/sanjaneya/Documents/toolkit-ios/Toolkit.xcworkspace -scheme ToolkitUITests
WARN [2018-11-07 14:16:32.86]: Command timed out after 3 seconds on try 1 of 4, trying again with a 6 second timeout...
DEBUG [2018-11-07 14:16:36.31]: Fetching available simulator devices
INFO [2018-11-07 14:16:36.99]: $ xcodebuild -showBuildSettings -workspace /Users/sanjaneya/Documents/toolkit-ios/Toolkit.xcworkspace -scheme ToolkitUITests
WARN [2018-11-07 14:16:40.00]: Command timed out after 3 seconds on try 1 of 4, trying again with a 6 second timeout...
DEBUG [2018-11-07 14:16:43.25]: Fetching available simulator devices

INFO [2018-11-07 14:16:43.79]: $ set -o pipefail && env NSUnbufferedIO=YES xcodebuild -workspace /Users/sanjaneya/Documents/toolkit-ios/Toolkit.xcworkspace -scheme ToolkitUITests -destination 'platform=iOS Simulator,id=65B5C280-4F76-43A8-BE58-D9CC4EE7AAAA' -derivedDataPath '/Users/sanjaneya/Library/Developer/Xcode/DerivedData/Toolkit-cbjnzqipycngqehjhegjjguoghlk' -enableCodeCoverage YES -only-testing:ToolkitUITests/LoginPageTests/testEmptyUserName -only-testing:ToolkitUITests/LoginPageTests/testForgotPassword -only-testing:ToolkitUITests/LoginPageTests/testOnlySpacePassword -only-testing:ToolkitUITests/LoginPageTests/testSpaceCharInPassword -only-testing:ToolkitUITests/LoginPageTests/testForSwitchBetweenUsers -only-testing:ToolkitUITests/LoginPageTests/testTrailingSpaceUserNameSuccess -only-testing:ToolkitUITests/AddContactTests/testVerifyAddingValidContact -only-testing:ToolkitUITests/AddContactTests/testVerifyErrorForNoEmailAddress -only-testing:ToolkitUITests/AddContactTests/testVerifyErrorForInvalidEmailAddress -only-testing:ToolkitUITests/AddContactTests/testVerifyErrorForNoEmailListSelection -only-testing:ToolkitUITests/AddContactTests/testVerifyErrorForSavingWithoutAnyEntries -only-testing:ToolkitUITests/LibraryPageTest/testSortLibraryImagesByName -only-testing:ToolkitUITests/LibraryPageTest/testSortLibraryImagesByMostRecent -only-testing:ToolkitUITests/LibraryPageTest/testEmptyFolder -only-testing:ToolkitUITests/LibraryPageTest/testBrowseAllFolder -only-testing:ToolkitUITests/LibraryPageTest/testBrowseAllImages -only-testing:ToolkitUITests/LibraryPageTest/testSwitchToListView -only-testing:ToolkitUITests/LibraryPageTest/testLastSelectedFolder -only-testing:ToolkitUITests/EditContactTests/testDeleteContact -only-testing:ToolkitUITests/EditContactTests/testEditExistingEmailAddress -only-testing:ToolkitUITests/EditContactTests/testEditExistingContact -only-testing:ToolkitUITests/SentDashboardTests/testViewSentDetailsByTappingBarChart -only-testing:ToolkitUITests/SentDashboardTests/testViewSentDetailsByTappingSentOpensClicks -only-testing:ToolkitUITests/SentDashboardTests/testTapCampaignName -only-testing:ToolkitUITests/SentDashboardTests/testTappingLeftRightArrows -only-testing:ToolkitUITests/ActiveContactsTests/testSortingContacts -only-testing:ToolkitUITests/ActiveContactsTests/testSortByDateAdded -only-testing:ToolkitUITests/ActiveContactsTests/testSortByFirstName test-without-building | tee '/Users/sanjaneya/Library/Logs/scan/Toolkit.Dev-ToolkitUITests.log' | xcpretty --report html --output '/Users/sanjaneya/Documents/toolkit-ios/reports/ui-tests/report-3.html' --report junit --output '/Users/sanjaneya/Documents/toolkit-ios/reports/ui-tests/report-3.junit' --report junit --output '/var/folders/nc/x2sh_3t17692n9jf3pybxt2n8c1sfm/T/junit_report20181107-2669-m9nkxc'
INFO [2018-11-07 14:16:43.79]: ▸ Loading...
INFO [2018-11-07 14:16:45.96]: ▸ 2018-11-07 14:16:45.965 xcodebuild[10224:3385639] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
INFO [2018-11-07 14:16:46.10]: ▸ No devices are booted.
INFO [2018-11-07 14:16:46.11]: ▸ 2018-11-07 14:16:46.114 xcodebuild[10215:3385571] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
INFO [2018-11-07 14:16:46.11]: ▸ /Users/sanjaneya/Library/Developer/Xcode/DerivedData/Toolkit-cbjnzqipycngqehjhegjjguoghlk/Logs/Test/Test-ToolkitUITests-2018.11.07_14-16-45--0500.xcresult/1_Test/Diagnostics/ToolkitUITests-C92D2117-5977-4CE4-A9CF-A6DB3B03CC27/ToolkitUITests-FF52330D-0463-4079-A42C-DAEABAFF779C/Session-ToolkitUITests-2018-11-07_141646-TsuWXZ.log
INFO [2018-11-07 14:16:46.11]: ▸ 2018-11-07 14:16:46.115 xcodebuild[10215:3385497] [MT] IDETestOperationsObserverDebug: (E496A2EE-0377-4394-AA91-21BBDD14E38D) Beginning test session ToolkitUITests-E496A2EE-0377-4394-AA91-21BBDD14E38D at 2018-11-07 14:16:46.115 with Xcode 10A255 on target {
INFO [2018-11-07 14:16:46.11]: ▸ SimDevice: iPhone 8 (65B5C280-4F76-43A8-BE58-D9CC4EE7AAAA, iOS 12.0, Shutdown)
INFO [2018-11-07 14:16:46.11]: ▸ } (12.0 (16A366))
INFO [2018-11-07 14:17:00.20]: ▸ Selected tests
INFO [2018-11-07 14:17:00.20]: ▸ Test Suite ToolkitUITests.xctest started
INFO [2018-11-07 14:17:00.21]: ▸ ActiveContactsTests
INFO [2018-11-07 14:17:46.63]: ▸ ✓ testSortByDateAdded (46.429 seconds)
INFO [2018-11-07 14:18:29.97]: ▸ ✓ testSortByFirstName (43.332 seconds)
INFO [2018-11-07 14:19:24.01]: ▸ ✓ testSortingContacts (54.046 seconds)
INFO [2018-11-07 14:19:24.06]: ▸ AddContactTests
INFO [2018-11-07 14:20:19.08]: ▸ ✓ testVerifyAddingValidContact (55.066 seconds)
INFO [2018-11-07 14:21:13.32]: ▸ ✓ testVerifyErrorForInvalidEmailAddress (54.237 seconds)
INFO [2018-11-07 14:22:09.76]: ▸ ✓ testVerifyErrorForNoEmailAddress (56.440 seconds)
INFO [2018-11-07 14:23:04.86]: ▸ ✓ testVerifyErrorForNoEmailListSelection (55.105 seconds)
INFO [2018-11-07 14:24:00.36]: ▸ ✓ testVerifyErrorForSavingWithoutAnyEntries (55.498 seconds)
INFO [2018-11-07 14:24:00.40]: ▸ EditContactTests
INFO [2018-11-07 14:25:15.90]: ▸ ✓ testDeleteContact (75.536 seconds)
INFO [2018-11-07 14:26:16.83]: ▸ ✓ testEditExistingContact (60.925 seconds)
INFO [2018-11-07 14:27:12.85]: ▸ ✓ testEditExistingEmailAddress (56.028 seconds)
INFO [2018-11-07 14:27:12.89]: ▸ LibraryPageTest
INFO [2018-11-07 14:29:19.18]: ▸ ✓ testBrowseAllFolder (126.324 seconds)
INFO [2018-11-07 14:31:20.16]: ▸ ✓ testBrowseAllImages (120.977 seconds)
INFO [2018-11-07 14:31:55.16]: ▸ ✓ testEmptyFolder (35.005 seconds)
INFO [2018-11-07 14:32:33.93]: ▸ ✓ testLastSelectedFolder (38.763 seconds)
INFO [2018-11-07 14:33:07.97]: ▸ ✓ testSortLibraryImagesByMostRecent (34.048 seconds)
INFO [2018-11-07 14:33:42.35]: ▸ ✓ testSortLibraryImagesByName (34.375 seconds)
INFO [2018-11-07 14:42:15.98]: ▸ ✓ testSwitchToListView (513.625 seconds)
INFO [2018-11-07 14:42:16.21]: ▸ LoginPageTests
INFO [2018-11-07 14:42:38.25]: ▸ ✓ testEmptyUserName (22.273 seconds)
INFO [2018-11-07 14:42:54.70]: ▸ ✓ testForgotPassword (16.452 seconds)
INFO [2018-11-07 14:44:01.96]: ▸ ✓ testForSwitchBetweenUsers (67.258 seconds)
INFO [2018-11-07 14:44:28.23]: ▸ ✓ testOnlySpacePassword (26.265 seconds)
INFO [2018-11-07 14:44:52.85]: ▸ ✓ testSpaceCharInPassword (24.624 seconds)
INFO [2018-11-07 14:45:28.70]: ▸ ✗ testTrailingSpaceUserNameSuccess, Failed to find "sentDashboard" Button after 15.0 seconds
INFO [2018-11-07 14:45:28.73]: ▸ ✗ testTrailingSpaceUserNameSuccess, Asynchronous wait failed: Exceeded timeout of 15 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "sentDashboard" Button".
INFO [2018-11-07 14:45:31.39]: ▸ SentDashboardTests
INFO [2018-11-07 14:46:09.09]: ▸ ✓ testTapCampaignName (37.699 seconds)
INFO [2018-11-07 14:46:46.56]: ▸ ✓ testTappingLeftRightArrows (37.475 seconds)
INFO [2018-11-07 14:47:28.05]: ▸ ✓ testViewSentDetailsByTappingBarChart (41.486 seconds)
INFO [2018-11-07 14:48:15.16]: ▸ ✓ testViewSentDetailsByTappingSentOpensClicks (47.118 seconds)
INFO [2018-11-07 14:48:15.20]: ▸ ToolkitUITests.LoginPageTests
INFO [2018-11-07 14:48:15.20]: ▸ testTrailingSpaceUserNameSuccess, Failed to find "sentDashboard" Button after 15.0 seconds
INFO [2018-11-07 14:48:15.20]: ▸ CampaignsDashboardPage.swift:21
INFO [2018-11-07 14:48:15.20]: ▸ testTrailingSpaceUserNameSuccess, Asynchronous wait failed: Exceeded timeout of 15 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "sentDashboard" Button".
INFO [2018-11-07 14:48:15.20]: ▸ HelperExtensions.swift:32
INFO [2018-11-07 14:48:15.20]: ▸ Executed 28 tests, with 2 failures (0 unexpected) in 1874.940 (1874.961) seconds
INFO [2018-11-07 14:48:15.20]: ▸
INFO [2018-11-07 14:48:40.71]: ▸ 2018-11-07 14:48:40.709 xcodebuild[10215:3385497] [MT] IDETestOperationsObserverDebug: 1914.605 elapsed -- Testing started completed.
INFO [2018-11-07 14:48:40.71]: ▸ 2018-11-07 14:48:40.709 xcodebuild[10215:3385497] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
INFO [2018-11-07 14:48:40.71]: ▸ 2018-11-07 14:48:40.709 xcodebuild[10215:3385497] [MT] IDETestOperationsObserverDebug: 1914.605 sec, +1914.605 sec -- end
INFO [2018-11-07 14:48:41.20]: ▸ 2018-11-07 14:48:41.208 xcodebuild[11403:3508409] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
INFO [2018-11-07 14:48:42.86]: ▸ An error was encountered processing the command (domain=NSMachErrorDomain, code=-308):
INFO [2018-11-07 14:48:42.93]: ▸ The operation couldn’t be completed. (Mach error -308 - (ipc/mig) server died)
INFO [2018-11-07 14:48:46.57]: ▸ Failing tests:
INFO [2018-11-07 14:48:46.57]: ▸ LoginPageTests.testTrailingSpaceUserNameSuccess()
INFO [2018-11-07 14:48:46.57]: ▸ LoginPageTests.testTrailingSpaceUserNameSuccess()
INFO [2018-11-07 14:48:46.57]: ▸ * TEST EXECUTE FAILED *
2018-11-07 14:16:45.965 xcodebuild[10224:3385639] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
No devices are booted.
2018-11-07 14:16:46.114 xcodebuild[10215:3385571] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
/Users/sanjaneya/Library/Developer/Xcode/DerivedData/Toolkit-cbjnzqipycngqehjhegjjguoghlk/Logs/Test/Test-ToolkitUITests-2018.11.07_14-16-45--0500.xcresult/1_Test/Diagnostics/ToolkitUITests-C92D2117-5977-4CE4-A9CF-A6DB3B03CC27/ToolkitUITests-FF52330D-0463-4079-A42C-DAEABAFF779C/Session-ToolkitUITests-2018-11-07_141646-TsuWXZ.log
2018-11-07 14:16:46.115 xcodebuild[10215:3385497] [MT] IDETestOperationsObserverDebug: (E496A2EE-0377-4394-AA91-21BBDD14E38D) Beginning test session ToolkitUITests-E496A2EE-0377-4394-AA91-21BBDD14E38D at 2018-11-07 14:16:46.115 with Xcode 10A255 on target {
SimDevice: iPhone 8 (65B5C280-4F76-43A8-BE58-D9CC4EE7AAAA, iOS 12.0, Shutdown)
} (12.0 (16A366))
Selected tests
Test Suite ToolkitUITests.xctest started
ActiveContactsTests
✓ testSortByDateAdded (46.429 seconds)
✓ testSortByFirstName (43.332 seconds)
✓ testSortingContacts (54.046 seconds)
AddContactTests
✓ testVerifyAddingValidContact (55.066 seconds)
✓ testVerifyErrorForInvalidEmailAddress (54.237 seconds)
✓ testVerifyErrorForNoEmailAddress (56.440 seconds)
✓ testVerifyErrorForNoEmailListSelection (55.105 seconds)
✓ testVerifyErrorForSavingWithoutAnyEntries (55.498 seconds)
EditContactTests
✓ testDeleteContact (75.536 seconds)
✓ testEditExistingContact (60.925 seconds)
✓ testEditExistingEmailAddress (56.028 seconds)
LibraryPageTest
✓ testBrowseAllFolder (126.324 seconds)
✓ testBrowseAllImages (120.977 seconds)
✓ testEmptyFolder (35.005 seconds)
✓ testLastSelectedFolder (38.763 seconds)
✓ testSortLibraryImagesByMostRecent (34.048 seconds)
✓ testSortLibraryImagesByName (34.375 seconds)
✓ testSwitchToListView (513.625 seconds)
LoginPageTests
✓ testEmptyUserName (22.273 seconds)
✓ testForgotPassword (16.452 seconds)
✓ testForSwitchBetweenUsers (67.258 seconds)
✓ testOnlySpacePassword (26.265 seconds)
✓ testSpaceCharInPassword (24.624 seconds)
✗ testTrailingSpaceUserNameSuccess, Failed to find "sentDashboard" Button after 15.0 seconds
✗ testTrailingSpaceUserNameSuccess, Asynchronous wait failed: Exceeded timeout of 15 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "sentDashboard" Button".
SentDashboardTests
✓ testTapCampaignName (37.699 seconds)
✓ testTappingLeftRightArrows (37.475 seconds)
✓ testViewSentDetailsByTappingBarChart (41.486 seconds)
✓ testViewSentDetailsByTappingSentOpensClicks (47.118 seconds)

ToolkitUITests.LoginPageTests
testTrailingSpaceUserNameSuccess, Failed to find "sentDashboard" Button after 15.0 seconds
CampaignsDashboardPage.swift:21

testTrailingSpaceUserNameSuccess, Asynchronous wait failed: Exceeded timeout of 15 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "sentDashboard" Button".
HelperExtensions.swift:32

 Executed 28 tests, with 2 failures (0 unexpected) in 1874.940 (1874.961) seconds

2018-11-07 14:48:40.709 xcodebuild[10215:3385497] [MT] IDETestOperationsObserverDebug: 1914.605 elapsed -- Testing started completed.
2018-11-07 14:48:40.709 xcodebuild[10215:3385497] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
2018-11-07 14:48:40.709 xcodebuild[10215:3385497] [MT] IDETestOperationsObserverDebug: 1914.605 sec, +1914.605 sec -- end
2018-11-07 14:48:41.208 xcodebuild[11403:3508409] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
An error was encountered processing the command (domain=NSMachErrorDomain, code=-308):
The operation couldn’t be completed. (Mach error -308 - (ipc/mig) server died)
Failing tests:
LoginPageTests.testTrailingSpaceUserNameSuccess()
LoginPageTests.testTrailingSpaceUserNameSuccess()
* TEST EXECUTE FAILED *
ERROR [2018-11-07 14:48:46.65]: Exit status: 65
+--------------------+----+
| Test Results |
+--------------------+----+
| Number of tests | 29 |
| Number of failures | 2 |
+--------------------+----+

DEBUG [2018-11-07 14:48:47.06]: Scan failed with Tests have failed
INFO [2018-11-07 14:48:47.09]: Re-running scan on only failed tests
INFO [2018-11-07 14:48:49.27]: $ xcodebuild -showBuildSettings -workspace /Users/sanjaneya/Documents/toolkit-ios/Toolkit.xcworkspace -scheme ToolkitUITests
WARN [2018-11-07 14:48:52.28]: Command timed out after 3 seconds on try 1 of 4, trying again with a 6 second timeout...
DEBUG [2018-11-07 14:48:56.30]: Fetching available simulator devices
INFO [2018-11-07 14:48:57.10]: $ xcodebuild -showBuildSettings -workspace /Users/sanjaneya/Documents/toolkit-ios/Toolkit.xcworkspace -scheme ToolkitUITests
WARN [2018-11-07 14:49:00.11]: Command timed out after 3 seconds on try 1 of 4, trying again with a 6 second timeout...
DEBUG [2018-11-07 14:49:03.61]: Fetching available simulator devices

INFO [2018-11-07 14:49:04.16]: $ set -o pipefail && env NSUnbufferedIO=YES xcodebuild -workspace /Users/sanjaneya/Documents/toolkit-ios/Toolkit.xcworkspace -scheme ToolkitUITests -destination 'platform=iOS Simulator,id=65B5C280-4F76-43A8-BE58-D9CC4EE7AAAA' -derivedDataPath '/Users/sanjaneya/Library/Developer/Xcode/DerivedData/Toolkit-cbjnzqipycngqehjhegjjguoghlk' -only-testing:ToolkitUITests/LoginPageTests/testTrailingSpaceUserNameSuccess -only-testing:ToolkitUITests/LoginPageTests/testTrailingSpaceUserNameSuccess test-without-building | tee '/Users/sanjaneya/Library/Logs/scan/Toolkit.Dev-ToolkitUITests.log' | xcpretty --report html --output '/Users/sanjaneya/Documents/toolkit-ios/reports/ui-tests/report-4.html' --report junit --output '/Users/sanjaneya/Documents/toolkit-ios/reports/ui-tests/report-4.junit' --report junit --output '/var/folders/nc/x2sh_3t17692n9jf3pybxt2n8c1sfm/T/junit_report20181107-2669-1ma55n5'
INFO [2018-11-07 14:49:04.16]: ▸ Loading...
INFO [2018-11-07 14:49:06.47]: ▸ 2018-11-07 14:49:06.476 xcodebuild[11499:3511025] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
INFO [2018-11-07 14:49:06.62]: ▸ No devices are booted.
INFO [2018-11-07 14:49:06.63]: ▸ 2018-11-07 14:49:06.634 xcodebuild[11492:3510912] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
INFO [2018-11-07 14:49:06.63]: ▸ /Users/sanjaneya/Library/Developer/Xcode/DerivedData/Toolkit-cbjnzqipycngqehjhegjjguoghlk/Logs/Test/Test-ToolkitUITests-2018.11.07_14-49-06--0500.xcresult/1_Test/Diagnostics/ToolkitUITests-591C2C78-CF44-4DBF-8B64-9E7E5D40BFC5/ToolkitUITests-D773E584-23D3-47A5-8C2A-6826250CC5A1/Session-ToolkitUITests-2018-11-07_144906-v0fUFA.log
INFO [2018-11-07 14:49:06.63]: ▸ 2018-11-07 14:49:06.634 xcodebuild[11492:3510779] [MT] IDETestOperationsObserverDebug: (1B85DD0C-D2AB-4579-9632-C0B62B5797A8) Beginning test session ToolkitUITests-1B85DD0C-D2AB-4579-9632-C0B62B5797A8 at 2018-11-07 14:49:06.635 with Xcode 10A255 on target {
INFO [2018-11-07 14:49:06.63]: ▸ SimDevice: iPhone 8 (65B5C280-4F76-43A8-BE58-D9CC4EE7AAAA, iOS 12.0, Shutdown)
INFO [2018-11-07 14:49:06.65]: ▸ } (12.0 (16A366))
INFO [2018-11-07 14:49:27.28]: ▸ Selected tests
INFO [2018-11-07 14:49:27.28]: ▸ Test Suite ToolkitUITests.xctest started
INFO [2018-11-07 14:49:27.28]: ▸ LoginPageTests
INFO [2018-11-07 14:50:06.46]: ▸ ✗ testTrailingSpaceUserNameSuccess, Failed to find "sentDashboard" Button after 15.0 seconds
INFO [2018-11-07 14:50:06.48]: ▸ ✗ testTrailingSpaceUserNameSuccess, Asynchronous wait failed: Exceeded timeout of 15 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "sentDashboard" Button".
INFO [2018-11-07 14:50:13.84]: ▸ ToolkitUITests.LoginPageTests
INFO [2018-11-07 14:50:13.84]: ▸ testTrailingSpaceUserNameSuccess, Failed to find "sentDashboard" Button after 15.0 seconds
INFO [2018-11-07 14:50:13.84]: ▸ CampaignsDashboardPage.swift:21
INFO [2018-11-07 14:50:13.84]: ▸ testTrailingSpaceUserNameSuccess, Asynchronous wait failed: Exceeded timeout of 15 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "sentDashboard" Button".
INFO [2018-11-07 14:50:13.84]: ▸ HelperExtensions.swift:32
INFO [2018-11-07 14:50:13.84]: ▸ Executed 1 test, with 2 failures (0 unexpected) in 46.560 (46.563) seconds
INFO [2018-11-07 14:50:13.84]: ▸
INFO [2018-11-07 14:50:32.90]: ▸ 2018-11-07 14:50:32.901 xcodebuild[11492:3510779] [MT] IDETestOperationsObserverDebug: 86.278 elapsed -- Testing started completed.
INFO [2018-11-07 14:50:32.90]: ▸ 2018-11-07 14:50:32.901 xcodebuild[11492:3510779] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
INFO [2018-11-07 14:50:32.90]: ▸ 2018-11-07 14:50:32.901 xcodebuild[11492:3510779] [MT] IDETestOperationsObserverDebug: 86.278 sec, +86.278 sec -- end
INFO [2018-11-07 14:50:33.20]: ▸ 2018-11-07 14:50:33.203 xcodebuild[11658:3518771] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
INFO [2018-11-07 14:50:36.07]: ▸ Failing tests:
INFO [2018-11-07 14:50:36.07]: ▸ LoginPageTests.testTrailingSpaceUserNameSuccess()
INFO [2018-11-07 14:50:36.07]: ▸ LoginPageTests.testTrailingSpaceUserNameSuccess()
INFO [2018-11-07 14:50:36.07]: ▸ * TEST EXECUTE FAILED *
2018-11-07 14:49:06.476 xcodebuild[11499:3511025] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
No devices are booted.
2018-11-07 14:49:06.634 xcodebuild[11492:3510912] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
/Users/sanjaneya/Library/Developer/Xcode/DerivedData/Toolkit-cbjnzqipycngqehjhegjjguoghlk/Logs/Test/Test-ToolkitUITests-2018.11.07_14-49-06--0500.xcresult/1_Test/Diagnostics/ToolkitUITests-591C2C78-CF44-4DBF-8B64-9E7E5D40BFC5/ToolkitUITests-D773E584-23D3-47A5-8C2A-6826250CC5A1/Session-ToolkitUITests-2018-11-07_144906-v0fUFA.log
2018-11-07 14:49:06.634 xcodebuild[11492:3510779] [MT] IDETestOperationsObserverDebug: (1B85DD0C-D2AB-4579-9632-C0B62B5797A8) Beginning test session ToolkitUITests-1B85DD0C-D2AB-4579-9632-C0B62B5797A8 at 2018-11-07 14:49:06.635 with Xcode 10A255 on target {
SimDevice: iPhone 8 (65B5C280-4F76-43A8-BE58-D9CC4EE7AAAA, iOS 12.0, Shutdown)
} (12.0 (16A366))
Selected tests
Test Suite ToolkitUITests.xctest started
LoginPageTests
✗ testTrailingSpaceUserNameSuccess, Failed to find "sentDashboard" Button after 15.0 seconds
✗ testTrailingSpaceUserNameSuccess, Asynchronous wait failed: Exceeded timeout of 15 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "sentDashboard" Button".

ToolkitUITests.LoginPageTests
testTrailingSpaceUserNameSuccess, Failed to find "sentDashboard" Button after 15.0 seconds
CampaignsDashboardPage.swift:21

 Executed 1 test, with 2 failures (0 unexpected) in 46.560 (46.563) seconds

2018-11-07 14:50:32.901 xcodebuild[11492:3510779] [MT] IDETestOperationsObserverDebug: 86.278 elapsed -- Testing started completed.
2018-11-07 14:50:32.901 xcodebuild[11492:3510779] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
2018-11-07 14:50:32.901 xcodebuild[11492:3510779] [MT] IDETestOperationsObserverDebug: 86.278 sec, +86.278 sec -- end
2018-11-07 14:50:33.203 xcodebuild[11658:3518771] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
Failing tests:
LoginPageTests.testTrailingSpaceUserNameSuccess()
LoginPageTests.testTrailingSpaceUserNameSuccess()
* TEST EXECUTE FAILED *
ERROR [2018-11-07 14:50:36.14]: Exit status: 65
+--------------------+---+
| Test Results |
+--------------------+---+
| Number of tests | 2 |
| Number of failures | 2 |
+--------------------+---+

DEBUG [2018-11-07 14:50:36.55]: Scan failed with Tests have failed
INFO [2018-11-07 14:50:36.55]: ------------------------------------------------------
INFO [2018-11-07 14:50:36.55]: --- Starting test run on testable 'ToolkitUITests' ---
INFO [2018-11-07 14:50:36.55]: ------------------------------------------------------
INFO [2018-11-07 14:50:41.82]: $ xcodebuild -showBuildSettings -workspace /Users/sanjaneya/Documents/toolkit-ios/Toolkit.xcworkspace -scheme ToolkitUITests
WARN [2018-11-07 14:50:44.83]: Command timed out after 3 seconds on try 1 of 4, trying again with a 6 second timeout...
DEBUG [2018-11-07 14:50:48.54]: Fetching available simulator devices
INFO [2018-11-07 14:50:49.28]: $ xcodebuild -showBuildSettings -workspace /Users/sanjaneya/Documents/toolkit-ios/Toolkit.xcworkspace -scheme ToolkitUITests
WARN [2018-11-07 14:50:52.29]: Command timed out after 3 seconds on try 1 of 4, trying again with a 6 second timeout...
DEBUG [2018-11-07 14:50:56.82]: Fetching available simulator devices

INFO [2018-11-07 14:50:57.50]: $ set -o pipefail && env NSUnbufferedIO=YES xcodebuild -workspace /Users/sanjaneya/Documents/toolkit-ios/Toolkit.xcworkspace -scheme ToolkitUITests -destination 'platform=iOS Simulator,id=65B5C280-4F76-43A8-BE58-D9CC4EE7AAAA' -derivedDataPath '/Users/sanjaneya/Library/Developer/Xcode/DerivedData/Toolkit-cbjnzqipycngqehjhegjjguoghlk' -enableCodeCoverage YES -only-testing:ToolkitUITests/ActiveContactsTests/testSortByCompanyName -only-testing:ToolkitUITests/ActiveContactsTests/testSortByEmailAddress -only-testing:ToolkitUITests/ActiveContactsTests/testViewContactDetails -only-testing:ToolkitUITests/ContactDetailsTests/testViewEmailListsDetails -only-testing:ToolkitUITests/ContactDetailsTests/testEditContactFromContactDetailsPage -only-testing:ToolkitUITests/ContactDetailsTests/testViewContactDetails -only-testing:ToolkitUITests/ContactDetailsTests/testVerifyModifyingEmailLists -only-testing:ToolkitUITests/ContactDetailsTests/testVerifyAtleastOneEmailListError -only-testing:ToolkitUITests/DraftDashboardTests/testViewDraftCampaignDetails -only-testing:ToolkitUITests/DraftDashboardTests/testSendTestCampaign -only-testing:ToolkitUITests/DraftDashboardTests/testRemoveCampaignDetails -only-testing:ToolkitUITests/SearchContactsTests/testVerifyZeroSearchResults -only-testing:ToolkitUITests/SearchContactsTests/testValidContactSearch -only-testing:ToolkitUITests/SearchContactsTests/testVerifySortInContactsSearch -only-testing:ToolkitUITests/SearchContactsTests/testViewContactDetailsInContactsSearch -only-testing:ToolkitUITests/SearchContactsTests/testTapSearch -only-testing:ToolkitUITests/CampaignMetricsTests/testResendCampaignFlow -only-testing:ToolkitUITests/ScheduleDashboardTests/testScheduleDashboard -only-testing:ToolkitUITests/ScheduledCampaignTests/testScheduleCampaignTitleTextBlocks -only-testing:ToolkitUITests/ScheduledCampaignTests/testScheduleCampaignButtonLinkImgBlocks -only-testing:ToolkitUITests/ScheduledCampaignTests/testUnschueduleCampaign -only-testing:ToolkitUITests/ScheduledCampaignTests/testScheduleDraftCampaign -only-testing:ToolkitUITests/ScheduledCampaignTests/testScheduleNoSendToListError -only-testing:ToolkitUITests/ContactsAcceptanceTests/testContactsUserScenario1 -only-testing:ToolkitUITests/ContactsAcceptanceTests/testContactsUserScenario2 -only-testing:ToolkitUITests/ContactsAcceptanceTests/testContactsUserScenario3 -only-testing:ToolkitUITests/ContactsAcceptanceTests/testContactsUserScenario4 -only-testing:ToolkitUITests/ContactsAcceptanceTests/testContactsUserScenario5 test-without-building | tee '/Users/sanjaneya/Library/Logs/scan/Toolkit.Dev-ToolkitUITests.log' | xcpretty --report html --output '/Users/sanjaneya/Documents/toolkit-ios/reports/ui-tests/report-5.html' --report junit --output '/Users/sanjaneya/Documents/toolkit-ios/reports/ui-tests/report-5.junit' --report junit --output '/var/folders/nc/x2sh_3t17692n9jf3pybxt2n8c1sfm/T/junit_report20181107-2669-vvfehi'
INFO [2018-11-07 14:50:57.50]: ▸ Loading...
INFO [2018-11-07 14:51:00.15]: ▸ 2018-11-07 14:51:00.151 xcodebuild[11776:3522577] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
INFO [2018-11-07 14:51:00.29]: ▸ No devices are booted.
INFO [2018-11-07 14:51:00.30]: ▸ 2018-11-07 14:51:00.304 xcodebuild[11766:3522395] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
INFO [2018-11-07 14:51:00.30]: ▸ /Users/sanjaneya/Library/Developer/Xcode/DerivedData/Toolkit-cbjnzqipycngqehjhegjjguoghlk/Logs/Test/Test-ToolkitUITests-2018.11.07_14-50-59--0500.xcresult/1_Test/Diagnostics/ToolkitUITests-E6B3589A-5D54-4CF5-8B06-72C1EAB84BCC/ToolkitUITests-18F04C1B-2B85-41BA-9E5C-E1276629845E/Session-ToolkitUITests-2018-11-07_145100-SovC2P.log
INFO [2018-11-07 14:51:00.30]: ▸ 2018-11-07 14:51:00.304 xcodebuild[11766:3522250] [MT] IDETestOperationsObserverDebug: (F63EBBCA-F15D-4C9F-92BE-FA0FC6C20D13) Beginning test session ToolkitUITests-F63EBBCA-F15D-4C9F-92BE-FA0FC6C20D13 at 2018-11-07 14:51:00.305 with Xcode 10A255 on target {
INFO [2018-11-07 14:51:00.30]: ▸ SimDevice: iPhone 8 (65B5C280-4F76-43A8-BE58-D9CC4EE7AAAA, iOS 12.0, Shutdown)
INFO [2018-11-07 14:51:00.30]: ▸ } (12.0 (16A366))
INFO [2018-11-07 14:51:19.26]: ▸ Selected tests
INFO [2018-11-07 14:51:19.26]: ▸ Test Suite ToolkitUITests.xctest started
INFO [2018-11-07 14:51:19.27]: ▸ ActiveContactsTests
INFO [2018-11-07 14:52:11.04]: ▸ ✓ testSortByCompanyName (51.769 seconds)
INFO [2018-11-07 14:52:57.62]: ▸ ✓ testSortByEmailAddress (46.582 seconds)
INFO [2018-11-07 14:53:43.82]: ▸ ✓ testViewContactDetails (46.199 seconds)
INFO [2018-11-07 14:53:43.85]: ▸ CampaignMetricsTests
INFO [2018-11-07 14:54:40.92]: ▸ ✓ testResendCampaignFlow (57.097 seconds)
INFO [2018-11-07 14:54:40.95]: ▸ ContactDetailsTests
INFO [2018-11-07 14:55:35.93]: ▸ ✓ testEditContactFromContactDetailsPage (55.012 seconds)
INFO [2018-11-07 14:56:29.64]: ▸ ✓ testVerifyAtleastOneEmailListError (53.709 seconds)
INFO [2018-11-07 14:57:33.18]: ▸ ✓ testVerifyModifyingEmailLists (63.538 seconds)
INFO [2018-11-07 14:58:19.72]: ▸ ✓ testViewContactDetails (46.542 seconds)
INFO [2018-11-07 14:59:06.24]: ▸ ✓ testViewEmailListsDetails (46.518 seconds)
INFO [2018-11-07 14:59:06.26]: ▸ ContactsAcceptanceTests
INFO [2018-11-07 15:00:27.19]: ▸ ✓ testContactsUserScenario1 (80.951 seconds)
INFO [2018-11-07 15:01:29.38]: ▸ ✓ testContactsUserScenario2 (62.195 seconds)
INFO [2018-11-07 15:02:32.28]: ▸ ✓ testContactsUserScenario3 (62.893 seconds)
INFO [2018-11-07 15:03:51.01]: ▸ ✓ testContactsUserScenario4 (78.732 seconds)
INFO [2018-11-07 15:05:05.86]: ▸ ✓ testContactsUserScenario5 (74.854 seconds)
INFO [2018-11-07 15:05:05.95]: ▸ DraftDashboardTests
INFO [2018-11-07 15:06:36.59]: ▸ ✓ testRemoveCampaignDetails (90.725 seconds)
INFO [2018-11-07 15:07:49.27]: ▸ ✓ testSendTestCampaign (72.677 seconds)
INFO [2018-11-07 15:08:56.34]: ▸ ✓ testViewDraftCampaignDetails (67.075 seconds)
INFO [2018-11-07 15:08:56.37]: ▸ ScheduledCampaignTests
INFO [2018-11-07 15:11:42.96]: ▸ ✓ testScheduleCampaignButtonLinkImgBlocks (166.613 seconds)
INFO [2018-11-07 15:14:06.51]: ▸ ✓ testScheduleCampaignTitleTextBlocks (143.554 seconds)
INFO [2018-11-07 15:16:03.03]: ▸ ✓ testScheduleDraftCampaign (116.521 seconds)
INFO [2018-11-07 15:17:58.04]: ▸ ✓ testScheduleNoSendToListError (115.000 seconds)
INFO [2018-11-07 15:20:35.74]: ▸ ✓ testUnschueduleCampaign (157.701 seconds)
INFO [2018-11-07 15:20:35.82]: ▸ SearchContactsTests
INFO [2018-11-07 15:21:24.81]: ▸ ✓ testTapSearch (49.068 seconds)
INFO [2018-11-07 15:22:16.12]: ▸ ✓ testValidContactSearch (51.317 seconds)
INFO [2018-11-07 15:23:20.02]: ▸ ✓ testVerifySortInContactsSearch (63.891 seconds)
INFO [2018-11-07 15:24:08.75]: ▸ ✓ testVerifyZeroSearchResults (48.735 seconds)
INFO [2018-11-07 15:25:02.15]: ▸ ✓ testViewContactDetailsInContactsSearch (53.399 seconds)
INFO [2018-11-07 15:25:02.18]: ▸ Executed 27 tests, with 0 failures (0 unexpected) in 2022.866 (2022.887) seconds
INFO [2018-11-07 15:25:02.18]: ▸
INFO [2018-11-07 15:25:02.49]: ▸ 2018-11-07 15:25:02.492 xcodebuild[11766:3522250] [MT] IDETestOperationsObserverDebug: 2042.199 elapsed -- Testing started completed.
INFO [2018-11-07 15:25:02.49]: ▸ 2018-11-07 15:25:02.492 xcodebuild[11766:3522250] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
INFO [2018-11-07 15:25:02.49]: ▸ 2018-11-07 15:25:02.492 xcodebuild[11766:3522250] [MT] IDETestOperationsObserverDebug: 2042.199 sec, +2042.199 sec -- end
INFO [2018-11-07 15:25:02.93]: ▸ 2018-11-07 15:25:02.935 xcodebuild[14113:3668423] [MT] DVTDeveloperPaths: WARNING: Environment variable XCODE_DEVELOPER_DIR_PATH is set, but is not an absolute path. The variable will be ignored. ()
INFO [2018-11-07 15:25:04.68]: ▸ An error was encountered processing the command (domain=NSMachErrorDomain, code=-308):
INFO [2018-11-07 15:25:04.69]: ▸ The operation couldn’t be completed. (Mach error -308 - (ipc/mig) server died)
INFO [2018-11-07 15:25:08.06]: ▸ Test execute Succeeded
+--------------------+----+
| Test Results |
+--------------------+----+
| Number of tests | 27 |
| Number of failures | 0 |
+--------------------+----+

WARN [2018-11-07 15:25:09.81]: Lane Context:
INFO [2018-11-07 15:25:09.81]: {:DEFAULT_PLATFORM=>:ios, :PLATFORM_NAME=>:ios, :LANE_NAME=>"ios toolkit_ui_tests", :SCAN_DERIVED_DATA_PATH=>"/Users/sanjaneya/Library/Developer/Xcode/DerivedData/Toolkit-cbjnzqipycngqehjhegjjguoghlk", :SCAN_GENERATED_PLIST_FILES=>["/Users/sanjaneya/Library/Developer/Xcode/DerivedData/Toolkit-cbjnzqipycngqehjhegjjguoghlk/Logs/Test/Test-ToolkitUITests-2018.11.07_14-50-59--0500.xcresult/TestSummaries.plist"], :SCAN_GENERATED_PLIST_FILE=>"/Users/sanjaneya/Library/Developer/Xcode/DerivedData/Toolkit-cbjnzqipycngqehjhegjjguoghlk/Logs/Test/Test-ToolkitUITests-2018.11.07_14-50-59--0500.xcresult/TestSummaries.plist"}
ERROR [2018-11-07 15:25:09.81]: Tests have failed
INFO [2018-11-07 15:25:09.82]: Successfully generated documentation at path '/Users/sanjaneya/Documents/toolkit-ios/fastlane/README.md'

+------+-------------------------------------------+-------------+
| fastlane summary |
+------+-------------------------------------------+-------------+
| Step | Action | Time (in s) |
+------+-------------------------------------------+-------------+
| 1 | Verifying fastlane version | 0 |
| 2 | default_platform | 0 |
| 3 | clear_derived_data | 0 |
| 4 | mkdir -p ../reports/uitest-coverage/ | 0 |
| 5 | mkdir -p ../reports/uitest-xcov-coverage/ | 0 |
| 6 | cocoapods | 12 |
| 💥 | multi_scan | 6089 |
+------+-------------------------------------------+-------------+

+-------------+--------------+----------------+
| Plugin updates available |
+-------------+--------------+----------------+
| Plugin | Your Version | Latest Version |
+-------------+--------------+----------------+
| test_center | 3.5.8 | 3.5.9 |
+-------------+--------------+----------------+
INFO [2018-11-07 15:25:09.97]: To update all plugins, just run
INFO [2018-11-07 15:25:09.97]: $ bundle exec fastlane update_plugins

ERROR [2018-11-07 15:25:09.97]: fastlane finished with errors

#

fastlane 2.108.0 is available. You are on 2.106.0.

You should use the latest version.

Please update using `bundle update fastlane`.

#

2.108.0 Improvements

[action] add support for arch parameter to slather (#13514) via Björn Dahlgren
[sigh] support macOS and Developer ID in get_provisioning_profile action (#13607) via Matt Moriarity
[action] use train_version to detect build on download_dsym action (#13585) via Kohki Miki
[snapshot] iOS SnapshotHelper.swift Set locale only when not empty (#13556) via river2202
[snapshot] fixed typo in deliver options.rb (#13606) via Oleksii Kalentiev
[action] integrate Managed Google Play with get_managed_play_store_publishing_rights and create_app_on_managed_play_store (#13421) via Jan Piotrowski

2.107.0 Improvements

[match] need to encrypt before save (#13546) via Josh Holtz
[scan] suppress xcode output and also open report if tests fail (#12274) via Iulian Onofrei

2.106.2 Fixes for Deliver init and Match nuke

[deliver] require open uri for downloading file (#13526) via Kazuya Gosho
[match] remove un-needed encryption from match nuke (#13533) via Josh Holtz
[frameit] allow specifying frame color override in Framefile.json via Mark DiFranco

To see all new releases, open https://github.com/fastlane/fastlane/releases

Please update using bundle update fastlane
bundler: failed to load command: fastlane (/usr/local/lib/ruby/gems/2.3.0/bin/fastlane)
FastlaneCore::Interface::FastlaneTestFailure: [!] Tests have failed
/usr/local/lib/ruby/gems/2.3.0/gems/fastlane-2.106.0/fastlane_core/lib/fastlane_core/ui/interface.rb:172:in test_failure!' /usr/local/lib/ruby/gems/2.3.0/gems/fastlane-2.106.0/fastlane_core/lib/fastlane_core/ui/ui.rb:17:inmethod_missing'
/usr/local/lib/ruby/gems/2.3.0/gems/fastlane-plugin-test_center-3.5.8/lib/fastlane/plugin/test_center/actions/multi_scan.rb:25:in run' /usr/local/lib/ruby/gems/2.3.0/gems/fastlane-2.106.0/fastlane/lib/fastlane/runner.rb:253:inblock (2 levels) in execute_action'
/usr/local/lib/ruby/gems/2.3.0/gems/fastlane-2.106.0/fastlane/lib/fastlane/actions/actions_helper.rb:50:in execute_action' /usr/local/lib/ruby/gems/2.3.0/gems/fastlane-2.106.0/fastlane/lib/fastlane/runner.rb:232:inblock in execute_action'
/usr/local/lib/ruby/gems/2.3.0/gems/fastlane-2.106.0/fastlane/lib/fastlane/runner.rb:227:in chdir' /usr/local/lib/ruby/gems/2.3.0/gems/fastlane-2.106.0/fastlane/lib/fastlane/runner.rb:227:inexecute_action'
/usr/local/lib/ruby/gems/2.3.0/gems/fastlane-2.106.0/fastlane/lib/fastlane/runner.rb:157:in `trigger_action_by_name'

Sandhu412-zz on 7 Nov 2018

The reason why the summary is not printing in this case is because multi_scan is raising an exception to satisfy the :fail_build option _before_ it prints the summary. That is a bug that I can fix as part of this Issue.

lyndsey-ferguson on 8 Nov 2018

Can you modify your Pluginfile per my instructions below, run bundle install, and then run your fastlane again?

Pluginfile:

gem 'fastlane-plugin-test_center', :git => "git://github.com/lyndsey-ferguson/fastlane-plugin-test_center.git", :branch => "issue-79-multi_scan-fails-for-xcodebuild-server-death"

Please attach the console output as a text file to this issue (makes it easier for me to review).

lyndsey-ferguson on 8 Nov 2018

@lyndsey-ferguson -- The last time we did something like this in the pluginfile it complained about access rights, then you gave me the http URL for the repo. So I will just use this URL instead: https://github.com/lyndsey-ferguson/fastlane-plugin-test_center.git

gem 'fastlane-plugin-test_center', :git => "https://github.com/lyndsey-ferguson/fastlane-plugin-test_center.git", :branch => "issue-79-multi_scan-fails-for-xcodebuild-server-death"

Sandhu412-zz on 8 Nov 2018

Thanks!

lyndsey-ferguson on 8 Nov 2018

@lyndsey-ferguson -- I ran with the pluginfile configuration as mentioned above and attached is the fastlane log from the run.
Multi_Scan_TestRun-Results.txt

After all the runs and retries, as a summary, only the below tests have failed (Total of 5 tests, but count as 10 since they are two instances of each).
LoginPageTests.testForSwitchBetweenUsers()
LoginPageTests.testForSwitchBetweenUsers()
LoginPageTests.testTrailingSpaceUserNameSuccess()
LoginPageTests.testTrailingSpaceUserNameSuccess()
EditDraftTests.testEditDraftsAddImageLibrary()
EditDraftTests.testEditDraftsAddImageLibrary()
EditDraftTests.testEditDraftsAddImageLinkError()
EditDraftTests.testEditDraftsAddImageLinkError()
LoginPageTests.testForMultiUserLogin()
LoginPageTests.testForMultiUserLogin()

But multi_scan results shows total of 21 tests as failed which does not account for some of those 21 tests that PASSED in retry. A good example is in report-5 "testUnschueduleCampaign" fails, and in the retry it PASSED. But multi_scan still lists that test as failed in the multi_scan results section.

Please let me know if you need anything else from me. Thanks and much appreciated!

NOTE: Please ignore the log messages that says "Application state is still unknown for com.constantcontact.toolkit.dev (pid:10032)" It is probably something to so with the app state when the tests were running but the re-try for the same tests looks good.

Sandhu412-zz on 9 Nov 2018

This is super interesting:

collate_junit_reports with ["/Users/sanjaneya/Documents/toolkit-ios/reports/ui-tests/report.junit", "/Users/sanjaneya/Documents/toolkit-ios/reports/ui-tests/report-2.junit", "/Users/sanjaneya/Documents/toolkit-ios/reports/ui-tests/report-3.junit", "/Users/sanjaneya/Documents/toolkit-ios/reports/ui-tests/report-4.junit", "/Users/sanjaneya/Documents/toolkit-ios/reports/ui-tests/report-5.junit", "/Users/sanjaneya/Documents/toolkit-ios/reports/ui-tests/report-6.junit"]\
> collating last report file /Users/sanjaneya/Documents/toolkit-ios/reports/ui-tests/report-6.junit\
> collating testsuite ToolkitUITests.ScheduledCampaignTests\
    collate_testsuite with testcase testUnschueduleCampaign\
    replacing <testcase classname='ToolkitUITests.ScheduledCampaignTests' name='testUnschueduleCampaign'>\
      <failure message='Failed to find &quot;Drafts&quot; Other after 30.0 seconds'>DraftDetailsPage.swift:28</failure>\
    </testcase> with <testcase classname='ToolkitUITests.ScheduledCampaignTests' name='testUnschueduleCampaign' time='159.752'/>\
    target_testcase after replacement <testsuite failures='2' name='ToolkitUITests.ScheduledCampaignTests' tests='6'>\
    <testcase classname='ToolkitUITests.ScheduledCampaignTests' name='testScheduleCampaignButtonLinkImgBlocks' time='164.965'/>\
    <testcase classname='ToolkitUITests.ScheduledCampaignTests' name='testScheduleCampaignTitleTextBlocks' time='142.455'/>\
    <testcase classname='ToolkitUITests.ScheduledCampaignTests' name='testScheduleDraftCampaign' time='96.183'/>\
    <testcase classname='ToolkitUITests.ScheduledCampaignTests' name='testScheduleNoSendToListError' time='92.002'/>\
    <testcase classname='ToolkitUITests.ScheduledCampaignTests' name='testUnschueduleCampaign' time='159.752'/>\
    <testcase classname='ToolkitUITests.ScheduledCampaignTests' name='testUnschueduleCampaign'>\
      <failure message='Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: &quot;Expect predicate `exists == 1 AND hittable == 1` for object &quot;Drafts&quot; Other&quot;.'>HelperExtensions.swift:32</failure>\
    </testcase>\
  </testsuite>\
< collating testsuite ToolkitUITests.ScheduledCampaignTests\
< collating last report file /Users/sanjaneya/Documents/toolkit-ios/reports/ui-tests/report-6.junit\

Do you see that the collation successfully replaced:

<testcase classname='ToolkitUITests.ScheduledCampaignTests' name='testUnschueduleCampaign'>\
      <failure message='Failed to find &quot;Drafts&quot; Other after 30.0 seconds'>DraftDetailsPage.swift:28</failure>\
    </testcase>

with:

<testcase classname='ToolkitUITests.ScheduledCampaignTests' name='testUnschueduleCampaign' time='159.752'/>

Yet, after that replacement, the final testsuite has 2 copies of the failure, but the other has a different failure reason?

<testsuite failures='2' name='ToolkitUITests.ScheduledCampaignTests' tests='6'>\
    <testcase classname='ToolkitUITests.ScheduledCampaignTests' name='testScheduleCampaignButtonLinkImgBlocks' time='164.965'/>\
    <testcase classname='ToolkitUITests.ScheduledCampaignTests' name='testScheduleCampaignTitleTextBlocks' time='142.455'/>\
    <testcase classname='ToolkitUITests.ScheduledCampaignTests' name='testScheduleDraftCampaign' time='96.183'/>\
    <testcase classname='ToolkitUITests.ScheduledCampaignTests' name='testScheduleNoSendToListError' time='92.002'/>\
    <testcase classname='ToolkitUITests.ScheduledCampaignTests' name='testUnschueduleCampaign' time='159.752'/>\
    <testcase classname='ToolkitUITests.ScheduledCampaignTests' name='testUnschueduleCampaign'>\
      <failure message='Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: &quot;Expect predicate `exists == 1 AND hittable == 1` for object &quot;Drafts&quot; Other&quot;.'>HelperExtensions.swift:32</failure>\
    </testcase>\
  </testsuite>

That, coupled with your comment about 2 instances of each test, adds a lot of data. This would account for the discrepancy.

Why are there two instances of each test?

lyndsey-ferguson on 9 Nov 2018

@lyndsey-ferguson -- When a test fails it seems to reference two files and hence makes two entries, one for each file. For ex, as shown below, it is still just one test (testUnschueduleCampaign) but 1 failure is in the test method and the 2nd failure is in the helper extensions file. This happens for every test case that fails (I will look into seeing if I can make only one entry show for each failed case by not using that helper extensions file). But this was the case from a long time, and I didn't have this issue with multi_scan reporting the second failure. Please let me know your thoughts.

```ToolkitUITests.ScheduledCampaignTests
testUnschueduleCampaign, Failed to find "Drafts" Other after 30.0 seconds
DraftDetailsPage.swift:28

testUnschueduleCampaign, Asynchronous wait failed: Exceeded timeout of 30 seconds, with unfulfilled expectations: "Expect predicate exists == 1 AND hittable == 1 for object "Drafts" Other".
HelperExtensions.swift:32

Sandhu412-zz on 9 Nov 2018

So, technically, the test named testUnschueduleCampaign fails in HelpExtension.swift, another test testUnschueduleCampaign does not fail in DraftDetailsPage.swift. A test with the same exists in both files, and they are both in the same Target. Yes?

lyndsey-ferguson on 9 Nov 2018

Let me rephrase, one of test steps in DraftDetailsPage.swift uses a method in HelpExtension.swift and when the method fails to find the element in HelpExtension.swift, along with marking HelpExtension.swift as failed, it also marks DraftDetailsPage.swift as failed too. This is how the XCUITest framework reports the failures seems like, I dont do anything in particular.

So yes, technically it fails in HelpExtension.swift and its only one test and in the same target.

Sandhu412-zz on 9 Nov 2018

Does DraftDetailsPage.swift call a method that calls the testUnschueduleCampaign? Forgive me if it is obvious to you, I'm not trying to be obtuse. I don't yet grasp why the test is being called twice.

lyndsey-ferguson on 9 Nov 2018

The test case testUnschueduleCampaign -> has a step that calls DraftDetailsPage,swift and one of the verifications in DraftDetailsPage.swift is to look for an XCUIElement -> and the method to look for that XCUIElement is implemented in HelperExtensions.swift. So the flow is:

testUnschueduleCampaign -> DraftDetailsPage,swift -> HelperExtensions.swift

Sandhu412-zz on 9 Nov 2018

Are you saying that testUnschueduleCampaign runs, and then there is some helper test code in DraftDetailsPage.swift that looks for "Drafts", that fails, yet continues to run and then there is more helper test code that hangs in HelperExtensions.swift?

And then, in the second run, these two failures don't happen and the test passes?

lyndsey-ferguson on 9 Nov 2018

The helper test code that looks for "Drafts", is the method defined in HelperExtensions.swift file. So HelperExtensions.swift helps to find the Drafts element and if it reaches the timeout limit without finding the element, it fails.

HelperExtensions.swift file has a "waitForExpectations" helper method which raises an exception and fails the test if the timeout is reached.

Sandhu412-zz on 9 Nov 2018

Ok, so do you have the setting to continue running set to false, continueAfterFailure?

lyndsey-ferguson on 9 Nov 2018

Yes, continueAfterFailure = false in my tests.

Sandhu412-zz on 9 Nov 2018

Okay, that helps a lot. I'll have to think about the right way to fix this so that multi_scan does not produce invalid reports.

lyndsey-ferguson on 9 Nov 2018

Thanks @lyndsey-ferguson As I said, I will look into seeing if I can avoid the second failed exception to be logged by avoiding the waitForExpectations method and trying an alternative until then. So basically, multi_scan is replacing the passed instance with just one of the errors and the other error is getting reported as a test failure. If I can make the test failure report only one error, then this scenario will be eliminated. Am I correct in my assumption?

Sandhu412-zz on 9 Nov 2018

Yes, I believe your assumption is correct. @Sandhu412, you give me the most interesting challenges, thank you! A pleasure working with you.

lyndsey-ferguson on 9 Nov 2018

Nothing compared to your efforts and your super duper plugin! Appreciate it! I will keep you posted on the progress of getting rid of the second error.

Sandhu412-zz on 9 Nov 2018

👍1

Okay, here is my idea.

When collating junit file into the master junit file, I first:

check for duplicate test cases in the master junit file, call it file A.
If I find a duplicate test case, I move all <failure/> elements into the one test case.
I remove the extra test case, keeping only the first one with all the <failure/> elements.

Then, when collating files into the master junit file, for each of the non-master junit files, I:

check for duplicate test cases in the junit file to be collated into file A, call that file B.
If I find a duplicate test case, I move all <failure/> elements into the one test case.
I remove the extra test case, keeping only the first one with all the <failure/> elements.
I then replace the test cases that are in A with matching ones in B. That will ensure that we do not have duplicate test cases, and we don't lose the failures.

What do you think?

lyndsey-ferguson on 9 Nov 2018

@lyndsey-ferguson -- Those steps surely sounds like a good approach to eliminate all the duplicate entries when collating into the master junit file. Also, I dont think you really need to move all failures (if its too much work?) into one test case as long as we can remove the extra test case and keep the first one.

Also, I did some reading today and I found a better way to use waitForExpectations without having it fail the test case when the timeout is reached and return the control back to the main method from the helper method. So that way, there will be only one entry for each failed test case. I will try to implement this early next week and keep you posted. I should have an answer for you by EOD Monday or Tuesday.

Thanks again!

Sandhu412-zz on 10 Nov 2018

@Sandhu412, I believe I have fixed the issue. Can you remove the entries for fastlane-plugin-test_center in your Gemfile.lock, and re-run bundle install and then run your lane and let me know if it works?

lyndsey-ferguson on 10 Nov 2018

I've also created the pull reqeust

lyndsey-ferguson on 10 Nov 2018

@lyndsey-ferguson -- Thanks so much. Shall I just delete the existing Gemfile.lock and then run bundle install ? It should be the same effect right? Also, I should still retain the change the plugin file pointing to your branch correct?

Sandhu412-zz on 11 Nov 2018

Yeah, deleting the Gemfile.lock is far easier. Yes, keep the change in the Pluginfile so that it points to that branch so we can confirm before I merge it into master and release it.

🙇

lyndsey-ferguson on 11 Nov 2018

Sounds good. Thanks. I will get back to you with the update.

Sandhu412-zz on 11 Nov 2018

@lyndsey-ferguson -- The fix looks good 💯 🥇 It does consolidate the two errors into one and the final multi_scan result count is correct. Attached is the fastlane output. As per the fix, the tests that are passed in retry are taken into account when collating the junit reports.

:+1: This PR looks great: https://github.com/lyndsey-ferguson/fastlane-plugin-test_center/pull/84 - it's ready to merge! :shipit:

Sandhu412-zz on 11 Nov 2018

Fixed in v3.5.10

lyndsey-ferguson on 11 Nov 2018

Encountered exact same issue only to find its fixed in latest version. Thank you @lyndsey-ferguson and @Sandhu412

jineshqa on 11 Jan 2019

👍1

@Sandhu412 Do you what causes application state is still unknown?

jineshqa on 11 Jan 2019

@Sandhu412 Just out of curiosity , how do you generate reports when multiscan is performed?
What type of report are you generating as well?

I know its not fair to comment on the closed ticket , but i have no other choice.

SathyaNarayanCodeX on 21 Feb 2019

Was this page helpful?

0 / 5 - 0 ratings

Related issues

Getting bundler: failed to load command inconsistently

jordan-romeroporter-earnin · 8Comments

Double param `-resultBundlePath` in xcargs with multi_scan

b3hni4 · 7Comments

multi-scan parallel run with 1+ devices will only run on one of those devices

bpoplauschi · 9Comments

only_testing flag results in undefined method `sort' error

ericwengelking · 10Comments

Collated HTML reports display the tests outside of the coloured grid rows

lyndsey-ferguson · 5Comments

Fastlane-plugin-test_center: when XCTestCase is configured to continue after failures, failing testcases are reported multiple times and are not collated correctly

New Issue Checklist

Issue Description

Complete output when running fastlane, including the stack trace and command used

Environment

All 42 comments

#

fastlane 2.108.0 is available. You are on 2.106.0.

You should use the latest version.

Please update using bundle update fastlane.

#

🙇

Related issues

Please update using `bundle update fastlane`.