Fastlane-plugin-test_center: Survey for the next feature

[lyndsey-ferguson]

I am looking to prioritize the next feature for my test_center plugin and wanted to get your input. As you've interacted with the plugin before, I wanted to reach out to you to see if you had a preference for one of these features, or a suggestion for something I hadn't considered yet.

I greatly appreciate you taking the time to share your thoughts. It is only one question and shouldn't take more than 30 seconds of your time (unless you suggest something else I hadn't thought of).

https://www.surveymonkey.com/r/TT97XGH

Note: I'll close the survey on July 15, 2020. Thanks!

@MostafaGazar
@ilyavelilyaev
@axtonpitt
@ChMrWang
@NicholasTD07
@zhentan
@limamedeiros
@giginet
@ChiWhiteSox56
@alihen
@mikebob
@szymon-kazmierczak
@Frankatvend
@coldmint
@jpricke1-ford
@Edward-Bazko
@vovanl10n
@jainhitesh3
@bananita
@JaviSoto
@yurapriv
@tinder-swchung
@horowitz
@ricardomsm
@BorisOsipov
@artFintch
@krzemienski
@THIRISANGU
@paciej00
@zanizrules
@chika-kasymov
@bootstraponline
@Zeta611
@piohhmy
@wilferrel
@vdavydovHH
@jrkissel
@baorsmile
@apetrone
@ffittschen
@0rtm
@ripunjayasaran
@kyleturner
@pilgwon
@qatodev
@dylanbeadle
@Monolith13
@UnsafePointer
@gfioretti
@amelendezSGY
@levibostian
@ferranabello
@grigorye
@fabiochoupina
@nebiros
@quintuschen
@alephao
@Ananpipi
@kinwahlai
@eploko
@zacoid55
@lucasecf
@ryanisnhp
@04n0
@joshbuhler
@AdamSliwakowski
@jacks205
@LucianoTurrini
@zh-se
@chuganzy
@sesang06
@JasonDobo
@mtnbarreto
@sirfarmalot
@mr-v
@ljesus
@matthiaszarzecki
@robnadin
@MahmoudAbdelhameed
@kerryknight
@krezzoid
@tendai-moffatt
@rap-morgan
@pumano
@armando-leal
@yashwanth-l
@saoudrizwan
@kamvoick
@AsahiJung
@henryhardy
@BohdanOrlov
@SiarheiFedartsou
@evandcoleman
@jonathanyee
@getogrand
@saltylight
@sandra2335
@rastersize
@crazyjooe
@croberts22
@roland9
@asos-petergoldsmith
@Vkt0r
@T-Jedsada
@cisum-la
@yousefhamza
@pschneider
@prolificcoder
@romanofficial
@fabioknoedtbcgdv
@westdorp
@HerdzikWallapop
@phil-alekhin
@alteral
@KrauseFx
@bearjaw
@BobbyRohweder
@romy-aikon
@fahmisdk6
@danielblokus
@bartvandendriessche
@BarryDuggan
@kananats
@hash3r
@xavierLowmiller
@techinpark
@navartis
@revolter
@denisq1991
@shkhaliq
@DominikBucher12
@littlebobert
@Cyberbeni
@lquigley
@kaansariveli
@pruinis
@sokiaoba
@AllAcesHussam
@dharr419-ford
@dpwolfe
@rjstelling
@lecksfrawen
@JoeAlamo
@romyilano
@Mazyod
@lachlan351
@almazrafi
@kwstasna
@jazz-mobility
@dral3x
@pfrischmuth
@biow0lf
@adamcumiskey
@gewashington
@dcgorgan
@oklimberg
@slacklab

[lyndsey-ferguson]

Here are the results!

There was a tie between "A new action that reads the passing and failing tests from an xcresult bundle" and "Generate a single Junit report for all test targets rather than a report for each". I'll break the tie and choose the one that interests me the most:

"A new action that reads the passing and failing tests from an xcresult bundle"

[lyndsey-ferguson]

Thank you everyone!

[lyndsey-ferguson]

BTW, one person wrote:

I am not suggesting you to implement this, but I want to share how our teams uses multiscan. We use it for initial test run of uitests. Then after it finishes we collect failed tests and rely them using regular scan. We found this approach to be more stable on our pipeline

Keep in mind that multi_scan is calling scan under the hood, which then calls xcodebuild test. So, by running multiple test "tries" in multi_scan you are doing the same thing...._unless_ you are calling scan with the failed tests in a different CI agent, which would mean that the environment is different (including the CoreSimulator service). Conceptually, you could reset the CoreSimulator between test runs in multi_scan if that is what you're looking for.

If so, you could do that now with a testrun_completed_block by calling the test_center action quit_core_simulator_service; just don't do that when you've specified parallel_testrun_count as you'll stop any other parallel runs from running.

[lyndsey-ferguson]

I created the new action to get test results from your xcresult bundles! 🎉

Check it out here

[lyndsey-ferguson]

Next up, generate 1 single junit from multiple junits (due to multiple test targets in the same project).

[lyndsey-ferguson]

Generating 1 single junit from multiple junit reports from different test targets is complete.

[lyndsey-ferguson]

Next up, control of how the batches are created.

[lyndsey-ferguson]

Want to explicitly control how your tests are batched rather than dividing them up evenly? The fastlane plugin, test_center has got you covered in 3.15.0!

[lyndsey-ferguson]

Fastlane plugin test_center users: do you want to stop retrying your fragile tests early? Or do you want to change what tests are run in the next retry?

Checkout v3.15.0

[lyndsey-ferguson]

Generating 1 single junit from multiple junit reports from different test targets has been released publicly in v3.14.3

[lyndsey-ferguson]

I would like to share an article I wrote about using fastlane and HashiCorp Vault. If you develop and release mobile applications, this can help you manage code-signing credentials in a secure and semi-automated manner.

https://medium.com/appian-engineering/automate-secure-code-signing-assets-3331d09d7c7c

[lyndsey-ferguson]

Control which batches are tested, and control what happens next after a testrun has completed in v.3.14.4