Mvvmcross: continuous integration and pre-release builds

[ghuntley]

When https://github.com/reactiveui/ReactiveUI/pull/1092 closes it would be kinda neat to rip out the CICD pipeline from ReactiveUI and implement it here in MvvmCross. Assign this issue to @ghuntley

[mufid]

Any update on this?

[Cheesebaron]

I've talked to the Bitrise guys, they only support Android and iOS currently and they didn't say much about adding support for more platforms in the future. They say their stuff is best with creating apk/ipa's as outcome of the build and not dll's as we do now. They are looking into our current build issues on their system. I showed them at Evolve the errors we get etc.

[ghuntley]

RxUI is going down the path of Bitrise for test-harness (android, ios) for running in simulator unit tests and AppVeyor for building/packaging _everything_ on _every platform_. It's the only way to do packaging in a sane manner. Then on GitHub configure three hooks, a test hook for ios/andorid/mac, a test hook for all other platforms, and a packaging hook.

@mufid I'd love to rip the RxUI pipeline out of there and implement here - you just have to wait a little longer as it's still in development.

[ferrydeboer]

Since Tamarin support TeamCity also maybe http://codebetter.com/codebetter-ci/ is worth considering. While TC imo offers most flexibility it does require more effort setting it up. I personally still prefer this over a simple setup where you have much less control over the quality.

[ghuntley]

Not that keen on CodeBetter. The complexity isn't with the vendor of CI
services, it's with just how hard it is to target this many platforms and
package put the end results. Additionally the only way to do proper
validation is release builds on physical devices as the linker does not run
on iOS simulator etc.

On 14 May 2016 at 15:26, Ferry de Boer [email protected] wrote:

Since Tamarin support TeamCity also maybe
http://codebetter.com/codebetter-ci/ is worth considering. While TC imo
offers most flexibility it does require more effort setting it up. I
personally still prefer this over a simple setup where you have much less
control over the quality.

—
You are receiving this because you were assigned.
Reply to this email directly or view it on GitHub
https://github.com/MvvmCross/MvvmCross/issues/1301#issuecomment-219201815

[ferrydeboer]

Do you mean that because Bitrise has the infra in place to (amongst others) build on iOS it takes away a lot of the complexity for setting up the pipeline?

[ghuntley]

@ferrydeboer unless builds are done on a single worker then packaging up the artifacts becomes fragile/difficult. Bitrise is perfect for running the Android, Mac and iOS test harnesses to validate unit tests but not useful for packaging as it cannot target WPA/WPF/UWP. AppVeyor is perfect for packaging but cannot run the Mac/iOS test harnesses thus a hybrid approach of a) appveyor for packaging b) bitrise for validation is needed.

The RxUI PR was merged today, the project now has continuous builds to MyGet which means Mvx draws near - want to polish up the script some more first before proceeding.

[ghuntley]

RxUI (and MvvmCross) blocked/waiting for https://twitter.com/IfErrThrowBrick/status/764188808627625984 to be promoted from Alpha to Stable channel. Once this is done AppVeyor can be used to build and package - end to end for all platforms. Bitrise will be still needed to build sample apps and perform integration tests (i.e. running apps/test assemblies on simulators)

[ktngoykalolo]

Hello I see this thing is marked up for grabs. what are the requirements for the CI pipeline exactly ?

[Cheesebaron]

In short: Build all repositories and produce releasable NuGet packages.

[ktngoykalolo]

is there a specific tool that you guys want to use ?

[Cheesebaron]

Well if you are going to build Xamarin.Mac the MvvmCross projects only build on Mac currently. So preferably somewhere, where you can provide your own build agents.

Going to look into Mobile Center from MS soon. In general most CI systems work fine.

As for build scripts, Cake or Fake would be great.

Although, I would maybe wait a bit till we have discussed #1500

[ktngoykalolo]

yes, Mobile Center might be a good option.

[Cheesebaron]

Hmm looks like Mobile Center is mainly for Apps and not libraries which produce nuget packages, similar to what Bitrise does.

[ktngoykalolo]

and what about VS Team Services ?

[Cheesebaron]

VSTS, TeamCity, Jenkins, Bamboo all green from here. I can provide a couple of build agents if needed.

[ghuntley]

I'd encourage you to look at the ReactiveUI CI scripts as it will save you a heap of time. Contains build, automatic versioning, release workflow, release notes generation etc.

Additionally use AppVeyor which now works for all platforms as long as you are generating MSIL.

Right now the core problem that needs solving is how to automatically release mvvmcross on every commit to MyGet across multiple repos. My vote is to merge all repos back into single repos and throw away the notion of plugins being a separate unit. It's just mvvmcross; a change to core is a new alpha, so would be a commit to the plugins. Version the software as a collective unit and pin all packages to the version that was built.

[ktngoykalolo]

I will look into ReactiveUI. I saw the discussion about merging all of the repos, I am leaning more towards keeping all of the repos separate.

[ghuntley]

The key is to treat everything as a collective releasable unit that shares the same version and that version is pinned. if the decision is made to keep everything as separate repos then I suppose you could achieve the later by adjusting the bootstrap to clone all repos and any commit in any upstream repo triggers build of everything but then you won't be able to implement an automatic release strategy.

[ktngoykalolo]

yes indeed. if you merge all the repos, aren't they going to get too big ?

[ghuntley]

No, that's not a valid reason for not merging. Technicality Git can handle repos in the multiple * GB range.

[kjeremy]

I wouldn't worry about the size so much. The one big issue I see with merging everything into one repo is that a critical commit to fix a crash in the file download plugin causes a new release of everything. Still that may not be a big deal if we have a CI system set up.

[ktngoykalolo]

no I mean the nuget packages resulting from that repos ? the binaries themselves

[ghuntley]

@kjeremy treat mvvmcross for what it is. Ruby on Rails. A single commit to plugins or anywhere should yield a new release. That release should constrain all dependencies or nuget packages to that particular version. This will make your life as an open source maintainer so much better. Now a release doesn't have to be released on nuget. ResctiveUI uses gitflow without support branches. Just develop and master. When a commit lands on develop that's a new release to myget. Every now and then the core group decides to do a pull request from develop to master and that tags the repo and triggers a release to nuget.

[ktngoykalolo]

nice approach

[kjeremy]

@ghuntley Right. I don't think my concern is that big of a deal once we have a working CI system.

[ghuntley]

Agreed. When the product is versioned as a collective unit the question of which version are you running becomes a single answer. Tell the consumer to upgrade to latest version of the product (not product plus add-in plus Android support ugh) managing versioning of dependencies sucks for the maintainer and is equally confusing for consumers who lack the project domain knowledge.

Happy to jump on a video chat and share experiences. I've spent the last 8 months mulling over and implementing this stuff. Lot's of failure on the way to success. It seems simple at first but it's not. Lot's of little subtleties such as amount of platforms, some platforms only build on certain CI hosts, versioning, packaging, release workflow, issue mgmt and PR approvals.

For whomever takes up the challenge; effort wise set aside at least one month part time. It takes approx eight minutes to build all of RxUI which provides 5 attempts per hour. Mvx has more code so that means that the developer feedback cycle will be slower.

[Cheesebaron]

no I mean the nuget packages resulting from that repos ? the binaries themselves

No nothing will change in how the NuGet packages are released.

[Cheesebaron]

CI is up and running

[ghuntley]

Nice.

[Dexyon]

@Cheesebaron Nice work !

[Cheesebaron]

CI build now dumps NuGets on MyGet for the develop branch here: https://www.myget.org/F/mvvmcross/api/v2

[Cheesebaron]

Now we only need it to make Git Releases.