theia 🚀 - Set up CI system to create desktop installer of vanilla Theia

@thegecko FWIW, Travis now has a Windows VM for you to use. You may also use Azure pipelines as well.

sr229 on 15 Jun 2019

Thanks. The Mac ci may be tricky

thegecko on 15 Jun 2019

use Azure pipelines

+1

The Mac ci may be tricky

We have not had problems producing a dmg for macOS with Azure so far.

kittaakos on 2 Jul 2019

We have not had problems producing a dmg for macOS with Azure

Have you tried signing the image (if we care about that)?

thegecko on 2 Jul 2019

Have you tried signing the image

No.

kittaakos on 2 Jul 2019

At this point we can have our CI migrated to Azure Pipelines since it supports MacOS, Windows and Linux already @thegecko @kittaakos

sr229 on 3 Jul 2019

How about using GitHub actions instead? https://github.blog/2019-08-08-github-actions-now-supports-ci-cd/#fast-ci-cd-for-any-os-any-language-and-any-cloud

akosyakov on 26 Aug 2019

GitHub actions instead

Does it support downloadable artifacts without creating a new release? Azure does.

Otherwise, I do not have any objections.

kittaakos on 26 Aug 2019

GitHub CI is still in beta so we'll need to wait for their november GA.

sr229 on 27 Aug 2019

How about using GitHub actions

I believe GH actions use docker containers behind the scenes, so are Linux only.

This would mean we couldn't sign installers for Mac / Windows :(

thegecko on 3 Sep 2019

It was just a suggestion, I would be happy with anything working :)

akosyakov on 3 Sep 2019

I would be happy with anything working

I get the hint :p

thegecko on 3 Sep 2019

😄1

Any progress on this?

chrisjj on 2 Feb 2020

No progress as yet, many options have been suggested (Azure, Travis), but as I only have experience building and signing using Jenkins servers more help will be needed if we take a different approach.

As I see it, there are 3 options:

Setup up Jenkins servers taking advantage of the eclipse foundation build systems. This approach builds on a known release strategy we use for Mbed Studio, but requires access to eclipse servers and may diverge our CI from what we already have
Create something from scratch using Travis. I would need some support from someone with Travis foo as I don't have time to research/learn it :). We can use existing Jenkins scripts as a starting point.
Create something from scratch using Azure. Again, I would need some support from someone with Azure foo as I don't have time to research/learn it :). We can use existing Jenkins scripts as a starting point.

Any solution we choose needs to support native MacOS, Windows and Linux build machines in order to sign any installers (if that's a requirement).

Thoughts/volunteers?

thegecko on 3 Feb 2020

I somehow like to reuse eclipse build systems. I think it is fine that it is a separate CI, bundling from monorepo does not work anyway, correct? It would be also close to what others have to do. Could scripts be open sourced?

akosyakov on 3 Feb 2020

We can sanitize the scripts and share them to use as a base line. Do we have a contact at eclipse to access their build system?

thegecko on 3 Feb 2020

opening a bugzilla to get CI instance at https://ci.eclipse.org/theia ?

benoitf on 3 Feb 2020

Thanks all.

chrisjj on 3 Feb 2020

https://bugs.eclipse.org/bugs/show_bug.cgi?id=559794

thegecko on 3 Feb 2020

Any solution we choose needs to support native MacOS, Windows and Linux build machines in order to sign any installers (if that's a requirement).

Just a quick mention that Travis CI meets this requirement of supporting CI for our 3 platforms. Not sure about The Foundation's CI infrastructure - I guess they'll let us know in the bugzilla above.

marcdumais-work on 4 Feb 2020

👍1

Travis CI meets this requirement of supporting CI for our 3 platforms

Excellent, I just need a volunteer with travis foo :)

thegecko on 5 Feb 2020

Excellent, I just need a volunteer with travis foo :)

A few of us have played with Travis and can help.

When we talk about "vanilla Theia", do we mean the Electron Example application from the main repo, e.g. built and packaged nightly?

We have this example electron app that can already generate native packages for our 3 platforms (manually for now, not included in repo's CI). The electron-builder config could be used as an example, e.g. used along with the Electron Example app's package.json.

marcdumais-work on 5 Feb 2020

Yes. https://github.com/theia-ide/theia-apps/blob/master/theia-electron/electron-builder.yml is a really good start. After 1.0.0 it may be worth adding more improvements. For example:

Enabling Asar (It will require few changes in Theia code. Quite important to reduce the disk size and installation time of Theia)
Updating electron-builder version (There are few known issues in newer versions that may break backward compatibility)
Adding electron-updater for auto-updates

arekzaluski on 5 Feb 2020

👍2

There will be some preliminary tasks before we get to the CI/Travis aspect:

for each of our supported platforms, chose which packaging(s) we want. Note: For Linux there are packagings that are distro-specific (e.g. .deb, .rpm) and one that's generic (AppImage).
for each wanted packaging, ideally find a public registry where we can push the Theia app, once packaged. e.g. for Docker images, we use dockerhub. If in some cases we can't find such a public registry, we can consider using a generic file hosting service (e.g. Eclipse Foundation infrastructure maybe).
- create an account on each of these registries. Ideally they can provide a revocable authentication token that we can use to authenticate during CI, before pushing. We can add the tokens as Travis environment variables, like we did for dockerhub and npmjs

marcdumais-work on 5 Feb 2020

We have this example electron app that can already generate native packages for our 3 platforms

I think this is the ideal candidate

There will be some preliminary tasks before we get to the CI/Travis aspect:

IMO, getting the installers building and signing will be the hardest part here. The content and publish location doesn't necessarily block work in this area. I'm happy to share some (potentially redacted!) Jenkins scripts to support anyone trying this on Travis. Let me know where to put them.

for each of our supported platforms, chose which packaging(s) we want. Note: For Linux there are packagings that are distro-specific (e.g. .deb, .rpm) and one that's generic (AppImage).

After a lot of research and trial/error on our side, we recommend the following:

Linux .deb (and .sh) installer containing an AppImage as it's far easier to upgrade
Windows .msi (in an .exe)
MacOS .pkg

for each wanted packaging, ideally find a public registry where we can push the Theia app, once packaged. e.g. for Docker images, we use dockerhub. If in some cases we can't find such a public registry, we can consider using a generic file hosting service (e.g. Eclipse Foundation infrastructure maybe).

DockerHub makes sense for the docker images (shall we create one now?). For the installers, we could simply add them to the GitHub releases on Theia?

create an account on each of these registries. Ideally they can provide a revocable authentication token that we can use to authenticate during CI, before pushing. We can add the tokens as Travis environment variables, like we did for dockerhub and npmjs

Sensible.

thegecko on 11 Feb 2020

Most frequent complain for last 2 days after Theia 1.0: https://dev.to/abuassar/comment/n7h0

I'm keeping saying that we target adopters not end users, but having installer won't harm.

akosyakov on 2 Apr 2020

👍3

@akosyakov I agree with you wholeheartedly. But I believe you can still fish for adopters with installers. Even if it's a simple "hey check out Theia as a desktop app".

hsheikhali on 2 Apr 2020

I'm not going to have time to dedicate to this for the next Month or so at least I'm afraid.

thegecko on 2 Apr 2020

But I believe you can still fish for adopters with installers.

Installer would be nice, but I'd settle for just a working .exe.

I gave up looking for an official example .exe and the best substitutes I found were Arm mbed and Arduino Theia-based IDE. Given how poor those bests are by the standards of desktop software, an official example IDE (assuming it is better) would avoid discouragement of potential adopters likewise diverted to those substitutes.

chrisjj on 2 Apr 2020

@chrisjj Agreed. @thegecko mentioned he didn't have time for this for the next month -- thing about open source is that someone else can pick up that project :)

hsheikhali on 2 Apr 2020

Typically, the core of Electron apps is an .asar file, if the compile environment is limited but with a local Electron runtime, one can just produce the .asar file, and run it with the Electron runtime itself.

If there is any problem on the platforms, or I'd like to say, "the Chromium part", and the .asar file can run swiftly on the native Electron (VSCode and Atom's .asar files can't. They modified the Chromium part without writing something try...catch in the JavaScript), then there is a choice to release the .asar file of Theia for those who already got / would like to download an Electron runtime.

This sinks the file size, and also costs less energy. But the result is either not a installer nor a portable program, and there is no visible sign that Electron would support the .asar file and encourage people to download a runtime and make use of the .asar files directly. We still need the installers, and maybe some portable versions / solutions in the future.

Crystal-RainSlide on 6 Apr 2020

👍1

I would love to try a more flexible layout than VSCODE but it is really not understandable that there is no action on this that an end-user can install this on the mayor three platforms like VSCODE. It seems like there is a big interest and momentum on this.

ride4sun on 25 May 2020

Three months and still there is apparently no published build of vanilla
desktop Theia. I'm hoping that's because such a build isn't possible, since
that's probably more fixable than the alternative, being that leading
contributors it not worth providing.

On Monday, 25 May 2020, ride4sun notifications@github.com wrote:

I would love to try a more flexible layout than VSCODE but it is really
not understandable that there is no action on this that an end-user can
install this on the mayor three platforms like VSCODE. It seems like there
is a big interest and momentum on this.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/eclipse-theia/theia/issues/5481#issuecomment-633336455,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABO7GEPU66H5TMQG7P5L53TRTHD4NANCNFSM4HYJHQQQ
.

chrisjj on 25 May 2020

Three months and still there is apparently no published build of vanilla desktop
Theia. I'm hoping that's because such a build isn't possible, since that's
probably more fixable than the alternative, being that leading contributors
it not worth providing.

Correction: that leading contributors consider it not worth providing.

On Monday, 25 May 2020, chrisjj notifications@github.com wrote:

Three months and still there is apparently no published build of vanilla
desktop Theia. I'm hoping that's because such a build isn't possible, since
that's probably more fixable than the alternative, being that leading
contributors it not worth providing.

On Monday, 25 May 2020, ride4sun notifications@github.com wrote:

I would love to try a more flexible layout than VSCODE but it is really
not understandable that there is no action on this that an end-user can
install this on the mayor three platforms like VSCODE. It seems like
there
is a big interest and momentum on this.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
issuecomment-633336455>,
or unsubscribe
ABO7GEPU66H5TMQG7P5L53TRTHD4NANCNFSM4HYJHQQQ>
.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/eclipse-theia/theia/issues/5481#issuecomment-633557504,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABO7GEKTAVOIX3ID2O764S3RTJSZFANCNFSM4HYJHQQQ
.

chrisjj on 25 May 2020

@chrisjj This is open-source - PRs are welcome

marcdumais-work on 25 May 2020

👍2

You'd think so. Yet still this one hasn't appeared.

On Monday, 25 May 2020, Marc Dumais notifications@github.com wrote:

@chrisjj https://github.com/chrisjj This is open-source - PRs are
welcome

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/eclipse-theia/theia/issues/5481#issuecomment-633591811,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABO7GEM7BR72CEWQXFVMI7DRTJ4G3ANCNFSM4HYJHQQQ
.

chrisjj on 25 May 2020

👎5

@chrisjj Instead of complaining, why don't you take initiative and make the PR to support the builds? Desktop builds are possible but its something the team isn't yet willing to provide since there are more important issues to tackle for now (such as better extension support).

sr229 on 25 May 2020

👍4 😕1

I am wondering if that is not done (better extension support) yet why even that big announcement for version 1.0. Maybe that was a bit premature. I thought Theia is a competition of VSCODE. The strength of VSCODE is multiplatform. The weakness of VSCODE is the non-flexible Layout and lousy multiple screen support. Big announcement but sorry - you can not try it out on your code. (Really)

Sorry, an installer is something I would just call "basic feature". There is always something more important.

ride4sun on 27 May 2020

I am wondering if that is not done (better extension support) yet why even that big announcement for version 1.0. Maybe that was a bit premature. I thought Thea is a competition of VSCODE. The strength of VSCODE is multiplatform. The weakness of VSCODE is the non-flexible Layout and lousy multiple screen support. Big announcement but sorry - you can not try it out on your code. (Really)

Sorry, an installer is something I would just call "basic feature". There is always something more important.

@ride4sun based on your comment it looks like you might have misunderstood what Theia is.
Theia is in fact a framework or platform to build IDE-like applications and it is not an IDE in itself (like vscode). The project allows application-developers to build and customize their own IDE-like products while also providing the flexibility and extensibility which allow for a wide range of use-cases.

Having an installer for the 'default' application (ex: example-browser or example-electron applications present in the repo) are actually nice-to-haves and is not a 'basic feature's like you mentioned.

vince-fugnitto on 27 May 2020

Sorry, but it sounded a good alternative to VSCODE to me when I was reading the announcements:

https://www.globenewswire.com/news-release/2020/03/31/2009010/0/en/The-Eclipse-Foundation-Releases-Eclipse-Theia-1-0-a-True-Open-Source-Alternative-to-Visual-Studio-Code.html

Quote:
Theia is designed from the ground to run on both Desktop and Cloud... Eclipse Theia is designed to work as a native desktop application as well as in the ...

Native Desktop which oh sorry you can not install because it is not important enough?

Even the title says that:
The Eclipse Foundation Releases Eclipse Theia 1.0, a True Open Source Alternative to Visual Studio Code

So let me get the correct... it is an alternative I can just not install it????

Quote: (https://devclass.com/2020/03/31/eclipses-theia-sees-1-0-declared-not-your-parents-ide/)
Mike Milinkovich, executive director of the Eclipse Foundation, said in a statement: “Visual Studio Code is one of the world’s most popular development environments. Not only does Theia allow developers to install and reuse VS Code extensions, it provides an extensible and adaptable platform that can be tailored to specific use cases.”

So I can install an extension like in VSCODE but I can not install THEIA???

ride4sun on 27 May 2020

>

@chrisjj https://github.com/chrisjj Instead of complaining, why don't
you take initiative and make the PR to support the builds?

I've no idea how to make such a PR. Plus as I said, the fact that far more
able persons than I haven't done it suggests it is much more difficult that
you imply.

C

>

chrisjj on 27 May 2020

ride4sun, let's add:

The most significant differences between Eclipse Theia and VS Code are:

Theia's architecture is more modular and allows for significantly more
customizations
Theia is designed from the ground to run on both Desktop and Cloud
Theia is developed under the community-driven and vendor-neutral
governance of the Eclipse Foundation.

which at the moment should also include

Theia doesn't give you an .exe because that's not important enough.

C

chrisjj on 27 May 2020

@ride4sun if you look at the quotes you referenced you'll see the following:

Not only does Theia allow developers to install and reuse VS Code extensions, it provides an extensible and adaptable platform that can be tailored to specific use cases.

In the EclipseFoundation announcement you'll see:

The Eclipse Foundation, one of the world’s largest open source foundations, today announced the release of Theia 1.0, a true open source alternative to Microsoft’s popular Visual Studio Code (VS Code) software. Eclipse Theia is an extensible platform to develop multi-language Cloud and Desktop Integrated Development Environments (IDEs) with state-of-the-art web technologies that enable developers, organizations, and vendors to create new, extensible developer experiences.

It clearly illustrates that the project is a framework which can build IDE-like applications which in fact rival and is an open-source alternative to VS Code. The main target audience for the framework are application-developers who are in need of a framework to build their own white label IDE-like applications.

The idea of providing an executable or installer to end-users is for the ability for more of an audience to get a feel of the default functionality and behavior of the framework but once again it will only be one example of many possible use-cases. Nothing prevents you today to build you own application using Theia.

In any case, the addition of desktop installers for an example Theia application, is being actively worked on by the main community and hopefully it will bring forth more visibility and users to the framework.

vince-fugnitto on 27 May 2020

I worked on many software projects myself and I have to admit that the installer is something nobody wants to work on. I have the feeling that is the non-outspoken reason here.

That is really stupid because I really want to try the new layout engine and you guys should be eager to get a big install base to gain momentum and get feedback.

ride4sun on 27 May 2020

@vince-fugnitto

Let make it clear than....

Theia 1.0, a true open source alternative to Microsoft’s popular Visual Studio Code

This is _NOT_ a Vscode alternative because it addresses another use case. Totally misleading!

ride4sun on 27 May 2020

@ride4sun

I worked on many software projects myself and I have to admit that the installer is something nobody wants to work on. I have the feeling that is the non-outspoken reason here.

That is really stupid because I really want to try the new layout engine and you guys should be eager to get a big install base to gain momentum and get feedback.

@ride4sun it is being worked on :)
Providing end-users with installers is something the community wants to ensure is done right, and the necessary planning, background tasks, setting up the CI jobs and permissions.

Let make it clear than....
This is NOT a Vscode alternative because it addresses another use case. Totally misleading!

It can be argued that it is an open-source alternative since vscode is not truly open-source, we are providing vscode extension support, we have worked on providing an open-source alternative to the marketplace for downstream applications to use (open-vsx), and the default commands, implementation and user-interface are vscode inspired.

If you disagree then that is totally your opinion, I think the multiple comments (re-iterating the same point) are sufficient for this particular issue.

vince-fugnitto on 27 May 2020

It clearly illustrates that the project is a framework which can build
IDE-like applications ...

It would do... if it were not contradicted by e.g. "a True Open Source
Alternative to Visual Studio Code".

The fact Theia is a platform for building IDEs whereas VSCode is an IDE is
probably the most important entry that should be on Theia's "The most
significant differences between Eclipse Theia and VS Code are..."

None of which bears on the deficiency of an actual .exe for vanilla Theia.

It can be argued that it is an open-source alternative since vscode is
not truly open-source

No. That's merely makes Theia a more open-source non-alternative.

C

chrisjj on 27 May 2020

I agree with that statement but that is quite a different statement than the press releases.

It can be argued that it is an open-source alternative since vscode is not truly open-source, 
we are providing vscode extension support, we have worked on providing an open-source
alternative to the marketplace for downstream applications to use (open-vsx), and the default commands, implementation and user-interface are vscode inspired.

Can you share the community efforts where people working on the installer?

ride4sun on 27 May 2020

Can you share the community efforts where people working on the installer?

The main community has discussed this feature and other features in the weekly dev-meeting.

vince-fugnitto on 27 May 2020

The main community has discussed this feature and other features in the
weekly dev-meeting
https://github.com/eclipse-theia/theia/wiki/Dev-Meetings.

Any sign of "actively working on"? As per your "the addition of desktop
installers for an example Theia application, is being actively worked on by
the main community".

On Wed, 27 May 2020 at 00:31, Vincent Fugnitto notifications@github.com
wrote:

Can you share the community efforts where people working on the installer?

The main community has discussed this feature and other features in the
weekly dev-meeting
https://github.com/eclipse-theia/theia/wiki/Dev-Meetings.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/eclipse-theia/theia/issues/5481#issuecomment-634334124,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABO7GELLEYRKQZMNJVMS7MLRTRGMRANCNFSM4HYJHQQQ
.

chrisjj on 27 May 2020

@chrisjj the issue has been discussed and planned by the main community, and one of the main contributing companies are currently working on implementing the feature. In the dev-meeting, the community has discussed on what the best approach would be to handle the automation of the installers, the CI setup (for different operating systems), and other important factors.

vince-fugnitto on 27 May 2020

Is there a ETA?

ride4sun on 27 May 2020

Is there a ETA?

@ride4sun no, there is none as of today. We need to make sure that the automation of the installers are done correctly and with quality as ultimately end-users will be the ones to consume them. If you are pressed for time, building you own IDE is quite easy, we have many examples and you can start using it today. For example, if you require a desktop Theia IDE, you can take a look at the theia-electron example which when built and packaged you can continue using like any standard application.

vince-fugnitto on 27 May 2020

Locked except for committers. Pestering the project for one's pet feature is not productive.

As @vince-fugnitto said, Theia is a framework. It's targeted at IDE Developers that want an alternative to forking VS Code and/or something VS Code-like that also runs well in the Cloud. This project was never meant to host an IDE product based on Theia - there will be other Eclipse Foundation projects for that eventually. Here are the details: https://projects.eclipse.org/projects/ecd.theia

For those that just want to use (vs make/design/maintain) an alternate desktop IDE, like VS Code but open-source, your best bet right now is VSCodium. I have no doubt we'll have installers here soon, but what it will install will not be a full-fledged IDE product.

@thegecko would you share your plans as per the dev-meeting? I can then link to here from the minutes :)

marcdumais-work on 27 May 2020

@thegecko would you share your plans as per the dev-meeting

Sure, as discussed in the dev meeting yesterday, the plan is to stand up a travis build system for the theia apps project.

For now we want to prove this will work so Initially we aren't going to worry about signing any installers and will also defer decisions around the channels to be released.

To set delivery expectations, this is something I'm doing in my spare time, so please offer help if you want this accelerated. I see Windows support on travis is experimental, so we may hit blockers quickly.

thegecko on 27 May 2020

Thanks Rob.

I see Windows support on travis is experimental, so we may hit blockers quickly.

True, but we've been using Travis on this repo here, for our Windows CI, since we moved to the Eclipse Foundation's org, and as far as I remember, we've had little issues caused by the service (windows-specific issues, e.g. related to the specificities of their file system, are another matter)

See here for the windows part of the Travis config we use here.

marcdumais-work on 27 May 2020

In order to correctly sign these

@thegecko, you have some experience with creating and shipping a signed electron-based app. Can you please tell us how long does it roughly take to sign the _Arm Mbed Studio_ for macOS? Have you managed to pack with asar? Thank you!

I see Windows support on travis is experimental, so we may hit blockers quickly.

Recently, I have managed to sign a dummy electron app with GH Actions so if Travis CI cannot do the Windows part, we can use GH Actions.

Unfortunately, I was hitting timeout issues (with GH Actions) when I wanted to sign a Theia-based app. Probably because it's a giant, almost 500MB (VS Code is ~240 MB).

kittaakos on 29 May 2020

Can you please tell us how long does it roughly take to sign

Builds are about 40 minutes in total for each platform

Have you managed to pack with asar

We don't use asar

...if Travis CI cannot do the Windows part, we can use GH Actions

I wasn't aware GH actions were available for Windows, I assumed they were all docker images

thegecko on 29 May 2020

Builds are about 40 minutes in total for each platform

Does this 40 min include the singing against the Apple server?

kittaakos on 29 May 2020

Does this 40 min include the singing against the Apple server

Yes, it includes macOS notarization

thegecko on 29 May 2020

@thegecko I noticed that the VSCodium project generates packages for many OS, and it looks like they sign the Mac ones using Travis. Maybe this can serve as inspiration.

marcdumais-work on 29 May 2020

Yes, it includes macOS notarization

Thank you, @thegecko. It was a very useful input. Does this mean you do not have the "sign" : true explicitly set in the config, you run the notarization only but not an explicit singing, right?

also defer decisions around the channels to be released.

I propose not using channels (well, just the default, latest), but having a separate _insider_ build which gets an update each evening. VS Code uses this pattern, it works perfectly.

If anyone is wondering what the channels are: https://www.electron.build/tutorials/release-using-channels

kittaakos on 3 Jun 2020

PR for a first pass of this here: https://github.com/theia-ide/theia-apps/pull/370

thegecko on 6 Jun 2020

First releases of theia-example v1.2.0 are now appearing here:

https://download.eclipse.org/theia/

Still unsigned and currently untested.

thegecko on 29 Jul 2020

Signed versions now available:

https://download.eclipse.org/theia/1.2.0/

MacOS version still needs to be notarised.

thegecko on 18 Aug 2020