Theia: ci: replace travis with github actions
Bug Description:
Currently, there is no longer CI for master and pull-requests since we have exhausted our allocated credits (~1000 min) for travis. A potential replacement would be to migrate to github actions which is currently free for open-source projects. We also currently use workflows in this repository and others.
Alternatively, if anyone has other solutions besides github actions we can take a look into it.
Additional Information:
vince-fugnitto
All 21 comments
@marcdumais-work, can you please help us get the secrets for the GH Actions? Thanks!
kittaakos
on 24 Nov 2020
As I see, we need one single secret for the GH Actions workflow: NPM_AUTH_TOKEN. I have access to this secret but I cannot edit the repository settings. Who can help to configure the secrets? Can we name someone on this issue? Thanks!
kittaakos
on 24 Nov 2020
As I see, we need one single secret for the GH Actions workflow:
NPM_AUTH_TOKEN. I have access to this secret but I cannot edit the repository settings. Who can help to configure the secrets? Can we name someone on this issue? Thanks!
@kittaakos I believe the only person who can update the repository settings would be the @eclipsewebmaster, perhaps @marcdumais-work can confirm?
vince-fugnitto
on 24 Nov 2020
Hi @kittaakos ,
I think @vince-fugnitto may be correct - the repo's settings available to me are almost nothing - some moderation options and such. Nothing about integrations at all. It the org level, I do not even see the settings link. Travis was awesome in its configurability - letting anyone with write access configure pretty much anything without org admin help.
The way forward is to open a bugzilla, from memory, under something like: eclipse foundation -> community -> GH, and request that they set a secret you provide. They'll want a secure way to exchange it, such as a GPG-encrypted email. If you can't easily send such an email, there are alternatives - ping me for an easy one I personally used.
update: start here
marcdumais-work
on 24 Nov 2020
Hello,
you don't need to ping webadmins for secrets
all committers can do it
Here an example on how I used it for QUAY.io credentials on some che repositories
https://github.com/benoitf/github-secrets-generator
benoitf
on 24 Nov 2020
https://developer.github.com/v3/actions/secrets/#create-or-update-a-repository-secret
(you don't have a very nice UI without admin rights but you can still do it with REST :-)
( I used python library long time ago because attempts with tweetsodium failed (I had a successful encoded value but it was not working)
benoitf
on 24 Nov 2020
Thanks @benoitf . It will be nice to have a UI eventually, but better a REST API than nothing.
marcdumais-work
on 24 Nov 2020
For the record, the current Travis build does not work as we would expect. Here is the problematic part:
https://github.com/eclipse-theia/theia/blob/6ef08676314a2ceca93023ddd149579493ae7914/.travis.yml#L100-L103
We used to support the THEIA_SKIP_NPM_PREPARE env variable to skip the prepare npm script, do the dependency install, then make some magic with the Electron codecs before the tsc build (I never understood what), finally run the tsc build.
Unfortunately, we ignore THEIA_SKIP_NPM_PREPARE and we do a yarn install, tsc build, then make some magic with the Electron codecs after the tsc build, finally, we run the tsc build again.
Here is the relevant part from Travis logs:
468.87s$ THEIA_SKIP_NPM_PREPARE=1 yarn install --skip-integrity-check
yarn install v1.13.0
$ node-gyp install
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | linux | x64
gyp http GET https://nodejs.org/download/release/v12.19.1/node-v12.19.1-headers.tar.gz
gyp http 200 https://nodejs.org/download/release/v12.19.1/node-v12.19.1-headers.tar.gz
gyp http GET https://nodejs.org/download/release/v12.19.1/SHASUMS256.txt
gyp http 200 https://nodejs.org/download/release/v12.19.1/SHASUMS256.txt
12.19.1
gyp info ok
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
info [email protected]: The platform "linux" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
info [email protected]: The platform "linux" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[4/5] Linking dependencies...
warning " > [email protected]" has unmet peer dependency "chai@^4.1.2".
warning "workspace-aggregator-9ca6415d-85f1-41d5-b9f3-94dec8a6f61b > @theia/application-manager > [email protected]" has unmet peer dependency "font-awesome@>=4.3.0".
warning "workspace-aggregator-9ca6415d-85f1-41d5-b9f3-94dec8a6f61b > @theia/debug > [email protected]" has unmet peer dependency "request@2.*.*".
warning "workspace-aggregator-9ca6415d-85f1-41d5-b9f3-94dec8a6f61b > @theia/messages > [email protected]" has unmet peer dependency "react@>=16.3.3".
warning "workspace-aggregator-9ca6415d-85f1-41d5-b9f3-94dec8a6f61b > @theia/messages > [email protected]" has unmet peer dependency "react-dom@>=16.3.3".
warning "workspace-aggregator-9ca6415d-85f1-41d5-b9f3-94dec8a6f61b > @theia/scm > [email protected]" has unmet peer dependency "react@^0.14.0 || ^15.0.0 || ^16.0.0".
warning "workspace-aggregator-9ca6415d-85f1-41d5-b9f3-94dec8a6f61b > @theia/scm > [email protected]" has unmet peer dependency "react-dom@^0.14.0 || ^15.0.0 || ^16.0.0".
warning "workspace-aggregator-9ca6415d-85f1-41d5-b9f3-94dec8a6f61b > @theia/application-manager > font-awesome-webpack > [email protected]" has incorrect peer dependency "less@^2.3.1".
[5/5] Building fresh packages...
$ node scripts/post-install.js
@theia/electron last logs:
(Fri, 20 Nov 2020 18:12:59 GMT) info: Downloading tmp-8047-0-ffmpeg-v9.3.3-linux-x64.zip
[> ] 0.0% (0 B/s)
Downloading tmp-8047-0-ffmpeg-v9.3.3-linux-x64.zip
[> ] 0.0% of 4.52 MB (0 B/s)
Downloading tmp-8047-0-ffmpeg-v9.3.3-linux-x64.zip
[============================================>] 100.0% of 4.52 MB (4.52 MB/s)
Downloading tmp-8047-0-ffmpeg-v9.3.3-linux-x64.zip
[============================================>] 100.0% of 4.52 MB (4.52 MB/s)
Downloading tmp-8047-0-ffmpeg-v9.3.3-linux-x64.zip
[============================================>] 100.0% of 4.52 MB (4.52 MB/s)
Downloading tmp-8047-1-SHASUMS256.txt-9.3.3
[> ] 0.0% (0 B/s)
Downloading tmp-8047-1-SHASUMS256.txt-9.3.3
[> ] 0.0% of 6 kB (0 B/s)
Downloading tmp-8047-1-SHASUMS256.txt-9.3.3
[============================================>] 100.0% of 6 kB (6 kB/s)
Downloading tmp-8047-1-SHASUMS256.txt-9.3.3
[============================================>] 100.0% of 6 kB (6 kB/s)
Downloading tmp-8047-1-SHASUMS256.txt-9.3.3
[============================================>] 100.0% of 6 kB (6 kB/s)
(Fri, 20 Nov 2020 18:13:00 GMT) info: Downloaded ffmpeg shared library { version: "9.3.3", dist: "/home/travis/build/eclipse-theia/theia/node_modules/electron/dist" }.
(Fri, 20 Nov 2020 18:13:00 GMT) info: Successfully replaced "/home/travis/build/eclipse-theia/theia/node_modules/electron/dist/libffmpeg.so".
(Fri, 20 Nov 2020 18:13:00 GMT) info: "/home/travis/build/eclipse-theia/theia/node_modules/electron/dist/libffmpeg.so" does not contain proprietary codecs (16 found).
$ yarn prepare:travis && yarn prepare:references && yarn prepare:build && yarn prepare:hoisting && yarn download:plugins
yarn run v1.13.0
$ node scripts/prepare-travis.js
yarn run v1.13.0
$ node scripts/compile-references.js
yarn run v1.13.0
$ yarn build && run lint && run build "@theia/example-*" --stream --parallel
$ tsc -b configs/root-compilation.tsconfig.json
lerna info version 2.11.0
@theia/application-package: $ theiaext lint
[...]
yarn run v1.13.0
$ theia download:plugins
--- downloading plugins ---
+ vscode-builtin-csharp: downloaded successfully
+ vscode-builtin-clojure: downloaded successfully
+ vscode-builtin-coffeescript: downloaded successfully
+ vscode-builtin-configuration-editing: downloaded successfully
+ vscode-builtin-cpp: downloaded successfully
+ vscode-builtin-css: downloaded successfully
+ vscode-builtin-debug-auto-launch: downloaded successfully
+ vscode-builtin-bat: downloaded successfully
[...]
$ yarn prepare
yarn run v1.13.0
$ yarn prepare:travis && yarn prepare:references && yarn prepare:build && yarn prepare:hoisting && yarn download:plugins
$ node scripts/prepare-travis.js
$ node scripts/compile-references.js
$ yarn build && run lint && run build "@theia/example-*" --stream --parallel
$ tsc -b configs/root-compilation.tsconfig.json
lerna info version 2.11.0
@theia/application-package: $ theiaext lint
[...]
$ theia download:plugins
--- downloading plugins ---
- vscode-builtin-bat: already downloaded - skipping
- vscode-builtin-clojure: already downloaded - skipping
- vscode-builtin-coffeescript: already downloaded - skipping
- vscode-builtin-configuration-editing: already downloaded - skipping
- vscode-builtin-cpp: already downloaded - skipping
- vscode-builtin-csharp: already downloaded - skipping
[...]
- vscode-references-view: already downloaded - skipping
install.5
0.02s$ scripts/check_git_status.sh
The support for the THEIA_SKIP_NPM_PREPARE env variable was introduced here, and if I am not mistaken we lost it here.
Summing up: I work on the config to be able to move the workflow to GH Actions, my proposed change won't try to fix the Electron codec thing if that must happen before the tsc build.
kittaakos
on 25 Nov 2020
I think we do not use lerna.js anymore, so we lost https://github.com/eclipse-theia/theia/pull/5239 too. Can someone else please double-check? I have the feeling nobody really knows what does our CI execute 馃槉
kittaakos
on 25 Nov 2020
I have the feeling nobody really knows
Looks like it's just me. Here is the caller:
kittaakos
on 25 Nov 2020
@marechal-p
my proposed change won't try to fix the Electron codec thing if that must happen before the tsc build.
What the status about that aspect? Have we preserved this check?
marcdumais-work
on 26 Nov 2020
Nvm - found the answer on the PR and it looks like we're fine. Nice job guys.
marcdumais-work
on 26 Nov 2020
The only thing remaining to close the issue I believe would be to setup the secrets (subsequent issues can be used to track other features/improvements to the workflow). @benoitf @marcdumais-work who will take care of updating the secrets for the repository?
vince-fugnitto
on 1 Dec 2020
secrets.GITHUB_TOKEN is available by default.
kittaakos
on 1 Dec 2020
@vince-fugnitto I'll take care of it - I have access to the npm account and can generate a new token and set it using @benoitf's suggested way.
marcdumais-work
on 2 Dec 2020
I think I have succeeded at setting the npm token, but we can check if next versions get published now I guess?
marcdumais-work
on 2 Dec 2020
I've restarted the latest job
marcdumais-work
on 2 Dec 2020
Despite the failed status I think it worked:
$ yarn info @theia/plugin-dev | grep "2020-12-02"
modified: '2020-12-02T22:52:39.338Z',
'1.9.0-next.f1e37fbd': '2020-12-02T22:52:35.351Z''
On Travis this type of glitch might have been handled by the retry. In short a small % of packages will take a little while to appear on npm, despite having been published successfully. In the case above it delayed enough for the check script to trip, but by the time I checked manually soon after, the package was visible on npm.
marcdumais-work
on 3 Dec 2020
@vince-fugnitto Good enough to close this issue?
marcdumais-work
on 3 Dec 2020
@vince-fugnitto Good enough to close this issue?
Thank you @kittaakos and @marechal-p for resolving the issue 馃憤
vince-fugnitto
on 3 Dec 2020
Related issues
akosyakov
路
3Comments
pwFoo
路
3Comments
vinokurig
路
3Comments
kittaakos
路
3Comments
akosyakov
路
3Comments
Most helpful comment
Hello,
you don't need to ping webadmins for secrets
all committers can do it
Here an example on how I used it for QUAY.io credentials on some che repositories
https://github.com/benoitf/github-secrets-generator