Theia: [CI] isOutdated test failing intermittently. 10s timeout too short?

Created on 31 Jul 2020  路  2Comments  路  Source: eclipse-theia/theia

Bug Description:

I've noticed that post-PR-merge CI fails to successfully run the unit tests a few times recently due to an early failure in the unit test suite.

The failing test is from @theia/application-package and is named IsOutdated. It timesout sometimes on Travis, once the 10000ms timer is expired.

This test is run 3 times with different inputs - i've seen 2 or 3 fail on different runs.

For example here 2/3 failed: (https://travis-ci.com/github/eclipse-theia/theia/jobs/366717909#L3896-L3914)

theia/application-package:       2) @theia/[email protected]
@theia/application-package:   4 passing (22s)
@theia/application-package:   2 failing
@theia/application-package:   1) extension-package
@theia/application-package:        isOutdated
@theia/application-package:          @theia/core@next:
@theia/application-package:      Error: Timeout of 10000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/travis/build/eclipse-theia/theia/dev-packages/application-package/lib/extension-package.spec.js)
@theia/application-package:       at listOnTimeout (internal/timers.js:549:17)
@theia/application-package:       at processTimers (internal/timers.js:492:7)
@theia/application-package:   2) extension-package
@theia/application-package:        isOutdated
@theia/application-package:          @theia/[email protected]:
@theia/application-package:      Error: Timeout of 10000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/travis/build/eclipse-theia/theia/dev-packages/application-package/lib/extension-package.spec.js)
@theia/application-package:       at listOnTimeout (internal/timers.js:549:17)
@theia/application-package:       at processTimers (internal/timers.js:492:7)
@theia/application-package: Exit with failure status (2): nyc mocha --opts ../../configs/mocha.opts "./lib/**/*.*spec.js"

The failing test checks npm packages versions to determine if a package is outdated vs a proposed versiob. When I run locally, each of the testOutdated waits about 4 seconds for npm to respond - with the timeout of 10s, it runs ok. But on Travis is could be slower, such that 10s timeout is cutting it a bit too close.

Steps to Reproduce:

Unfortunately it seems left to (bad) luck. I suggest watching CI for newly merged PR for failures, and checking if it's this one. Note that the test suite goes on to do the integration tests ,so one may need to scroll-up quite a bit to reach the beginning of the log for unit tests.


Additional Information

  • Operating System:
  • Theia Version:
ci

Most helpful comment

We should wipe those tests; they're unused. ExtensionPackage#isOutdated is called only from the tests; we should remove isOutdated too.

All 2 comments

We should wipe those tests; they're unused. ExtensionPackage#isOutdated is called only from the tests; we should remove isOutdated too.

They also depend on npm, maybe there were connectivity issues with the registry again.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

akosyakov picture akosyakov  路  3Comments

tetchel picture tetchel  路  3Comments

vinokurig picture vinokurig  路  3Comments

vince-fugnitto picture vince-fugnitto  路  3Comments

jankeromnes picture jankeromnes  路  3Comments