Spectrum: Get sentry reports for failed worker jobs

[brianlovin]

We don't appear to be getting failure reports from workers like Hermes (470 failures in last 24hrs) - make sure these are reported to Sentry

[dev-drprasad]

@brianlovin I have not worked with sentry. but i am fine to look into it. If it is beginner friendly issue, i can take it

[brianlovin]

@dev-drprasad this may be beginner friendly, although I could be mistaken. Looking through some of the comments here: https://github.com/OptimalBits/bull/issues/8 - it looks like we just want an .on('failure'... event handler in the queue construction (probably in shared/bull dir - let me know if you come across anything!

[brianlovin]

We also report our errors to sentry, so if you just do a project search for Raven.captureException(... you'll see some examples here :)

[dev-drprasad]

@brianlovin we have already handling stalled event in queue. but both failure, stalled events emit when job fails. but i am unable to understand exact difference failure and stalled. you have any idea ??

I am thinking of capturing failure events also. But how do i simulate errors and check whether they are reported in sentry ??

[brianlovin]

@dev-drprasad - from the docs, I'm seeing:

.on('stalled', function(job){
  // A job has been marked as stalled. This is useful for debugging job
  // workers that crash or pause the event loop.
})

.on('failed', function(job, err){
  // A job failed with reason `err`!
})

So I think in this case we'd likely want sentry reporting for both.

In terms of testing: we recently switched our Sentry keys to be private, as we were getting a lot of sentry reports from people running their own fork locally. Right now I think the best thing will be to console log and debug locally to check for error events, and I will own the deployment and checks that errors are getting to our dashboard :)

[dev-drprasad]

@brianlovin Sorry to bother, how can i trigger some jobs related to hermes and make them fail ?

[brianlovin]

In a separate console tab yarn run dev:hermes and a separate tab yarn run dev:athena - then just create a thread somewhere and you should see athena running notifications jobs, which then pass jobs into hermes. From the hermes/send-new-thread-email.js file you can just manually throw an error, I think :)

[dev-drprasad]

@brianlovin I am unable to install dependencies. i am getting error after latest pull

[4/4] 📃  Building fresh packages...
error An unexpected error occurred: "/Users/wizard/Projects/spectrum/mobile/node_modules/detox: Command failed.
Exit code: 1
Command: sh
Arguments: -c scripts/postinstall.sh
Directory: /Users/wizard/Projects/spectrum/mobile/node_modules/detox
Output:
xcode-select: error: tool 'xcodebuild' requires Xcode, but active developer directory '/Library/Developer/CommandLineTools' is a command line tools instance
/Users/wizard/Library/Detox/ios/b70e6e7d1e281aa9791ee312c7e4e33c188ce685 was found, but could not find Detox.framework inside it. This means that the Detox framework build process was interrupted.
         deleting /Users/wizard/Library/Detox/ios/b70e6e7d1e281aa9791ee312c7e4e33c188ce685 and trying to rebuild.
Extracting Detox sources...
Building Detox.framework...".
info If you think this is a bug, please open a bug report with the information provided in "/Users/wizard/Projects/spectrum/mobile/yarn-error.log".

Where can i get help regarding spectrum ?? i don't want to ping you for every small problem :)

[mxstbr]

Can you try manually installing detox in the mobile directory? yarn add detox

[dev-drprasad]

@mxstbr @brianlovin ran yarn add detox in mobile dir , got following logs

warning "jest-expo > [email protected]" has unmet peer dependency "babel-core@^6.0.0 || ^7.0.0-0".
[4/4] 📃  Building fresh packages...
[-/6] ⢀ waiting...
[2/6] ⢀ sentry-cli-binary: fetching sentry-cli-Darwin-x86_64 10%
[-/6] ⢀ waiting...
[-/6] ⢀ waiting...
error An unexpected error occurred: "/Users/wizard/Projects/spectrum/mobile/node_modules/detox: Command failed.
Exit code: 1
Command: sh
Arguments: -c scripts/postinstall.sh
Directory: /Users/wizard/Projects/spectrum/mobile/node_modules/detox
Output:
xcode-select: error: tool 'xcodebuild' requires Xcode, but active developer directory '/Library/Developer/CommandLineTools' is a command line tools instance
Extracting Detox sources...
Building Detox.framework from ./ios_src...".

control returned back to terminal after 2 mins. should i run node shared/install-dependencies.js after this ??

[mxstbr]

Hmm, no that doesn't seem to work. I think you can just ignore the mobile directory and manually run yarn in all the worker directories for now?