Sentry-javascript: Directions for offline reporting?

[danse]

Hi, we have a web app with offline capabilities deployed, and it would be great to use Sentry for the application monitoring. We already use it for our back end and it works great!

Unfortunately i don't see support in raven js for this, so i think that we would be forced to roll out our own solution. Can you confirm this? What would be your suggestions about how to proceed? Would you recommend trying to reuse some logic from raven js?

[mattrobenolt]

Can you detect if the user is offline?

If so, it wouldn't be impossible to collect events and queue them up. When the user gets back online, send them through Raven.captureException again.

I can probably help with how that'd work or add a hook in to help this.

[danse]

Great, thanks for the collaboration. I could detect the user being online even just testing Sentry reachability. For this i could listen the transmission failure events, but then how to retry?

About using captureException, that would be fine, but i would like to keep the excellent handler raven provides for unhandled exceptions. I would need to add some middle logic between raven error detection and error transmission

[mattrobenolt]

Yeah, lemme think about this. I think there's a nice way that we can either just support this automatically in raven-js, or provide a hook to do it. I think that's fair.

[lauritzen]

I'm looking for the same funtionality as well. After examining the docs it seems that a work around could be to use shouldSendCallback, something like this:

shouldSendCallback: function(data) {
  localStorage['queued-errors'] = (localStorage['queued-errors'] || []).push(data);
  return false;
}

And then listen for network connectivity and process the queue using Raven.captureException:

window.addEventListener('online', checkAndProcessQueue);

[cappslock]

This is piggybacking a bit, but I have a similar issue. We can't report errors when the client loses connection to the host (though they are not necessarily offline). Trolling around, I see that there's a ravenFailure event being triggered on the window, but I can't get enough data from this event to accomplish what I'm trying to do: retry sending the error on failure.

I see a few possible approaches to this:

1. A per-instance or per-message retry option, which might specify how often to retry sending events, how many times to try before giving up, etc.
2. A callback/callback(s) on the captureException(),sendMessage()`, etc. methods which is/are triggered on success/failure
3. Promisifying those methods (meh)
4. Triggering events (where?) on failure and providing enough context in the event data to attempt a re-send of that event

[webberig]

You should probably have an offline storage facility so the error messages can be sent later in time. Possibly even after the user closes the app and later reopens it while online. Similar to an "offline first" approach:
http://offlinefirst.org/

[oliviertassinari]

I have faced the same issue. At first, I wanted to solve it similarly to how I'm solving the offline google-analytics issue with a service worker. The approach is well explained by Google here.

However, when targeting Cordova, Service Worker might not be available.
And a _hacky_ solution is needed. I have came up with this one:

https://gist.github.com/oliviertassinari/73389727fe58373eef7b63d2d2c5ce5d

import raven from 'raven-js';
import config from 'config';

const SENTRY_DSN = 'https://[email protected]/YYYY';

function sendQueue() {
  const sentryOffline = JSON.parse(window.localStorage.sentryOffline);

  if (sentryOffline.length > 0) {
    raven._send(sentryOffline[0]);
  }
}

// ... 

[cudasteve]

I agree with @webberig. I was actually a little surprised this wasn't already part of raven since it's pretty impressive in a lot of other cool ways. Depending on an app's architecture, there could potentially be a non-trivial percentage of the errors that are the RESULT of being offline. Being able to report on those is pretty critical in a lot of implementations.

So my vote is that this should be baked in and automatic. When there's an error sending the message to sentry, the message should be stored and tried again later (either on a timer, on an event, or even just on the next message transmission if you want to keep it simple).

Also, what's the "DDN" label mean?

[benvinegar]

I was actually a little surprised this wasn't already part of raven since it's pretty impressive in a lot of other cool ways. Depending on an app's architecture, there could potentially be a non-trivial percentage of the errors that are the RESULT of being offline.

Totally fair. But I'm not sure it should be done by default. There are many scenarios where, if a script doesn't load, the app isn't going to work no matter what. And I would say most apps just straight-up don't work offline, and don't expect to. Whether that's good practice or not is another thing.

It's sort of why we don't log AJAX errors by default, even though a good chunk of Raven users have written code to do so. But we'll try to make it easy to do so _if you want_. And I'd like to do the same thing here.

Also, what's the "DDN" label mean?

Not sure. @mattrobenolt?

[mattrobenolt]

Design decision needed. :)

[cudasteve]

I'm not sure it should be done by default.

Ya that makes sense.

But we'll try to make it easy to do so if you want. And I'd like to do the same thing here.

So, in the interest of helping make a design decision, here's some thoughts:

If I'm understanding the docs correctly, the transport config option allows one to basically take control of the last part of the pipeline where Sentry sends the data to the server and become responsible for making that transmission happen. In that case, that seems like the best place to plugin an offline queue. Instead of sending the data to the server, send it to a queue that uploads to the server in the background. That would be if someone were writing their own code to make it happen.

In a similar vein, would it be simple enough to also expose a config option (includeOffline?) that would replace the default HTTP transport with an official Raven-rolled offline-capable queue?

• For browser compatibility, maybe store the queue in localStorage and have it check for connection on a timeout.
• For better performance and stability, maybe leverage service workers to manage the queue so errors could even be uploaded after the page is closed. But is it a little presumptuous for a utility library to be spinning up its own service workers at this point?

[Freundschaft]

has this functionality now been built in somehow to raven?

[cudasteve]

@Freundschaft No, not yet. As I understand it, for now you'll have to override the transport option or use the shouldSendCallback to build a queue for sending later.

[Freundschaft]

@cudasteve thanks!
someone got a working sample implementation? if not I'll try to write one and post it here

[ddddm]

@Freundschaft this would be very helpful. we are facing the same problem

[matsomahoney]

++1 please

[marcelom56]

+1

[peresbruno]

+1

[ddddm]

Our solution:

Pass shouldSendCallback to Raven init

If there is no connection, out logStorageService will save the event's data

var options = {
    ...
    shouldSendCallback: function(data) {
        if (connectionStatus.check() && Raven.isSetup()) {
            return true;
        } else {
            // store log data somewhere
            logStorageService.set(data);
            return false;
        }
    }
    ...
};

Raven.config(SENTRY_KEY, options).install();

Queue trying to send logs

Queue fires this code every 25 seconds, eventually delivering all of the events to Sentry

queue.enqueue(function () {
                    logStorageService
                        .getKeys()
                        .then(function (keys) {
                            if (keys && keys[0]) {
                                logStorageService
                                    .get(keys[0])
                                    .then(function (log) {
                                        Raven.captureMessage('', log);
                                        logStorageService.remove(keys[0]);
                                    });
                            }
                            ...
                        });
                });

[waynebloss]

Some examples above showed a call to a "private" function. However, I am not able to call Raven._sendProcessedPayload or Raven._send.

Looking at your source code, I am not even sure how Raven() is being constructed as an object. I don't see "new Raven()" anywhere.

How can I call these functions?

[waynebloss]

Nevermind, the problem was only happened when raven.min.js and not raven.js so that answers that question.

[waynebloss]

So, I hunted down the minified name for the _sendProcessedPayload function and here's what I ended up with in my code (for now):

  /**
   * HACK: Using a private function that gets minified!
   * Pinned Raven-js to version 3.8.1.
   */
  Raven._sendProcessedPayload = Raven._sendProcessedPayload || Raven.Y;

  ...

  function processQueue(items) {
    // Stop if we're not online.
    if (!canSend())
      return;
    // Process the given items or get them from the queue.
    items = items || queue.getItems();
    if (!items || items.length < 1)
      return;
    // First in, first out.
    var next = items.shift();
    // Send the next item.
    Raven._sendProcessedPayload(next, function processed(error) {
      // If no errors, save the queue and process more items.
      if (!error) {
        queue.save(items);
        processQueue(items);
      }
    });
  }

  function shouldSend(data) {
    if (canSend())
      return true;
    if (data.extra.retry)
      return false;
    data.extra.retry = true;
    queue.add(data);
    return false;
  }

  ...

  setInterval(processQueue, OFFLINE_QUEUE_TIMEOUT);

This is far from ideal due to the horrendous HACK and also because more errors could be captured while I'm processing this queue, so they would be sent out of order...

[waynebloss]

Final note on this for today: I think you should make _sendProcessedPayload public. It picks up right where _send exits when shouldSendCallback returns false, so it's the obvious choice to call if you want to retry a send...

However, I don't like that it modifies the payload. So, it's not the ideal method.

[panbhag]

Has this feature been added to the library yet.

[alexcroox]

I ran into a need for this today as I have an offline capable feature. I store all my own data using indexdb but it would be nice to have an out of the box offline functionality for Sentry. If it it could use it's own local database to store events and periodically/on network changes attempt to send and clear out the db that would be super useful.

[kamilogorek]

Addressed here https://github.com/getsentry/raven-js/pull/1165

[nbrustein]

If I do store exceptions for sending later when I come online, is there a way for me to set the timestamp on the exceptions to indicate that they didn't happen now when I'm sending them to sentry, but at some point in the past? I guess I could just add it in the extra, but it would be nice to be able to set the actual timestamp.

I don't see any such option in captureException