Sentry-php: Array serialization truncation too aggressive

Created on 5 Jul 2016 · 18Comments · Source: getsentry/sentry-php

All arrays are currently being truncated to "Array of length N":

https://github.com/getsentry/sentry-php/blob/c8bc22e148bd7ad25bcb29d70182318c7b493b18/lib/Raven/ReprSerializer.php#L35

... and yet dictionaries are fully expanded. We are getting feedback from users with error data containing small arrays (e.g. 1-5 items) that are frustrated that they cannot see inside.

Source

benvinegar

👍14

Most helpful comment

Thanks @Jean85
Unfortunately setting the serializer was not useful in my case because the depth is set in Client::sanitize and wanted to set a greater depth for user data only.

I end up by ovewritting sanitize method to do the job. Here is an example in case it can help anyone:

class MyRavenClient extends \Raven_Client
{

    /**
     * @param array $data
     */
    public function sanitize(&$data)
    {
        // Serializes the session items as a json string
        foreach ($data['user']['data'] as $k => $v) {
            if (is_array($v)) {
                foreach ($v as $kk => $vv) {
                    $data['user']['data'][$k][$kk] = json_encode($vv, JSON_PRETTY_PRINT, 3);
                }
            }
        }


        // use parent sanitizer
        parent::sanitize($data);
    }

}

Be aware that serializer will limit the size of a string, and in this case it might be useful to provide an other serializer that has a greater limit on string size.

gsouf on 16 Nov 2017

👍3

All 18 comments

Yes, frustrating!

lupoair on 1 Aug 2016

I'm closing this out as we expand arrays in PHP.

dcramer on 1 Aug 2016

This issue still exists. Will it be fixed?

idoberko2 on 21 Aug 2017

Just ton confirm, where are these arrays not getting expanded? (which section of the event)

dcramer on 21 Aug 2017

Additional Data. For example, we receive JSON webhooks from 3rd parties and send events to sentry in case of failure. Attached an example of how one of these JSONs look in Sentry:

idoberko2 on 22 Aug 2017

We should anyway limit the depth or let it be configurable, to avoid sending data too "deep".

Jean85 on 22 Aug 2017

I'm trying to recall why we went this direction in the first place. I think there were certain things where we didn't want the full data repr, but honestly dont recall where.

dcramer on 22 Aug 2017

In my case, this additional data are required to make possible fix my issue, once that the problem is inside of this truncated array.

rentalhost on 25 Aug 2017

According to support this feature is intended to help keep error messages within the 100kb payload limit.

Maybe it would be possible to only flatten when the limit is hit, or make it opt-out? Those who are migrating from another solution (like us) are probably already having their error messages somewhat in check.

pontus-mp on 13 Nov 2017

In the 1.8.1 release that just got out, we added setters for the serializers. You can replace them with the logic you prefer.

Jean85 on 13 Nov 2017

Hi @Jean85 can you give details about this method?

In my case the data truncated from $_SESSION are required to reproduce the error. To workarround the depth limit, I currently send a json encoded copy of $_SESSION (with pretty print) in extra param.

gsouf on 16 Nov 2017

I'm referencing this piece of code: https://github.com/getsentry/sentry-php/blob/e1001f80d07f08cbf8d2648cb707ea43bf3e5794/lib/Raven/Client.php#L1457-L1471

Those 2 serializers are the classes responsible for the compression of those arrays. You can extend them and change the behavior for the desired stuff. But using the extra field like that is a feasible solution too!

Jean85 on 16 Nov 2017

👍1

Thanks @Jean85
Unfortunately setting the serializer was not useful in my case because the depth is set in Client::sanitize and wanted to set a greater depth for user data only.

I end up by ovewritting sanitize method to do the job. Here is an example in case it can help anyone:

class MyRavenClient extends \Raven_Client
{

    /**
     * @param array $data
     */
    public function sanitize(&$data)
    {
        // Serializes the session items as a json string
        foreach ($data['user']['data'] as $k => $v) {
            if (is_array($v)) {
                foreach ($v as $kk => $vv) {
                    $data['user']['data'][$k][$kk] = json_encode($vv, JSON_PRETTY_PRINT, 3);
                }
            }
        }


        // use parent sanitizer
        parent::sanitize($data);
    }

}

Be aware that serializer will limit the size of a string, and in this case it might be useful to provide an other serializer that has a greater limit on string size.

gsouf on 16 Nov 2017

👍3

Also having this:

This relates to issues created via overriden Monolog's RavenHandler in Yii2 app (don't ask....).

I'll probably normalize trace to flat array, but still, this is frustrating.

Wirone on 19 Oct 2018

Any movement on this? It would be great if serialization depth was configurable for each of the types of data this client attaches to Sentry events (user, extra, tags...).