Bull: SyntaxError: Unexpected token u in JSON at position 0

[Unibozu]

Hi,

Bull version: 3.3.1
Node version: 8.1.4
Redis version: 3.2.10

I've been working on implementing a small job runner abstraction on top of Bull, and this error is visible whenever I start a worker in debugging (NODE_DEBUG=bull). As far as I can see, this doesn't actually affect basic job processing (I don't use repeating jobs, and I feel like these may be affected). The providers are able to receive the job return value/error when it's completed.

It's a duplicate of #482 (which was closed as possibly fixed in v3).

Error stack (simplified)

SyntaxError: Unexpected token u in JSON at position 0
       at JSON.parse (<anonymous>)
       at Object.tryCatch (/node_modules/bull/lib/utils.js:6:15)
       at getReturnValue (/node_modules/bull/lib/job.js:497:21)
       at Function.Job.fromJSON (/node_modules/bull/lib/job.js:479:21)
       at /node_modules/bull/lib/queue.js:936:23
       at tryCatcher (/node_modules/ioredis/node_modules/bluebird/js/release/util.js:16:23)
 ```

After some digging, I think I may have found the underlying reason.

The root cause is a non-existing property used on https://github.com/OptimalBits/bull/blob/master/lib/job.js?utf8=%E2%9C%93#L479
```js
  job.returnvalue = getReturnValue(json.returnvalue);

As you can guess, if json.returnvalue doesn't exist, the expression returns undefined, which is coerced to a string "undefined" further down the file (line 496 - getReturnValue) as it goes through JSON.parse. This isn't a valid JSON which prompts an error only visible when debugging is activated as it is a soft error.

Apparently, the reason this is happening is that jobs are originally created in Redis without the hashkey returnvalue. When they are processed from pending to an active state (), the job's properties are converted from JSON to scalar values.

This is something I tracked down to Queue.getNextJob (https://github.com/OptimalBits/bull/blob/master/lib/queue.js?utf8=%E2%9C%93#L907) and, more precisely this line https://github.com/OptimalBits/bull/blob/master/lib/queue.js?utf8=%E2%9C%93#L935. The redis lua scripts show that the hash key returnvalue is only set once the job is successfully completed.

My very limited knowledge of Bull tells me you may want to consider skiping getReturnValue when a job returnvalue hash key doesn't exist, which seems like a plausible behaviour. Or eventually you could consider parsing this key based on the job status.

Hope this helps

[Mattchewone]

I too am suffering from this issue.

[StevePavlin]

Same issue here. @manast can you confirm this issue? I can create a PR.

[manast]

I will work on this today

[manast]

I submitted a PR, it solves most of the errors of this kind, however, there is still some cases when running the unit tests where I get this error:

corrupted returnvalue:  { value:
   SyntaxError: Unexpected end of JSON input
       at JSON.parse (<anonymous>)
       at Object.tryCatch (/Users/manuel/dev/bull/lib/utils.js:6:15)
       at getReturnValue (/Users/manuel/dev/bull/lib/job.js:502:21)
       at Function.Job.fromJSON (/Users/manuel/dev/bull/lib/job.js:483:23)
       at /Users/manuel/dev/bull/lib/job.js:90:48
...

The interesting thing is that the return value is {, how that ended as return value is a mystery right now. I will need to continue debugging tomorrow.

[manast]

ok, whatever I fixed it as well. New bull release tomorrow :).

[Mattchewone]

Great stuff thanks @manast

[Unibozu]

Thanks @manast

[imanju0805]

I submitted a PR, it solves most of the errors of this kind, however, there is still some cases when running the unit tests where I get this error:

corrupted returnvalue:  { value:
   SyntaxError: Unexpected end of JSON input
       at JSON.parse (<anonymous>)
       at Object.tryCatch (/Users/manuel/dev/bull/lib/utils.js:6:15)
       at getReturnValue (/Users/manuel/dev/bull/lib/job.js:502:21)
       at Function.Job.fromJSON (/Users/manuel/dev/bull/lib/job.js:483:23)
       at /Users/manuel/dev/bull/lib/job.js:90:48
...

The interesting thing is that the return value is {, how that ended as return value is a mystery right now. I will need to continue debugging tomorrow.

i'm also facing same issue .how can i solve