I am playing around with workers with different Python versions. One of the pitfalls is the pickle protocol version used. For python3.4 rq uses the highest available protocol version (4) but python2.7 only understand up to 2. This can be fixed by using a custom Job class and forcing python3.4 to use protocol version 2 but it's rather awkward.
I managed to use a custom Job class (overwriting the dumps method) to overcome this issue but exposing this as a flag might be more convenient.
:+1:
There's an issue for implementing a customizable args/kwargs
serializer. I think that's the better approach for this particular problem :)
Here's a link to the issue I mentioned: https://github.com/nvie/rq/issues/369 . I forgot to attach it when I closed this issue, sorry.
As a side note, while not particularly pretty, this appears to actually work:
import pickle
pickle.HIGHEST_PROTOCOL = 2
from rq import Queue
# from here on RQ will use pickle protocol 2
At least when using python 3.4 to queue the job, and 2.7 to read it.
Most helpful comment
As a side note, while not particularly pretty, this appears to actually work:
At least when using python 3.4 to queue the job, and 2.7 to read it.