Hydra: _pickle.PicklingError: Can't pickle typing.Union[str, NoneType]: it's not the same object as typing.Union

[omry]

Opening a dedicated issue to discuss an error from CloudPickle on 3.6 (spinoff from #421).
Will look into it in more details after #415 lands:
That PR is adding a method for users to define their configuration.
As a part of the PR, I am porting the joblib plugin configuration to the new method and this is where I saw the issue on 3.6.
I am many users to follow this pattern in the next version of Hydra.

In fact I already ran into the same issue while working on OmegaConf with the unit tests for regular Pickle and I worked around it somehow.
I am not sure this is a cloudpickle issue (unclear what is the relation between the standard pickle and cloudpickle).

Context:

Unrelated to the joblib issue on 3.6, I ran into a new issue in a branch I will land now that will give me another reason to block 3.6 for this plugin.
This is probably a week or so from landing. If you start seeing errors about:
_pickle.PicklingError: Can't pickle typing.Union[str, NoneType]: it's not the same object as typing.Union

This might be related to this bug that was just fixed in master by @pierreglaser: cloudpipe/cloudpickle#347

This is about silently dropping type annotations on dynamic classes in Python 3.6. If you really need to pickle type annotations on dynamic functions or classes (without silent drop), then there is this PR but it's quite complex and I don't have time to review it at the moment.

[omry]

@ogrisel opening another issue to keep the dicussion of the joblib segfault focused.
cloudpipe/cloudpickle#348 definitely sound related.

Actually OmegaConf is going to make runtime use of Python annotations, so dropping them is suboptimal.
See the structured configs in the docs for the next version

[ogrisel]

Please let us know if the solution implemented in https://github.com/cloudpipe/cloudpickle/pull/318 fixes the problem for you.

Note that joblib still vendors cloudpickle and loky in the joblib/externals subfolder so you have to copy the files there. You can use the helper script to do so:

pip install -e joblib
cd joblib/externals
bash vendor_cloudpickle.sh /path/to/cloudpicke_checkout_folder_with_the_right_branch

[omry]

My joblib plugin tests are passing with it on 3.6 so looks like it is solving the problem.

[omry]

@ogrisel, Is that fix fully serializing the type information or does it strip some down?

[ogrisel]

I haven't had the time to review cloudpipe/cloudpickle#318 yet but I believe it should preserve all the type info.

[omry]

Issue fixed in cloudpickle.