Nbconvert: --clear-output flag is broken

[spencermathews]

I appreciate the --clear-output convenience flag introduced in #619. However, unless I'm missing something it simply does not work.

These commands should both remove output, but the second does not:

jupyter nbconvert --ClearOutputPreprocessor.enabled=True --inplace example.ipynb
jupyter nbconvert --clear-output example.ipynb

The first form includes the following output (using the --debug option):

[NbConvertApp] Applying preprocessor: TagRemovePreprocessor
[NbConvertApp] Applying preprocessor: ClearOutputPreprocessor
[NbConvertApp] Applying preprocessor: coalesce_streams

However, the ClearOutputProcessor line is missing when using --clear-output.

See this StackOverflow answer for another user who observed the same issue.

Using Anaconda Python 3.6.5 + Jupyter 4.4.0

[t-makaro]

I can confirm that --clear-output is broken, but I can't for the life of me figure out why....

[rawlins]

I have tracked down (at some level) the why, but this is bound up in deep traitlets behavior that I cannot figure out at all, so I'm not sure what the fix is. In traitlets.config.Application.flatten_flags, which is called on flags, various processing happens. One of the things that this function apparently does, is replace all class names of classes that have a single subclass with the name of that subclass. Since ClearOutputProcessor has only one subclass, namely TagRemoveProcessor, ever since that subclass was added, the clear-output flag gets changed from:

{'NbConvertApp': {'use_output_suffix': False, 'export_format': 'notebook'}, 'ExecutePreprocessor': {'enabled': False}, 'FilesWriter': {'build_directory': ''}, 'ClearOutputPreprocessor': {'enabled': True}}

into

{'NbConvertApp': {'use_output_suffix': False, 'export_format': 'notebook'}, 'ExecutePreprocessor': {'enabled': False}, 'FilesWriter': {'build_directory': ''}, 'TagRemovePreprocessor': {'enabled': True}}

This is the crucial traitlets code:

            newflag = {}
            for cls, subdict in flagdict.items():
                children = mro_tree[cls]
                # exactly one descendent, promote flag section
                if len(children) == 1:
                    cls = children[0]

                if cls in newflag:
                    newflag[cls].update(subdict)
                else:
                    newflag[cls] = subdict

I have no clue what this traitlets code is even trying to do, though it really doesn't work as expected for this case. The best workaround I can think of for this specific case would be to not make TagRemovePreprocessor inherit from ClearOutputPreprocessor since it looks like that's mainly done for conceptual reasons; they really only share remove_metadata_fields; or to make them both inherit from an abstract class.

This does seem to be acknowledged as a problem in traitlets in https://github.com/ipython/traitlets/commit/c577973854d663771782fe8dd7db27ad9e83d40e ("The whole flatten_flags method seems like a very awkward workaround to make one priority ordering more important than another. It only 'flattens' one level of inheritance, and it ignores that parent classes may be instantiated.").

[TitouanT]

Any updates on this ? It is still broken on version 5.6.1 of nbconvert

[MSeal]

I will add it to the 6.0 release milestone to double check it's fixed. One of the larger dependency changes needed got done last week so 6.0 can move forward again.

[lindhe]

I think I'm running nbconvert 6.0.7, yet it seems to not work for me...

Any chance that jupyter notebook can run with another version than what jupyter nbconvert --version shows me? How do I check that?

$ jupyter nbconvert --version       
nbconvert        : 6.0.7

EDIT: I tried again, and magically it worked. Let's assume I made a typo or something the first time. Or perhaps it only works if jupyter notebook is stopped first?