Holoviews: Collecting exceptions to improve

[philippjfr]

One of the major annoyances when using HoloViews is poor exception messages. In this issue I would like to collect such cases so we can improve them.

[ea42gh]

Dimensions with periods can cause error messages that are hard to interpret:
I had

df = pd.DataFrame(np.random.randn(100,2), columns=['method1.a', 'method1.b'])
hv.Bivariate(df).redim(*{'method1.a':'x', 'method1.b':'y'})

instead of

df = pd.DataFrame(np.random.randn(100,2), columns=['method1.a', 'method1.b'])
hv.Bivariate(df).redim(**{'method1.a':'x', 'method1.b':'y'})

[ea42gh]

hv.Curve(x,y) yields the error message
kdims must be a Dimension..., not [ ... ]
when the second arg is not a dimension but a list, array, ...
it is likely that the user forgot the parens:
hv.Curve((x,y))

[ea42gh]

http://holoviews.org/reference/containers/matplotlib/HoloMap.html
omitting the kdims from
hv.HoloMap(curve_dict_2D, kdims=['phase', 'frequency'])

results in the message
KeyError: 'Key has to match number of dimensions.'

Since no keys were passed in, a message along the lines of
must specify a kdims list of length <n>, current value is kdims=[<...>]

philippjfr edit: Now raises:

KeyError: 'The data contains keys of length 2, but the kdims only declare 1 dimensions. Ensure that the number of kdims match the length of the keys in your data.'

[ea42gh]

truth_dmap   = hv.DynamicMap( truth_plot, streams = [s])
error_dmap   = hv.DynamicMap( lambda x: error_plot  (x, typ=err_str), streams=[s])

resulted in the error:
"Callable '' missing keywords to accept stream parameters: data"
which had me puzzled until I realized I had to keep the calling arguments of the functions
the same. Maybe a message along the line: you used argument 'x', did you mean 'data' ...

[philippjfr]

Thanks for filing all these, please keep going, this is all really helpful!

[ea42gh]

DynamicMap cannot be displayed without explicit indexing as 'i' dimension(s) are unbounded. 
Set dimensions bounds with the DynamicMap redim.range or redim.values methods.

I was quite puzzled since I had called redim.values.
It turned out that the call was passed an empty array as its argument.
I am not convinced about the desirability to check for such an eventuality in redim.values() though...

[philippjfr]

The .to method does not allow passing tuples as dimensions, but when you try:

hv_dt = hv.Dataset(df_ig, kdims=[('dt', 'Date'), ('group', "my group")])
hv_dt.to(hv.Curve, 'dt', vdims=[('value1', "value 1 (s)")])

you get an uninformative error message:

TypeError: not all arguments converted during string formatting

[ea42gh]

foo=hv.Dataset(df,kdims=[('err','Error'),('n','Test Number')])
foo.to( hv.Distribution, 'err')

can also result in
LinAlgError: singular matrix
one such case I ran into had but a single row in df

[drs251]

How about this one:

<ipython-input-13-1263cf8cb769> in import_image(filename, px_per_unit, unit, label)
 23 
 24     #dataset = hv.Dataset((y_coords, x_coords, image_array), [x_dim, y_dim, intensity_dim], label)
---> 25     dataset = hv.Dataset((x_coords, y_coords, image_array.T), ["x", "y", "intensity"], "Interferogram")
 26 
 27     return dataset

~/anaconda/lib/python3.6/site-packages/holoviews/core/data/__init__.py in __init__(self, data, kdims, vdims, **kwargs)
188 
189         initialized = Interface.initialize(type(self), data, kdims, vdims,
--> 190                                            datatype=kwargs.get('datatype'))
191         (data, self.interface, dims, extra_kws) = initialized
192         validate_vdims = kwargs.pop('_validate_vdims', True)

~/anaconda/lib/python3.6/site-packages/holoviews/core/data/interface.py in initialize(cls, eltype, data, kdims, vdims, datatype)
204                                   % (intfc.__name__, e))
205                 error = ' '.join([error, priority_error])
--> 206             raise DataError(error)
207 
208         return data, interface, dims, extra_kws

DataError: None of the available storage backends were able to support the supplied data format.`

I would say it could be a bit more specific about what it does not like about the data...

[ea42gh]

I have a complicated plot for a DynamicMap that currently(*) produces the assertion error
DynamicMap must only contain one type of object, not both Overlay and Curve.
( * ) it did not use to with hv 1.9.1
Is there a way to enhance the message to allow tracing back to which plot elements
trigger the assertion error?

[ea42gh]

%%opts VLine (color='indian_red')
results in a javascript error message:
Error: attempted to retrieve property value for property without value specification
(indian_red is not a valid colorspec)
Could this be caught earlier with a better message?

[jlstevens]

That latter one is generated by Bokeh though I am surprised there isn't a corresponding Python extension for us to display...maybe there is and it is being suppressed?

[drs251]

Here's an instance where an exception is missing altogether: if one tries to make an Overlay with only one element like this:

hv.Overlay(items=hv.Image(np.random.random((200, 200))))

instead of

hv.Overlay(items=[hv.Image(np.random.random((200, 200)))])

the result is an infinite loop. The same happens for Layout.

philippjfr edit: Now raises an error.

[drs251]

And in a similar case, when I try to make a Layout or Overlay which effectively contains only one element like this:

hv.Image(np.random.random((200,200))) * hv.Empty

I get this:

TypeError                                 Traceback (most recent call last)
~/anaconda3/lib/python3.6/site-packages/IPython/core/formatters.py in __call__(self, obj)
    330                 pass
    331             else:
--> 332                 return printer(obj)
    333             # Finally look for special method names
    334             method = get_real_method(obj, self.print_method)

~/anaconda3/lib/python3.6/site-packages/holoviews/ipython/display_hooks.py in pprint_display(obj)
    257     if not ip.display_formatter.formatters['text/plain'].pprint:
    258         return None
--> 259     return display(obj, raw=True)
    260 
    261 

~/anaconda3/lib/python3.6/site-packages/holoviews/ipython/display_hooks.py in display(obj, raw, **kwargs)
    236             html = grid_display(obj)
    237     elif isinstance(obj, (CompositeOverlay, ViewableElement)):
--> 238         with option_state(obj):
    239             html = element_display(obj)
    240     elif isinstance(obj, (Layout, NdLayout, AdjointLayout)):

~/anaconda3/lib/python3.6/contextlib.py in __enter__(self)
     79     def __enter__(self):
     80         try:
---> 81             return next(self.gen)
     82         except StopIteration:
     83             raise RuntimeError("generator didn't yield") from None

~/anaconda3/lib/python3.6/site-packages/holoviews/ipython/display_hooks.py in option_state(element)
    110 @contextmanager
    111 def option_state(element):
--> 112     optstate = dynamic_optstate(element)
    113     raised_exception = False
    114     try:

~/anaconda3/lib/python3.6/site-packages/holoviews/ipython/display_hooks.py in dynamic_optstate(element, state)
    104     # Temporary fix to avoid issues with DynamicMap traversal
    105     DynamicMap._deep_indexable = False
--> 106     optstate = StoreOptions.state(element,state=state)
    107     DynamicMap._deep_indexable = True
    108     return optstate

~/anaconda3/lib/python3.6/site-packages/holoviews/core/options.py in state(cls, obj, state)
   1536         """
   1537         if state is None:
-> 1538             ids = cls.capture_ids(obj)
   1539             original_custom_keys = set(Store.custom_options().keys())
   1540             return (ids, original_custom_keys)

~/anaconda3/lib/python3.6/site-packages/holoviews/core/options.py in capture_ids(cls, obj)
   1348         restored using the restore_ids.
   1349         """
-> 1350         return obj.traverse(lambda o: getattr(o, 'id'))
   1351 
   1352     @classmethod

~/anaconda3/lib/python3.6/site-packages/holoviews/core/dimension.py in traverse(self, fn, specs, full_breadth)
    646                 if el is None:
    647                     continue
--> 648                 accumulator += el.traverse(fn, specs, full_breadth)
    649                 if not full_breadth: break
    650         return accumulator

~/anaconda3/lib/python3.6/site-packages/holoviews/core/dimension.py in traverse(self, fn, specs, full_breadth)
    635         matches = specs is None
    636         if not matches:
--> 637             for spec in specs:
    638                 matches = self.matches(spec)
    639                 if matches: break

TypeError: 'bool' object is not iterable

[jlstevens]

I think hv.Empty was designed for + and not *. We should guard against * so that is another useful example, thanks!

[drs251]

It actually does not matter if it's * or +, the error message is the same - so perhaps something broke?

[philippjfr]

I think it's because you need to make an instance of Empty like this. Definitely still a case for better error handling though:

hv.Image(np.random.random((200,200))) + hv.Empty()

but the error for:

hv.Image(np.random.random((200,200))) * hv.Empty()

is just bizarre:

  File "<string>", line unknown
SyntaxError: Failed to parse remainder of string: ' _'


Out[8]:
:Overlay
   .Image.I :Image   [x,y]   (z)
   .Empty.I :Empty

[philippjfr]

Addressed a number of these, bumping the remainder to 1.11.

[ea42gh]

switching from the bokeh to the matplotlib backend typically forces changes to the options.
E.g., one might see

WARNING:root:main: Option 'line_width' for VLine type not valid for selected backend ('matplotlib'). Option only applies to following backends: ['bokeh']
WARNING:root:main: Option 'width' for Scatter type not valid for selected backend ('matplotlib'). Option only applies to following backends: ['bokeh']
WARNING:root:main: Option 'size' for Scatter type not valid for selected backend ('matplotlib'). Option only applies to following backends: ['bokeh']

It might be useful to suggest the alternate option name where feasible.
My current fast route is to introduce a misspelling, (e.g. line_width -> xline_width)
and use the resulting 'similar options are' error message

[ea42gh]

this one left me puzzled:

Unexpected option 'legend_position' for Layout types across all extensions. No similar options found.

I had attached options(legend_position=..) to the wrong element

[jlstevens]

Unexpected option 'legend_position' for Layout types across all extensions. No similar options found.

I agree it might be puzzling but apparently there are no other similar looking keywords - as far as I can tell it is correct. What would you suggest? This would be the same message for any unique looking invalid keyword for any option on any element...

[ea42gh]

what does across all extensions mean? Could you add which type of element the layout_option was applied to? What triggered it is that I added the option after one too many parentheses in a statement composing a lot of elements...

[philippjfr]

I've changed the message slightly in https://github.com/ioam/holoviews/pull/2354, once that's merged it will say something like:

Unexpected plot option 'legend_position' to Layout for loaded extensions matplotlib and bokeh. No similar options found.

[ea42gh]

calling hv.Image resulted in
Supplied cmap grays not found among matplotlib, bokeh, or colorcet colormaps
maybe add a reminder about how to list existing colormaps (possibly an http link?)

[ea42gh]

a call to a DynamicMap using numpy arrays yielded:

DataError: xarray Dataset must define coordinates for all defined kdims, [Dimension('x'), Dimension('y')] coordinates not found.

XArrayInterface expects gridded data, ...

The message is confusing since xarray was not used.

[ahuang11]

I was trying to plot two holomaps, but because their times were off by like a microsecond, it refused to plot into a layout so I had to first round it.

[ahuang11]

~/miniconda3/lib/python3.6/site-packages/holoviews/core/spaces.py in __mul__(self, other, reverse)
    258                 super_keys = self._dimension_keys()
    259             else: # neither is superset
--> 260                 raise Exception('One set of keys needs to be a strict subset of the other.')
    261 
    262             if isinstance(self, DynamicMap) or isinstance(other, DynamicMap):

Exception: One set of keys needs to be a strict subset of the other.

^ with this, would it be possible to output a set(keys).symmetric_difference(set(keys))?

Nevermind this error was completely unrelated to believing that the dimension values differed; it was because I forgot to name one of the HoloMap dimension label!

[ea42gh]

/usr/local/lib/python3.6/dist-packages/holoviews/core/util.py in bound_range(vals, density, time_unit)
   1791             density = full_precision_density
   1792     if density == 0:
-> 1793         raise ValueError('Could not determine Image density, ensure it has a non-zero range.')
   1794     halfd = 0.5/density
   1795     if isinstance(low, datetime_types):

ValueError: Could not determine Image density, ensure it has a non-zero range.

Image density conveys no information to the user:

what triggers this message? Empty data set? Data value range == 0?
If that were stated, I the user would have an idea of what to do.
I got the above message from a DynamicMap, leaving me quite puzzled