Pyro: [FR] Error messages should be more specific

[fritzo]

Users at PROBPROG 2018 requested more-specific error messages when models are invalid.

Specific improvements include:

• [x] #1509 Print sample site information when Distribution inputs are invalid. Currently it requires a debugger to see what line of the model is affected.
• [x] #1534 On error print shapes of sites in the trace: value shape for params, and value,dist,log_prob shapes for sample sites.
• [ ] On error, print contextual information: the current cond_indep_stack, the portion of stack trace under svi.step(), whether the model or guide is being executed, _DIM_ALLOCATOR sate, tracing state, etc. This could be accomplished via a try: except: raise block to add information to a caught error.
• others?

[kamathhrishi]

Can I work on this issue ?

[fritzo]

@kamathhrishi Can I work on this issue ?

Of course, pull requests are always welcome!

[kamathhrishi]

@fritzo I would like to go for this first. What exceptions do we handle under exception blocks? I noticed PEP8 format didn't allow me to create a blank except statement that could allow me to handle any kind of error.

[fritzo]

@kamathhrishi I think the cleanest way might be to add a new context manager to pyro.poutine.runtime.

# in pyro/poutine/runtime.py
class LogRuntimeOnError(object):
    def __init__(self, fn=None):
        self.fn = fn
        if fn is not None:
            functools.update_wrapper(self, fn)
    def __call__(self, *args, **kwargs):
        with self:
            return self.fn(*args, **kwargs)
    def __enter__(self):
        return self
    def __exit__(self, typ, value, traceback):
        if typ is not None:
            # TODO log runtime information here, e.g.
            logging.error('PYRO_STACK:')
            for handler in PYRO_STACK:
                logging.error(str(handler))
            # ...log other things...

(you could instead use contextlib2.contextmanager, but then you'd need to six.reraise, which seems kludgier) Then we can use this debugger in pyro.infer.enum.get_importance_trace

  # in pyro/infer/enum.py
+ @pyro.poutine.runtime.LogRuntimeOnError
  def get_importance_trace(graph_type, ...):
      ...

and probably somewhere else for HMC. @eb8680 does this seem reasonable?

[kamathhrishi]

@fritzo Thank you for the leads. I am unfamiliar with the codebase, is there any way I could learn about it quickly any relevant paper or document?

[fritzo]

@kamathhrishi I recommend starting with the tutorials and reading minipyro.py for an architectural overview. Feel free to put up a draft pull request following my above comment, and we can discuss changes on the PR.