Notebook: Support display of assignment result (for didactic resons)

[cknoll]

Imagine the following situation: somewhere in my notebook full of nontrivial math I want to solve an equation. I want the solution to be stored in a variable _and_ I want to display the solution for better understandability of what is going on.

Until now I write in a cell:

sol = sympy.solve(eqn, var)
sol

This requires the extra line sol.

In a teaching context (especially when not much experience with the used language (e.g. Python) can be assumed) it is often useful to display many intermediate results. However this causes a big amount of 'extra lines' which consume space, typing time and potentially reduce the clarity of the document.
And those lines contradict the programming principle "Don repeat yourself!".

I thus suggest the following feature:

If the line contains a special comment like e.g.#: and an variable assignment, then the result of the assignment is displayed as output of the cell.

The example above would then read:

sol = sympy.solve(eqn, var) #:

but also cause the content of sol to be displayed.

This feature could also be extended to trigger the display of more than the last result from one cell.

sol1 = sympy.solve(eqn1, var) #:
sol2 = sympy.solve(eqn2, var) #:

Obviously the details are up do discussion, but I am convinced that the clarity and thus the "didactic value" of notebooks would benefit.

[Carreau]

the problem is that this change the semantics of the language. The assignment statement does have a value in itself, so it is not obvious what to do. I can see that going into a rabbit hole if you start to deals with value unpacking ellipsis... and so on and so forth. You then get slippery slope and want full macros.

The display part is sufficiently easy, with from IPython.display import display, and it is not that hard to wrap in a function that returns its input:

from IPython.display import display
def pipe_display(obj):
    display(obj)
    return obj
p = pipe_display
a = p(1)
b = p(2)
c = p(a+b)

From the computation point of view, p is not there so you can do a lot of things not doable on your proposal :

p(3)+p(p(2)*p(7))

If pis ugly, you can define __or__ either on P, or your objects:

class P:

    def __or__(self, other):
        display(other)
        return other
p = P()

class MyObj:

    def __init__(self,value):
        self.value = value

    def __repr__(self):
        return str(self.value)

    def __or__(self, other):
        return other(self)

MyObj(1)|p
MyObj(2)|p
p|MyObj('hey')

Which is a less clutterd syntax in some sens.

[cknoll]

The piping in combination with overloading __or__ is a very nice trick. However x = a + b |p does not work on sympy expressions nor on numpy arrays, which are the two most important data types for my purpose. On the other hand x = p | a + b works but seems not preferable because it might confuse the reader to much.

As I understand IPython already does a lot of 'magic' behind the scene which sometimes change the semantics of python, e.g. %autocall, ? and ??. From my point of view this stuff is extremly useful. A special comment to pipe an assignment through display would only be another piece in that puzzle.

W.r.t to the rabbit hole argument I suggest the following:

1. Define an IPython cell-magic like %% display_pipe_comment. Which activates the respective mode.
2. Implement some java script hook which looks for lines ending on #: and containing a _single_ assignment. For such lines the right hand side of the assignment is passed through the display_pipe function defined above.
   For any other line, the comment has no effect.

IMHO that should avoid complicated cases like a, b = ab = range(2) #: while it serves for all cases where the systax the should be as clear and transparent as possible.

Of course, #: was just a quick suggestion, maybe there is a more appropriate choice.

[Carreau]

Well doing this with cell magics would be doable, but really complex. the only reliable way of doing it would be to have a way to get the AST with the comments, which is tough. I don't want to start re-parsing python. The %/%% syntax is already tough with comments and have some edge cases. Doing it in js is not more easy, and raise the question of language agnosticity.

Maybe @takluyver see a way of doing it through AST transform ?

[takluyver]

It would be easy to do an AST transform that would always display the value of an assignment as the last statement in a cell. But I think that would display many things you didn't want to show. Having a marker of some kind to for assignments that should be displayed is trickier, especially if you want to do it with more than just the last statement in a cell.

I suspect that it's better all round to explicitly display the things you want to be shown. This is especially true if you're showing multiple outputs from one cell, because you may well want to label them in some way (print('step 1:', ...)). I don't think we want to get further into redesigning how the language works.

[ellisonbg]

This is actually what Mathematica does so there is a bit of precedence. But
I don't think we would want to do that by default. And yeah, handling
multiple assignments gets a bit tricky...

On Sat, May 16, 2015 at 2:45 PM, Thomas Kluyver [email protected]
wrote:

It would be easy to do an AST transform that would always display the
value of an assignment as the last statement in a cell. But I think that
would display many things you didn't want to show. Having a marker of some
kind to for assignments that should be displayed is trickier, especially if
you want to do it with more than just the last statement in a cell.

I suspect that it's better all round to explicitly display the things you
want to be shown. This is especially true if you're showing multiple
outputs from one cell, because you may well want to label them in some way (print('step
1:', ...)). I don't think we want to get further into redesigning how the
language works.

—
Reply to this email directly or view it on GitHub
https://github.com/jupyter/notebook/issues/93#issuecomment-102700845.

Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
[email protected] and [email protected]

[takluyver]

If this is particularly relevant for symbolic mathematics, sympy might be interested in doing it. They already have some AST transforms for things like making undefined variables act as symbols, IIRC.

[cknoll]

While symbolic mathematics constitutes a big part of what we are doing, numerical math is almost as important. And I think it would also be helpful to understand 'general (python) code' if some important intermediate results are displayed. Thus, a sympy specific solution IMHO would be suboptimal.

I dont know Mathematica but Matlab (mainly concerned with numerical calculation) has the following behavior:

• Line ended without ; -> result displayed
• Line ended with ; -> result not displayed

And Maxima:

• Line ended with ; -> result displayed
• Line ended with $ -> result not displayed

So whats about the optional semicolon as python line ending: Is it available in the AST?

And whats about the inspect module? At least it has a function inspect.getcomments(object).

I tried to find out where an how features like %%autocall and ? at the line end are implemented. I guess its on js level but could not find the respective source. In my understanding the question of language agnosticity also arises in these cases, but has already been answered appropriately, I guess.

[takluyver]

We already have the ; to suppress output like Matlab, but it only works if Python would have displayed output otherwise. Semicolons are not part of the AST, so we have some rather awkward logic to check for them - and that can't really be reused for what you want, because it only checks after Python tries to display a result.

I think we should set quite a high bar for any new special syntax we come up with - the code to deal with it is always tricky, and it risks confusing people about what Python syntax is (we turned off autocall by default for this reason).

[cknoll]

+1 for turning off special syntax by default.

But still, I think my proposal would be useful in teaching context.

From the discussion so far I see the following possibility:

takluyver:

It would be easy to do an AST transform that would always display the value of an assignment as the last statement in a cell.

takluyver:

We already have the ; to suppress output like Matlab, but it only works if Python would have displayed output otherwise.

So, assuming the above mentioned AST-Transform is implemented and _explicitly_ activated, one could turn off the display for assignments where it is not wanted.

In fact, I would prefer it the other way around (turn display on only where it is desired) but this seems to imply the inversion of the ; detection.

To get an impression on my own: where is the logic for checking for semicolons implemented?

[takluyver]

Oh, if you want to explicitly activate it and do it for the last output of all cells, you can do it as an extension, with no need for changes in core IPython.

• Docs on writing extensions
• Docs on AST transformers
• Add an AST transformer to the list ip.ast_transformers when your extension is loaded.

The code checking for semicolons is here, though that uses a completely different mechanism, because semicolons aren't part of the AST.

[Carreau]

Closing as this should go into an extension if needed.

[cknoll]

+1 for closing this issue.
Just for the record: I created an extension which serves my needs. Thanks for guiding me there. Maybe, it is helpful for others:

https://github.com/cknoll/displaytools

[ihnorton]

Since I got here from google, x-ref where this was eventually implemented (6.2+ only): https://github.com/ipython/ipython/pull/10598

[chbrown]

Adding to ihnorton's SEO :), you can configure IPython at the command line to turn on this interactivity:

ipython --InteractiveShell.ast_node_interactivity=last_expr_or_assign

Unfortunately, it appears that the jupyter notebook ... command doesn't recognize the same syntax, but you can have Jupyter pick up this setting by adding a line to your ~/.ipython/profile_default/ipython_config.py (which, at least by default, Jupyter reads from):

c.InteractiveShell.ast_node_interactivity = 'last_expr_or_assign'

[Carreau]

Or just you the %config magic to change it dynamically.

Unfortunately, it appears that the jupyter notebook ... command doesn't recognize the same syntax

It does recognize the syntax, except it's an IPython configuration option, not a Jupyter configuration option.

~/.ipython/profile_default/ipython_config.py (which, at least by default, Jupyter reads from):

No : IPython reads from that, not Jupyter ;-)

IPython's "what's new" does explain that :

This option can be toggled at runtime with the %config magic, and will trigger on assignment a = 1, augmented assignment +=, -=, |= … as well as type annotated assignments: a:int = 2

[ned2]

Simple way to get this awesome IPython feature in your Jupyter notebook:

from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "last_expr_or_assign"

[ellisonbg]

Oh, I didn't know that, thanks for sharing that!