Vyper: Allow an optional "end"

[rocky]

Simple Summary

And an optional block terminator.

Abstract

I was encouraged to learn of Vyper's focus on provability and correctness. A lot of that enthusiasm was damped when I learned it uses a weakness of Python syntax - lack of a block terminator.

Could one be added, even if optionally?

Motivation

Aside from the countless time spent in fixing mistakes that Python programmers face when moving blocks of code around, there have been to other very serious flaws with the lack of having a terminator.

The first situation where this terminator is sorely needed is in printed code. Invariably something gets messed up with spacing. A common situation is that a proportional-space font is used when a monospace font was intended. Since there is no redundancy provided by the block terminator, it can be difficult to impossible to figure out the intended meaning.

The second area is when code is embeded in a templating system. So where as in say Ruby the templaters escaple direclty to the code of the programming langauge, in Python you need to have a _separate_ programming language like jinja2.

Specification

The simplest an least abtrusive way to address is to use "pass" as the block terminator, if "pass" is a statement of vyper. Better though is to use a reserved word, and in these kinds of languages where the opener is not a brace, end with or without the corresponding block name, e.g. endif, endfor is often used.

Backwards Compatibility

Old should programs will work with the new grammar. Programs only familiar with old grammar won't work. However the block end could be added as a meta comment, for this.

Copyright

Copyright and related rights waived via CC0

[Meai]

Especially in a security focused language there really isnt any excuse to allow such an obvious way to let mistakes happen. Just one wrong indentation at the wrong spot and you just changed the entire logic of the program in possibly fatal ways.
I think we should not spare the feelings of python programmers here, this is a security flaw and should not be tolerated.
I personally find how Julialang does it with their 'end' keyword very readable.

[fubuloubu]

So first off, I would not confuse "propensity for making a mistake" as a security error. I think that's a stretch that should not be made.

Secondly, smart contracts cannot typically invoke many nested loops without exceeding gas usage limits, so I think the likelihood of constructing a smart contract with this kind of propensity for error is fairly unlikely, although definitely possible.

Third, the degree of testing that a smart contract should go through in order to be released must be fairly high. If you did not catch a MAJOR logic change with your test suite (probably because you didn't have one) that is on you, not the language designers.

---

Given the above, I think the best way to solve this issue is to make a style guide recommendation to demarcate closures for if and for blocks. If a contract method has complex enough code to potentially encounter this error, it would be the recommendation of the style guide to add a comment denoting the end of that block.

Here are the most complex examples we have in our examples/ directory exemplifying nested logic:

https://github.com/ethereum/vyper/blob/a9021c0d74b0c9a72c59550b020e83cfe2e71475/examples/wallet/wallet.vy#L13-L17

https://github.com/ethereum/vyper/blob/a9021c0d74b0c9a72c59550b020e83cfe2e71475/examples/voting/ballot.vy#L43-L51

https://github.com/ethereum/vyper/blob/a9021c0d74b0c9a72c59550b020e83cfe2e71475/examples/voting/ballot.vy#L140-L147

None of them are complex enough where you cannot visually trace the end of the loop back to the start of it. I think a familiarity with Python is somewhat of a prerequisite for writing Vyper code, so you should be able to read the code pretty well in these cases.

An additional comment in one or two of these may help with readability and understanding, but I don't think an active keyword would do much more to mitgate it vs. a simple comment. In my personal opinion, the level of complexity involved in implementing this request would exceed it's utility, and a simple suggestion in the style guide to aid in readability of the contract (where deemed necessary) is very useful.

---

As a sidenote, I think suggestions like this occur from an understanding of software being written from an iterative perspective. Smart contracts do not lend themselves well to the iterative approach, at least not as frequently as traditional software, as they implement a lot of critical logic in very few lines of code. This requires well-documented and well-tested code for others to read and maintain that code into the future.

This is my personal opinion of course, but I think as smart contracts develop this intuition will be more apparent with time.

[rocky]

With respect to programming language design, here is something old but still I come back to; Larry Wall's Laziness, Impatiece and Hubris. It posits the idea that burden shouild be put on the computer, not the programmer.

[fubuloubu]

If we hear more complaints about problems with ambiguous block closures in practice, we will be sure to take this into account. I haven't heard a single complaint so far personally, I think most python programmers get very used to this very quickly and as stated above most Vyper programs will not require the depth of indentation that a general Python program encounters.

We have so far avoided modifying the python ast directly (we do some renamings, but that does not affect the ast) so my inclination is to avoid opening that pandora's box unless we go all in on a vyper-specific front-end parser that does not leverage the terrific efforts Python devs have gone through to build a robust front-end.

[rocky]

avoided modifying the python ast directly

This has nothing to do with Python's (or Vyper's) AST.

[fubuloubu]

So, we would pre-process and enforce block terminators (if found), before processing with Python ast.parse?

[rocky]

I'm just saying it has nothing to do with any AST.

[jacqueswww]

Note: I am still mulling this over.

From a practical/implementation point of view:

@fubuloubu we would be able to at an end statement check, it would look something like so, if we do go this route I would make it a 'strict' feature?

 def winning_proposal() -> int128: 
     winning_vote_count: int128 = 0 
     winning_proposal: int128 = 0 
     for i in range(2): 
         if self.proposals[i].vote_count > winning_vote_count: 
             winning_vote_count = self.proposals[i].vote_count 
             winning_proposal = i 
         else:
             pass
         endif
     endfor
     return winning_proposal 

I would definitely go for end{if,for} because it reads clearer. Having end is a similar problem, as you don't know which end goes to which statement. Which is the exact same problem you have with using brackets, there are just too many and you don't know which belong where.

The important thing the check would do, is ensure indentations and end{if,for} match, ~with no statements allowed after the end keywords.~

[rocky]

The bad thing about using indentation, is that it becomes ungainly when the language is embedded inside some sort of templating system. Again, think trying to replace jinja2 using Python as a language instead.

Given you are thinking about noting this via some sort of "strict" indicator, perhaps that could also be extended to allow for loose indentation.

(To make it clear that the looseness is there to allow templating or embedding, the option name could be geared more towards that effect: "templated" vs "loose spacing".)

[jacqueswww]

In what scenario would one want to template vyper? Given that the onchain factory mechanics are a lot more useful and secure?

[jacqueswww]

I do think strict indentation is a benefit for readibility, and would not consider removing that from vyper. Removing indentation requirements makes it much easier to write under handed code.

[rocky]

Numerous times I have noticed a tool I have written used in a very different manner than what I had imagined its uses would be. Nevertheless, used in a perfectly valid and reasonable way, if not novel, or game changing.

Here is something analogous. Ruby had followed Perl in its decision to allow parenthesis to be optional in a function call when the parenthesis is not needed to disambiguate. (Perl had extended the POSIX shell convention in that it allowed _optional_ parenthesis where as in POSIX shell you couldn't use parenthesis to enclose command arguments from the command name).

As a result, someone later realized that Ruby would make a good language to implement a DSL in. I doubt though that Matz specifically had that in mind. As a result, for a while Ruby got a big uptick in use.

As a user of a language, I personally would opt for a language that doesn't proscribe what the kinds of things that a the language is to be used for, but instead is built on solid principles, stearing clear of known pitfalls.

We know that indentation causes a problem in templating systems (if not also in printing). Why limit yourself based on whether you or I can currently imagine Vyper's use in one?

Perhaps others may want to offer situations, but I don't want to go down that rabbit hole. I propose something then all of a sudden we are discussing the merit of that and so on.

I do think strict indentation is a benefit for readibility,

I never said it didn't. In fact, I agree it does.

I am just saying that there are some situations where it is inappropriate. So when the intent is clearly indicated, for example, "I am am using vypper here as an embedded language so please skip the indentation check", then what's the harm?

[jacqueswww]

I do concede that, yes in other scenarios letting go of the tighter requirements as some type of opt-in would seem like a nice option, and I also agree that one can not tell how a language will be used in the future. Full disclosure: I am also a big fan of not having too many "strict" toggles at this stage - I think we should try and build the most audit able/readable language by default first. We can not envision how contracts will be written (or if they will be :tongue:) written in the future, and for those use cases we will definitely have to evaluate the trade offs between usability and security on a case by case basis (that's why I was asking about current use cases for templating).

Interestingly enough Vyper is basically a DSL of python3.6, and it's core focus is writing secure and audit-able contracts. It's our primary goal make decision around those core goals, "it should be maximally difficult to write underhanded code" (if I have to make a saying up on the spot hehe). Hence why I also prefer the 'do one thing and do on thing well' approach, because if there is only one way to for example indent or show block logic ending, one will not have to adapt to different styles when reading contracts.

In terms of audit-ability/readability/security of not clearly reading the end of a block is valid concern, and is the one I think we should focus in terms of this VIP, we are having our bi-weekly call on Monday 13th Aug 14:00 UTC, and the VIP will be on the agenda, please feel free to drop by and say hello and participate :) (note anyone reading this: it's an open call, everyone is invited).

[rocky]

If you want to post a link, I may just be able to make it.

not clearly reading the end of a block is valid concern, and is the one I think we should focus in terms of this VIP

It would be awesome if just this aspect and adding and ENDIF/ENDFOR were added. I probably shouldn't volunteer time, but I might be able to adjust the grammar (if things go satisfactorily that way).

I agree that putting of the embedding and printing issues can be as you say put off. Just note though that Python syntax, is problematic here.

[jacqueswww]

Approved as an opt in.