Solidity: LLL: plans for the future

[axic]

Currently LLL is in a limbo. It is part of the Solidity repo and there are a lot of uncertain aspects:

• Is it part of the Solidity release?
• If not, where and when is it released?
• If yes, are breaking changes aligned with the Solidity release cycle?
• Are LLL changes part of the Solidity changelog?
• Where to put LLL documentation?
• Where to put LLL tests? (Parser, semantics and end-to-end tests.)
• Shouldn't there be an Emscripten interface to lllc? (Similarly to Solidity.)

Currently LLL depends on libdevcore and libevmasm. Splitting it out would be most easily achieved by inserting Solidity as a submodule in LLL.

It would be nice answering these questions.

[zigguratt]

These are definitely questions that need answering - and addressing. I've been thinking that LLL should be split out of the Solidity repo. I can see why it was in there when Solidity was part of cpp-ethereum; both languages were in there and it made sense. It makes less sense for LLL to be in the stand-alone Solidity repo.

I think most of these questions have obvious answers if LLL is split from Solidity.

[axic]

My actual long term goal would be:

• Having a separate repo, release cycle, documentation and tests
• Move as much as possible from the parser to macros. Possibly reviewing the language syntax (parser) / semantics during the process.
• Implementing macros for WebAssembly s-expressions (this would then allow using LLL on top of eWASM)

[axic]

One addition to the above: it would be really useful to merge the two LLL dialects. The one in Solidity and the one used by Serpent (actually, Serpent also has the AST in s-expressions, which is not LLL).

[zigguratt]

I didn't know there were two dialects of LLL! Are there any big differences between the two? Also, since Serpent can compile directly to bytecode now, is there a continuing need for LLL there?

[VoR0220]

I could be wrong but doesn't moving LLL out have implications for inline assembly?

[axic]

@VoR0220 LLL is not used within Solidity. LLL depends on libevmasm, which is now maintained in Solidity.

[VoR0220]

Ah...good to know.

[axic]

@zigguratt I didn't know there were two dialects of LLL! Are there any big differences between the two?

See #1288.

@zigguratt Also, since Serpent can compile directly to bytecode now, is there a continuing need for LLL there?

I don't think it can directly compile to bytecode as it always uses LLL in the internal steps (including optimisation). Indeed it is able to output bytecode directly, as well as LLL.

[zigguratt]

I don't think it can directly compile to bytecode as it always uses LLL in the internal steps (including optimisation). Indeed it is able to output bytecode directly, as well as LLL.

So, it can or cannot compile directly to bytecode? Sorry, I'm a bit confused. :)

[axic]

I can see it became a bit convoluted :wink:

It can output bytecode directly to the user, but internally it will go through LLL in every case.

[zigguratt]

Ah, ok! So it even goes through LLL when it's outputting bytecode to the user as well? Looks like Serpent is stuck with an LLL dependency.

[axic]

It has its own LLL implementation though, no code dependency on this project.

[zigguratt]

Understood. So the idea here is to have the Solidity-based LLL support the Serpent-based LLL keywords that we lack. Is the plan to have Serpent possibly use the Solidity-based LLL compiler, or is it more a desire to have them be functionally equivalent? I can see with being useful for sure. until less so, but would still come in handy.

[axic]

I think it fits into the "review the syntax / semantics" point mentioned above. I would vouch for having a basic keywords set which makes the most sense and reviewing the knowledge from both trees seems like a good way to start this process.

[zigguratt]

That sounds reasonable! In that case I would vote against adding a comment keyword. :)

[axic]

I'm thinking about introducing panic as a keyword which does a known invalid jump. (I like panic better than throw, because the latter implies it can be caught. We could of course have a suboptimal implementation to catch throws.)

[zigguratt]

panic would work. It could actually be added as a built-in macro:

(def 'panic (jump 0x02))

In my LLL contracts I usually define invalid-location as 0x02 and then do:

(jump invalid-location)

[axic]

(jump 0x02) does not necessarily means an invalid jump. It depends on what is at location 0x02, and that is mostly beyond the control of the LLL source.

See the example implementation here: https://github.com/ethereum/solidity/tree/lll-panic

[zigguratt]

I was under the impression that Solidity used 0x02 as its throw destination. Maybe I misunderstood, but that's the answer I got when I asked the question on ethereum/solidity a few months back. So perhaps it needs compiler support after all.

[axic]

For Solidity that is always true, because every Solidity bytecode beings with a PUSH for setting up the end-of-memory pointer. Therefore at location 0x02 there won't ever be a JUMPDEST.

[zigguratt]

Ah, now it all makes sense! :grin: And your lll-panic code jumping to an error tag guarantees an invalid jump. So we really do need the panic keyword.

[axic]

I think it is safer, but then again, it is yet another keyword.

In the future, if we improve the (asm) keyword to support labels, we could hide it there, such as (def 'panic (asm PUSH [ErrorTag] JUMP)). (Similar to how __pushsize could be improved.)

[chriseth]

It's not true that 0x02 is always an invalid jump in Solidity because the optimizer may do what it pleases. Furthermore, Solidity would not rely on such a fragile feature :-)
Instead, you can just use Assembly::errorTag() which is assigned the smallest invalid position during the assembly phase.

Furthermore, the throw keyword was used in Solidity with the intention of actually allowing the exception to be caught.

[zigguratt]

That's odd. I believe you @chriseth , but people are certainly under the wrong impression on ethereum/solidity! When I asked back on July 1st of this year, a couple of people came up with 0x02 as the location to which Solidity jumped on throw.

It looks like we'll have a proper implementation soon, so it's no big deal.

[axic]

Instead, you can just use Assembly::errorTag()

The lll-panic branch does that :wink:

[axic]

Immediate todo list for LLL:

• [x] merge the strict parser (#1313)
• [x] fix the panic keyword (#1440)
• [x] add testing (#1442)
• [ ] support the with keyword (#1288)

[zigguratt]

Strict parser can be merged.

[onbjerg]

Hey! Any updates on this at all? Building LLL tooling is really hard in it's current state, since there's no versioning (or is there?) and because it is in the same repository as the Solidity compiler.

[veox]

@onbjerg lllc binaries report the same version as the solc binaries for a given release.

(Is this what you asked about?..)

[aj07]

This links seems useful for abhove discussion https://media.consensys.net/an-introduction-to-lll-for-ethereum-smart-contract-development-e26e38ea6c23

[ekpyron]

@axic What's your current opinion about this? We recently discovered that in our CI we currently don't even build LLL (except for osx), let alone test it and that the tests are currently broken.
And the team so far doesn't have consensus about whether to try to update it, always build it and run its tests or to simply remove it from the repo... [keeping the "to discuss" label for now]

[ekpyron]

From gitter discussion: it seems like @axic will work towards moving the ethereum test suite away from using LLL and at the latest once there's enough progress on that front, we will probably remove LLL from the solidity code base.

[leonardoalt]

From call: @axic is working on standardizing some assembler language for tests, and after that remove LLL.

[axic]

We recently discovered that in our CI we currently don't even build LLL (except for osx)

Btw, this was fixed with #7853 and #7854.