Lark: Lark runs on Pyodide! (Online IDE)

[phorward]

Lark runs out-of-the-box inside the browser using Pyodide:

Pyodide is a CPython 3.7 interpreter compiled to web-assembly (wasm). Here's the Python console from above: https://pyodide.cdn.iodide.io/console.html

Maybe this can be helpful as a quick start for all who quickly want to get into?

[MegaIng]

This is expected as Lark is pure python. Do you propose adding a link somewhere in Lark to Pyodide or an online example Terminal?

[phorward]

It could be possible to add a Pyodide-console like the one I've linked at to Lark that opens a pre-configured Lark environment, where the user can immediately start to use it. It must be hosted somewhere.

[julienmalard]

@phorward Thank you so much! I had just been looking for a way of getting Lark to run in JavaScript.

[phorward]

Thanks. If there is interest to serve a Pyodide-based Lark by the project maintainers, I can contribute some code for that.

[julienmalard]

Thank you @phorward. I am not a project maintainer, but would be interested in using a pyodide-based Lark-powered interactive feature on a website (basically, users write code in one text area, which would then be processed by Lark to an output in another box, a bit like the example on the CoffeeScript web site). Would be interested in any help/suggestions!

[erezsh]

Not sure what you're asking, but if you need my help to make it happen, let me know and I'll do what I can.

[phorward]

... would be interested in using a pyodide-based Lark-powered interactive feature on a website (basically, users write code in one text area, which would then be processed by Lark to an output in another box)

Something like this?
Source code lives here.

[erezsh]

Very nice! If anyone decides to set up a permanent link I'll link to it in the docs/readme.

It would be cool if there were buttons to load other examples.

And when I'll have some free time I can try to style it a little bit more.

[MegaIng]

Would it be possible to add this in a github.io page? Or even read-the-docs?

[julienmalard]

@phorward Wow! Exactly what I was hoping for. By the way, I am using npm and webpack to produce the build, and so can't use a Python script to download and serve Pyodide. Is there a best practice you would recommend for dynamically loading Pyodide directly from the distribution? Thanks!

[phorward]

Hi there, glad to hear your positive feedback!

If anyone decides to set up a permanent link I'll link to it in the docs/readme.
Would it be possible to add this in a github.io page? Or even read-the-docs?

The last version uses the Pyodide from the CDN, therefore its only the contents of the repo that need to be published somewhere. I'm haven't used github.io, yet, but you might test if you want.

It would be cool if there were buttons to load other examples.

https://github.com/phorward/lark-ide/commit/7c174802922abb69bcb994809dc713c8a2534c1d implements a static example selector and parser selector, see link again for a working version.
Sure there are many more nice things to add: Syntax highlighting, collapse-able tree view, more examples, better error diagnostics, stand-alone parser button...

And when I'll have some free time I can try to style it a little bit more.

Feel free to do so, I'm not that good with styling!

By the way, I am using npm and webpack to produce the build, and so can't use a Python script to download and serve Pyodide. Is there a best practice you would recommend for dynamically loading Pyodide directly from the distribution? Thanks!

You can use the CDN as done in the last commit. You first have to load Pyodide, then you can load/install packages and then run your own code. This magis is done in app.js from the repo, it is a standard procedure for us already. I'm not familiar with npm, therefore you have to try it on your own, or ask directly at pyodide issue tracker.

[erezsh]

So, github pages basically works: https://lark-parser.github.io/lark/

So it should be possible to host the ide there, but I think I'll have to copy the files to the lark git. @phorward Is that alright with you?

Or you could activate the github pages on your repo. I'm fine with either option.

[phorward]

@phorward Is that alright with you?

I'm fine with this, copy it wherever you like!

[erezsh]

https://lark-parser.github.io/lark/ide/app.html

[julienmalard]

Thanks for the tips @phorward ! Works a charm now. My page is still only available at localhost:8080, so I can't share a link to a working example just yet, but will as soon as I get it published.

[phorward]

@erezsh looks good :+1:

[erezsh]

Not bad for a programmer :)

Ideally I'd prefer to write the interface in javascript, and only call python when I need to actually parse stuff. Any idea on how to do that? (i.e. without defining a whole app)

[julienmalard]

Any idea on how to do that? (i.e. without defining a whole app)

@erezsh : Here is a work-in-progress example that I made with Vue.js (no whole Python app): https://github.com/lassi-samaaj/lassi-pakkam/blob/master/src/views/அறிமுகம்.vue#L230. On mounted, I load Pyodide with scriptjs and then load all Python packages asyncronously (right now I don't have Lark as an explicit dependency, but I tested it successfully before). Then, in any other method, I can use window.pyodide.runPython or window.pyodide.runPythonAsync to run Python directly, including using Lark. This should work with most any other JS framework too.

Hope this is useful!

[erezsh]

Looks like it should work!

Also, I wonder what's the best way to cache the pip installs, so it won't have to download everything on every refresh. I think it should be possible with localStorage, but I'm not sure how to supply it to pyodide.

[phorward]

Ideally I'd prefer to write the interface in javascript, and only call python when I need to actually parse stuff. Any idea on how to do that? (i.e. without defining a whole app)

I'm of the opinion that writing the app in Python is the "better" way, because its just possible and IMHO an easier and more maintainable way for writing apps and progams (and I also dislike JavaScript BTW). That is the reason why we build most of our apps in Python, running in the browser. But if there's one with more expertise in the JavaScript world for this use-case, then its okay.

Also, I wonder what's the best way to cache the pip installs, so it won't have to download everything on every refresh.

For this, we might build a package, which then can be served together with a self-hosted Pyodide (not the CDN!). I can also add Lark to the standard packages of Pyodide, but I'm aware that this is not directly wanted (because packages can also be installed with micropip as done in the IDE).

What do you think?

[erezsh]

@phorward I dislike javascript too, but it will be a lot easier that way to add syntax highlighting, or other ui features. And it would be nice to write it using something like svelte. But we'll see if I even have time for all of that :)

What do you think?

Actually, I was wrong. It seems that the browser cache already covers it.

What I really want is to reduce the load-time when refreshing (or returning the next day), but it seems like most of the time is spent loading pyodide, so perhaps there is no simple remedy.

[julienmalard]

Hello again @phorward , @erezsh . I just wanted to share a new programming framework called லஸ்ஸி (Lassi) that I built based on Lark: https://lassi-pakkam.herokuapp.com/.

It's all in Python, so I used Pyodide for the browser demonstration on the web page (you have to be a tiny bit patient for it to load, especially on the first runs).

Thank you both for all your help!

P.S. By the way, did you have to do anything special to get non-Python files such as common.lark to load with Pyodide? I ended up implementing a very ugly compilation step to get mine to work.

[erezsh]

Looks nice!

common.lark just comes naturally with pip install lark-parser, I don't think any special handling was necessary. Other data (like the examples) are just strings in a python file.

[julienmalard]

@erezsh Thank you! I think it may have to do with your use of __path__ = os.path.dirname(__file__) and path.abspath() while I was using relative paths. I will try your way now!

[ThatXliner]

Can you guys make use of cookies so that the textbox doesn't get reset every time you refresh?

[phorward]

Yeah, that sounds logical. I'm not sure how further development there should go on, @erezsh wanted to refactor this IDE using JavaScript?

[erezsh]

I did want to, but it doesn't look like I will get to it any time soon.

But if anyone does refactor it to Javascript, I'll be happy to add the script that persists the textbox :)

[julienmalard]

Hello,

I had built something similar in Vue.js (see above); if anybody wants to lead I'd be happy to contribute!
-Julien

[erezsh]

@jlaasonen Sounds interesting. Got a link?

[julienmalard]

@erezsh Here is the link!

The code is not very clean and technically speaking it does not call Lark, but rather the library லஸ்ஸி which itself then calls Lark, but it serves as a nice proof of concept. It also uses web workers to avoid freezing the main thread when Lark has to parse a large grammar.

[julienmalard]

@erezsh Would you be willing to publish the most recent versions of Lark as a .whl file to PyPI? It would allow me to update to the most recent Lark version for Pyodide (as of now I have to use version 0.8.5, which is the last one with a published wheel).
Thanks!

[MegaIng]

There is a .whl for 0.10.1

[julienmalard]

@MegaIng Thank you! I do not see it for 0.10.1, but I do see one for the most recent release candidate (0.11).

[MegaIng]

I see one for 0.10.1:
https://pypi.org/pypi/lark/json

But only on the pacakge lark, not on the package lark-parser.

[julienmalard]

@MegaIng Ah, thank you!