Giraffe: Is there a way to hot reload on development?

[mrkaspa]

[JonCanning]

Have a look at dotnet watch

https://docs.microsoft.com/en-us/aspnet/core/tutorials/dotnet-watch

[dustinmoris]

Thanks @JonCanning. Let me know if you need more help @mrkaspa!

[baronfel]

In the case of large projects, dotnet-watch times can be prohibitive. I've been thinking a bit about a solution and I wanted to outline it here to see if there's anything terribly out of line:

• If all views were moved to a separate assembly from the one hosting the app, then the assemblyresolve event could be hooked, and when the view-containing assembly was loaded a reference to it could be held.
• Separately, a filewatcher could be started on the view-containing assembly that when triggered would unload the view-containing assembly, allowing for a new assembly-load of the newly-built assembly
• Finally, this only works if domains are a thing, so it would only be feasible for .net framework and I believe .net core 3.0

[sergey-tihon]

FSharp.Compiler.PortaCode - I am not 100% sure how, but 98% sure that it should be an option =) At least, because it works for LiveUpdate in Fabulous, it should be possible to do with Giraffe.

Sort of dev-only endpoint that is listen for webhooks from PortaCode and on the next page load is uses latest view for rendering.

[forki]

It would be the coolest thing ever

Am Fr., 15. Feb. 2019, 07:22 hat Sergey Tihon [e-mail]
geschrieben:

FSharp.Compiler.PortaCode
https://github.com/fsprojects/FSharp.Compiler.PortaCode - I am not 100%
sure how, but 98% sure that it should be an option =) At least, because it
works for LiveUpdate in Fabulous, it should be possible to do with Giraffe.

Sort of dev-only endpoint that is listen for webhooks from PortaCode and
on the next page load is uses latest view for rendering.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/giraffe-fsharp/Giraffe/issues/93#issuecomment-463922599,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADgNKn0MslohtC2QfXaaHPp85AwQF5Qks5vNlIKgaJpZM4PD14m
.

[baronfel]

There's some good reference material over at the Fabulous repo: https://github.com/fsprojects/Fabulous/blob/master/src/Fabulous.LiveUpdate/LiveUpdate.fs#L205-L288

It looks like there are three major pieces:

• the portacode interpreter is started to watch some project(s) and notify an endpoint with new compilation results
• the code being interpreted has a kind of 'signal' member (in fabulous this is called program or programLiveUpdate and must match a particular type signature)
• an HTTP server is started to handle those compilation results, which means adding new compilation results to the interactive environment, finding that signal member, and invoking it

So from a giraffe point of view we might need to change the GiraffeMiddleware to have a mutable reference to the HttpHandler, and provide a member to overwrite and re-cache it given another HttpHandler.
EDIT: Design of the fabulous watcher: https://github.com/fsprojects/Fabulous/blob/352c983a459c85ea8dd99121350cec6cd4067200/docs/tools.md#design

[baronfel]

I've started a repository here with the beginnings of how portacode might be used to trigger hot-reloads. I've got a simple test harness set up and I've implemented a first version of points 2 and 3 above, now I need to hook up point 1. If this is something you're interested in contributing to please joing me over there and once we have a polished end result we can open a discussion about if/how such a thing should be incorporated back into Giraffe proper.

[baronfel]

Proof of concept of hot reloading is successful!

[dustinmoris]

That looks awesome. Just so I understand this better I've got a few questions:

• Finally, this only works if domains are a thing, so it would only be feasible for .net framework and I believe .net core 3.0

Your initial proposed solution sounded good to me, but I didn't understand why this would only be possible in netframework and netcore3.0? I'm not super familiar with how assemblies are being loaded atm.

From the PortaCode documentation:

The interpreter may also be useful for other live checking tools, because you get escape the whole complication of actual IL generation, Reflection emit and reflection invoke, and no actual classes etc are generated. We can also adapt the interpreter over time to do things like report extra information back to the host.

I don't know the history of PortaCode but wonder why does Fabulous need to interpret the code instead of compilation? Is this for perf reasons only because re-compilation takes too long?

In the case of large projects, dotnet-watch times can be prohibitive.

Overall, are we trying to achieve exactly what dotnet-watch would do, except a lot faster, particularly for large projects?

If yes, I guess your initial proposed solution was more tied to Giraffe, because you were thinking of the view engine? An idea to make this more useful beyond Giraffe could be to have a config which tells the "watcher" which assemblies to watch (> "... then the assemblyresolve event could be hooked ..")

I will re-open this issue!

[dsyme]

IOS doesn't have full JIT on device.

But TBH in general compilation should be preferred if at all possible - the interpreter is OK but code execution may be slow and have rough edges. For mobile apps interpretation is OK when the main aim is to see the updated view and play with it a bit.

For aspnet I'm not sure Interpretation is the right thing in the balance - compared to recompiling the DLL and thinking it in. No harm in trying it though!

[forki]

The problem with recompiling everything with dotnet watch is that it is
really slow. And restarting the app as well. For dev mode we would really
like to have faster feedback.

Am So., 17. Feb. 2019, 12:29 hat Don Syme [e-mail]
geschrieben:

IOS doesn't have full JIT on device.

But TBH in general compilation should be preferred if at all possible -
the interpreter is OK but code execution may be slow and have rough edges.
For mobile apps interpretation is OK when the main aim is to see the
updated view and play with it a bit.

For aspnet I'm not sure Interpretation is the right thing in the balance -
compared to recompiling the DLL and thinking it in. No harm in trying it
though!

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/giraffe-fsharp/Giraffe/issues/93#issuecomment-464445053,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADgNEk1Wgh_6WqlojbFPk7UHv8X3FGYks5vOT0UgaJpZM4PD14m
.

[dustinmoris]

But TBH in general compilation should be preferred if at all possible

Thanks for clarification, that was my thinking too.

The problem with recompiling everything with dotnet watch is that it is really slow. And restarting the app as well. For dev mode we would really like to have faster feedback.

I think this is the main motivation why @baronfel has re-opened this issue. I think his initial proposal made a lot of sense. If I understood it correctly we would have a sort of improved and must faster dotnet watch by only watching (and reloading) the most minimal set of DLLs which are required for a given task at hand. In his initial request it was focused around views, but I would propose to have it configurable, so that it can be useful beyond just Giraffe views (e.g. a web api written in a different .NET or F# framework could watch and reload a DLL which is responsible for all functionality behind a certain route, etc. - as long as the functionality has been separated out into its own assembly)

What are your thoughts on this @baronfel ?

[forki]

Keeping the app alive has other dev time advantages like you don't need to
repopulate caches or background tasks still run

Am So., 17. Feb. 2019, 12:43 hat Dustin Moris Gorski <
[email protected]> geschrieben:

But TBH in general compilation should be preferred if at all possible

Thanks for clarification, that was my thinking too.

The problem with recompiling everything with dotnet watch is that it is
really slow. And restarting the app as well. For dev mode we would really
like to have faster feedback.

I think this is the main motivation why @baronfel
https://github.com/baronfel has re-opened this issue. I think his
initial proposal made a lot of sense. If I understood it correctly we would
have a sort of improved and must faster dotnet watch by only watching (and
reloading) the most minimal set of DLLs which are required for a given task
at hand. In his initial request it was focused around views, but I would
propose to have it configurable, so that it can be useful beyond just
Giraffe views (e.g. a web api written in a different .NET or F# framework
could watch and reload a DLL which is responsible for all functionality
behind a certain route, etc.)

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/giraffe-fsharp/Giraffe/issues/93#issuecomment-464446328,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADgNBoCnpOkjKpRLppjOwsMV6I2xlOnks5vOUBcgaJpZM4PD14m
.

[sergey-tihon]

In the ideal case, would be nice to configure (or let watcher to decide) what should happened: page reload with new view or recompile and full app reload.

When I'm working on a page, most often I do HTML/CSS/JS changes and would like to see immediate page refresh, less often i change backend part (usually it just works from 1st attempt 😄 )

[baronfel]

So right now all the project does is introduce a new middleware that abstracts all the reloading logic independently of the rest of the Giraffe HttpHandler. It does this by

• defining a giraffe HttpHandler to represent the update API (based off of the one in the Fabulous project) and merging this in a choose block with the user's HttpHandler,
• defining a method to invoke when the wrapped HttpHandler should be Updated with a new one
• integrating WebSockets to trigger a client-side refresh automatically when the underlying handler has been updated

And all of this is invoked by the Portacode CLI on each refresh.

This is pretty simple to implement and orchestrate and has been fairly robust for me so far, given the constraint that there is one module-bound member of the form webApp: HttpHandler in the code that's being updated. The Giraffe Handler is a pretty nice point of reload, and I don't know of another refresh point in the Asp.Net Core framework that would be a nice reload point in a similar fashion. I didn't want to take on the task of allowing completely different WebHostBuilder configurations, for example.

Let me respond to a few other points that have been raised inline so we can keep the discussion going:

comments re: AppDomains

I might have been speaking out of bad knowledge here, but my initial attempt was to try to hook the AssemblyLoad events in a way similar to what the portacode interpreter does for me via its EvalContext. For some reason I thought that AppDomains weren't available on .Net Standard/.Net Core, but I'm looking at apisof.net right now and it appears they are, so that should be a reasonable solution. One thing that attracted me to portacode was that it handled the 'watch the proper files/references for this project and recompile as necessary' aspect of the hot-reload cycle, and I was a bit loathe to recreate that for a set of user-provided assemblies. I suppose it would be doable, it just sounds like work :D

An idea to make this more useful beyond Giraffe could be to have a config which tells the "watcher" which assemblies to watch (> "... then the assemblyresolve event could be hooked ..")

The more I tinker with this the more convinced I am that the super-simple use case of 'I have a webApp and anytime it changes I want to reload' is sufficient to cover the use cases that me and my team at work have while writing Giraffe. Right now in our ~17-project solution dotnet watch takes a decent amount of time and that's a major pain when we're just iterating on view code. The Portacode solution so far seems to be much faster than dotnet watch and has the nice property of not having to spin up the whole webserver config each time, just reload the httpHandler, which as a func itself is pretty fast to swap out. The ASP.Net Core pipelines all remain the same, just this one part changes. I'd propose that major configuration changes like url binding, changes to ConfigureServices/Configure are ok circumstances to do a proper restart of the app.

When I'm working on a page, most often I do HTML/CSS/JS changes and would like to see immediate page refresh

This would involve watcher copying the content files to the current output directory, right? So this is something that dotnet-watch has to detect and do right now? I suppose this could be done as well, but again it just involves cracking projects in the handler.

Thanks everyone for your interest in this effort so far! I'm hoping we can make this a _thing_ in the short term, I'm already having a ton of fun working with it enabled :)

[dsyme]

@baronfel Just to note another potential issue: the JSON serialization of the PortaCode format is not at all compact - you an easily get several MB of data for a small code file - 8x the compiled DLL size or more.

This isn't much of a problem when sending to a phone plugged into USB, but could be an issue if sending remote. In that case compiling to a DLL and sending the DLL may be faster (as well as simpler)

I may add an argument to ProcessCommandLine in PortaCode to simply compile the code and send the bytes of a DLL to use reflection-load on, so testing the difference would be easy.

[reinux]

Phillip Carter pointed out over on /r/fsharp that Giraffe (probably anything to do with Asp.net) pulls in a lot of assemblies, which accounts for quite a bit of a F# compiler time. He also points out that moving things into separate projects can improve compile time.

Could Giraffe.GiraffeViewEngine be moved into its own assembly so that the views can be done in a separate project with fewer dependencies? It's not a perfect solution, but I think it should help a little.

[dustinmoris]

Could Giraffe.GiraffeViewEngine be moved into its own assembly so that the views can be done in a separate project with fewer dependencies? It's not a perfect solution, but I think it should help a little.

Yes, let's open a new issue for that work though.

[dustinmoris]

@reinux

I've created a new repo called Giraffe.ViewEngine we should start by moving the view engine code there and prepare a new NuGet package.

My main question is do you want this new NuGet package to be completely dependency free? (it would be possible to not even have a dependency on Giraffe, as we can just create the necessary functions which follow the same signature as the HttpHandler without ever referencing Giraffe, but I would rather try to avoid this unless this is the main benefit of separating the code?)

Let's move the discussion of the new NuGet package over to #406.

[dustinmoris]

I've released Giraffe.ViewEngine.1.0.0 today and now a new pre-release of Giraffe.5.0.0-alpha-002 with a reference to Griaffe.ViewEngine. The Giraffe.ViewEngine has zero dependencies, so it can be referenced as a minimal package from a project with views and then used with dotnet watch for faster hot reloading.

Going to close this issue now. If there's anything else I can do please ping me to re-open again or open a new issue. Thanks for everyone's help and input!