Cake: Cache compiled script on disk

[patriksvensson]

Would be nice if we could cache compiled script to disk if it haven't changed.

[guillaume86]

FYI I experimented a bit with caching the compiled script. I'd like to use Cake as a task runner and the 4 sec penalty seemed a bit much for running tasks that take under 1 sec on their own.

I just used Emit(path) to generate an assembly and figured out how to call it by reflection, and it seems encouraging:

normal run

Analyzing build script...
Processing build script...

========================================
Default
========================================
Executing task: Default
Hello World!
Finished executing task: Default

Task                          Duration
--------------------------------------------------
Default                       00:00:00.0114282
--------------------------------------------------
Total:                        00:00:00.0114282


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 5
Milliseconds      : 448
Ticks             : 54489236
TotalDays         : 6,30662453703704E-05
TotalHours        : 0,00151358988888889
TotalMinutes      : 0,0908153933333333
TotalSeconds      : 5,4489236
TotalMilliseconds : 5448,9236

compile run (build cache and run)

Analyzing build script...
Processing build script...

========================================
Default
========================================
Executing task: Default
Hello World!
Finished executing task: Default

Task                          Duration
--------------------------------------------------
Default                       00:00:00.0146321
--------------------------------------------------
Total:                        00:00:00.0146321


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 5
Milliseconds      : 347
Ticks             : 53478949
TotalDays         : 6,1896931712963E-05
TotalHours        : 0,00148552636111111
TotalMinutes      : 0,0891315816666667
TotalSeconds      : 5,3478949
TotalMilliseconds : 5347,8949

cached run

Analyzing build script...
Processing build script...

========================================
Default
========================================
Executing task: Default
Hello World!
Finished executing task: Default

Task                          Duration
--------------------------------------------------
Default                       00:00:00.0095663
--------------------------------------------------
Total:                        00:00:00.0095663


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 689
Ticks             : 6890002
TotalDays         : 7,97453935185185E-06
TotalHours        : 0,000191388944444444
TotalMinutes      : 0,0114833366666667
TotalSeconds      : 0,6890002
TotalMilliseconds : 689,0002

It shaved off 4.5 secs for cached runs and is now acceptable to run quick tasks, it will probably help with #2179.

I attached the changed file if anyone is interested (lots of things missing for a proper pull request).
RoslynScriptSession.cs.txt

[alberto-chiesa]

Hi, I would like to try to tackle this task and I'm looking for some suggestion on how to proceed.
I have some questions:

1. Where should I store the precompiled assembly? I see two options: (a) beside the build.cake file as a build.cake.dll and (b) inside a subdirectory under the cake.exe runner (like ./tools/cake/cache)
2. Do you think the last modified date is a good enough heuristic to check if a new compilation is due? I mean, before running the cache is checked and, if a compiled dll is found, the compilation date is compared against the one of the cakefile. If the compilation is after the last modified date of the cakefile, the precached version is used. An alternative is storing the dll as build.cake.md5ofthecakefile.dll and check the hash.
3. Do you think the precaching should be enabled by default, opt-in or opt-out? In case some command line switch should be added, can I have some pointers on where to look for example code?

Thanks!

[gep13]

@alberto-chiesa there is an open RFC here: https://github.com/cake-build/rfcs/pull/2 which discusses this, and we would need to get sign off there on the approach before proceeding with any work on this feature.

[alberto-chiesa]

Thanks @gep13 ! I didn't knew about the RFC repo. It seems the linked proposal answers my exact same questions. Nice to see I wasn't too off-target. Continuing there about the points to discuss.

[alberto-chiesa]

Thanks @gep13 ! I didn't knew about the RFC repo. It seems the linked proposal answers my exact same questions. Nice to see I wasn't too off-target. Continuing there about the points to discuss.

[jamiegs]

Is there any updates on this? the slowness of the startup of cake has been fairly annoying for me and I'm about to look into switching to Nuke for this reason only.

[patriksvensson]

@jamiegs No update on this since no one have submitted a PR for it yet.

[devlead]

@jamiegs happy to take contributions on this, it's not that we don't want this Core team currently have a few other things with higher priority atm.

If all you want is a is a pre compiled .NET Core app you could take a look at Frosting or Cake.Bridge

https://github.com/cake-build/frosting

https://github.com/devlead/Cake.Bridge/tree/master/src/CakeConsoleRunner

Frosting is a more Asp.Net core style and Cake.Bridge is more like 1:1 mapping of Cake script to console app.

[GeertvanHorrik]

Our situation:

We currently use (quite large) shared build scripts in all our repositories. Each stage (initialize, build, test, package, deploy, finalize) is a separate stage and requires recompilation of the cake scripts. Compiling the build scripts takes about 20 seconds, so it would be very sweet if we could decrease this to 0.

Since we have different agents (although the build server tries to run everything for a specific build on the same agent), we would like to cache this. I see 2 options:

1. In the %temp% based on the commit id (a huge improvement for us would be if we could cache based on commit id (saves us compiling 5 times (so 1 instead of 6)).
2. In the %temp% based on the combined hash of the script files (I am pretty sure that calculating the hash is faster than compiling). Then we could re-use the cache for the same build scripts for different repros
3. In the repository itself, but then we would have to be aware of recompilation every time. In our case that's not really an issue since we use RepositorySync so we only have to do it once.

My preferred option is 2. If there is anything I can do to help, let me know.

[gep13]

@GeertvanHorrik thanks for your thoughts. Not sure if you have seen it, but there is an open RFC regarding the addition of caching into Cake. Before moving forward with any implementation, we need to get agreement regarding the way forward. This RFC is here:

https://github.com/cake-build/rfcs/pull/2

If you wanted to read through that (if you haven't already) and add some thoughts to the PR, we can continue to refine the RFC, and then begin on the implementation.

[devlead]

There's an open PR #2650 to be reviewed after 1.0 right?

[gep13]

@devlead yes, I believe that there is a PR for providing some of the functionality, however, we haven't agreed via the RFC exactly what we are hoping to achieve with this functionality, so until that is addressed, I don't think we can move forward with any PR.

[devlead]

Yip just wanted to make sure we didn't duplicate any effort