Sdk: Speed of dotnet watch is quite low

[muratg]

_From @janpieterz on May 24, 2016 8:34_

Before the watcher even reports a file change (3+ seconds), then it recompiles (it reports 3.84 seconds but my stopwatch certainly recorded 7+ seconds) and then the app restarts. Overall, it feels like I'm waiting a lot again.

DNX-watch, though it obviously called a different pipeline, was considerably faster and a lot more usable for quick development, where generally it was so fast I could save the file, flick to the browser/rest client and it would already pick it up.

Not sure if this is already work in progress, impossible or anything like that, but it seemed good to report and/or have a discussion about.

_Copied from original issue: aspnet/dotnet-watch#106_

[muratg]

_From @victorhurdugaci on May 24, 2016 18:2_

Thanks for reporting this. Can you please share a bit more details?

1. Do you feel the time it takes the watcher to detect a file changed is too slow or the entire restart process?
2. How do you measure the time?
3. How many files are in your application?
4. What OS do you use?

Maybe this is a bit much to ask but this is one of those bugs where a video showing us what's happening would be ideal.

[muratg]

_From @janpieterz on May 24, 2016 19:36_

No sweat, sure! Certainly worth the time investing ;)

1. Both really, the watcher takes quite a while and the restart process as well.
2. First time a stopwatch, now I used the milliseconds my video recorder shows
3. Not too much, besides a hefty node_modules (both in dnx-watch and dotnet-watch apps) I'd say ~ 100 max ? Basically the default and a couple of controllers extra, most of the stuff happens in external libraries.
4. Windows 10 x64

Videos are also not too much to ask. For fun and extremely easy comparison I've made one of dnx-watch and one of dotnet watch. Project startup of the dnx-watch actually does more (it initalizes NServiceBus as well), which I for the purpose of this demo actually removed from the dnx-watch project (to no avail). I think looking at the video it will be apparent right away what the difference is.
dnx-watch takes less than 77 milliseconds before the change is picked up (literally one frame I'm saving, next frame it's picked up). dotnet-watch takes 3106 ms before the change is picked up, compile time is also considerably slower.

Please see attached videos for more info, in case you want me to do anything else please let me know!

dnx-watch: https://drive.google.com/file/d/0BzlrR85b1aHuWHZDVDREcTI5UW8/view?usp=sharing
dotnet-watch: https://drive.google.com/file/d/0BzlrR85b1aHucXE1cnNHTHRmbTA/view?usp=sharing

[muratg]

_From @nil4 on May 25, 2016 1:15_

Re: hefty node_modules, potentially related to dotnet/cli#2847

[muratg]

_From @janpieterz on May 25, 2016 4:51_

Thanks @nil4, your suggestion seem to help. Removed the node_modules completely and the speed is significantly quicker.
https://drive.google.com/file/d/0BzlrR85b1aHuZjllZGJYdi1WWW8/view?usp=sharing

Good to note though that the whole process is still quite a bit slower than the dnx-watch video, while that application is quite a bit bigger (but I think that this is more the compilation/startup that somehow is slower).

Suggestions for resolving this? Since the other issue exist, where do we need to track this (is is more a dotnet-watch issue or cli?) and should I file a compilation/ speed issue somewhere else?

EDIT: Also good to note is that I have a hefty node_modules in both projects!

[muratg]

_From @victorhurdugaci on May 25, 2016 5:48_

dotnet compilation is slower than dnx because we no longer compile in memory but we first write the binaries on disk and then launch a new process.

[muratg]

_From @victorhurdugaci on May 25, 2016 5:49_

As for the node_modules folder we know it causes issues... Even though the watcher doesn't trigger on changes in that folder, it still watches (usually) the parent folder and all the children.

[muratg]

_From @janpieterz on May 25, 2016 6:19_

That's good to know, thanks.

Are there any options to let dotnet compile in memory? Either by passing this as an option to the watcher (or letting it do so by default, just like for example webpack in dev-server mode goes into a memory only mode)?
And is there any workaround for the node_modules to explicitly exclude it from being watched? It is not an inherent file watching problem I guess, since it used to work fine?

[muratg]

_From @victorhurdugaci on May 25, 2016 16:18_

No, dotnet doesn't support in memory compilation.

I'll do some investigations to see why the file trigger is slow, I don't know yet the root cause but you cannot exclude the node modules unless you move them outside of the project folder.

[muratg]

Thanks @janpieterz! We'll do further investigation on this.

[muratg]

_From @janpieterz on May 25, 2016 18:8_

Awesome guys! I'm more than happy to help out doing some testing, and obviously this all works already, so it's not a 'fix' thing, but if there is some time / resources available, I reckon the video clearly shows an improvement that is worth quite the while, both on the watching and the compiling front!

[blackdwarf]

This seems to be related to two things:

1. Not building in memory - this is something that we are not looking at doing at this time.
2. node-modules, that is, any large quantity of files affecting the build. This is tracked in dotnet/sdk#5968.

Given the above, I will close this issue since for the dotnet/sdk#4284 problem above we have a tracking issue that explains the problem already.

[dealdiane]

The issue tracked by dotnet/sdk#5968 is for building the project. This issue is about dotnet watch and that it's slow to pickup file system changes.

I feel like they're separate issues and this should've been kept open (unless it's tracked somewhere else in which case this should be referenced).