Roslyn: VS caches Source Generator aggressively - hinders iterative development

[vatsan-madhavan]

I'm just started writing a source generator, and I'm finding that Visual Studio is caching source generators aggressively, and it's making it very hard to do iterative development.

This is what I'm having to do to make even small changes.

• Close Visual Studio (devenv.exe) instance in which the source generator project and its consumer (i.e., the project that ProjectReference's the source generator)
• pskill /t servicehub.roslyncodeanalysisservice
• del $env:TEMP\VS\AnalyzerAssemblyLoader -Recurse -Force <- This is where the source generator binaries seem to get cached. Both devenv and servicehub.roslyncodeanalysisservice seems to hold handles to files under this location

Am I missing something simple to get the development process working more seamlessly ?

I'm on 5.0.100-rc.2.20473.20 + Visual Studio 2019 Enterprise 16.8.0 Preview 4.0 [30517.14.main

[CyrusNajmabadi]

VS support for source-generators is pre-alpha. I would not expect a good VS experience here for a while. Certainly not prior to the 16.9 timeframe.

Am I missing something simple to get the development process working more seamlessly ?

Nope, you're not missing anything. It will just be a while until SGs are really supported in VS.

[vatsan-madhavan]

Is there a way to view the generated sources except through VS/intellisense ?

[Youssef1313]

@vatsan-madhavan

For the purpose of testing/debugging, I was viewing them with a simple hack (which isn't good, but did the purpose for me).

https://github.com/Youssef1313/PrintMembersGenerator/commit/31e31ce65803f3aa6055aa4f621216931f66c911

Then after things got stable with me, I moved to unit testing to confirm the correctness of the generated sources.

https://github.com/Youssef1313/PrintMembersGenerator/blob/master/src/PrintMembersGeneratorTests/PrintMembersGeneratorTest.cs

[CyrusNajmabadi]

Is there a way to view the generated sources except through VS/intellisense ?

As of yesterday, you can now emit generated files to disk so you can inspect them: https://github.com/dotnet/roslyn/pull/47047

[vatsan-madhavan]

This is awesome! This means I can get by with commandline builds while VS catches up!

[jasonmalinowski]

So the one problem @vatsan-madhavan you're probably running into is once we've loaded your assembly...the CLR doesn't give us sane ways to unload it or load a different version if the assembly version hasn't changed. @chsienki or @cartermp any chance somebody already has some MSBuild magic to work around this in some way?

[jaredpar]

Curious: if the generators in this case are strong name signed could we potentially manipulate the version here and load the new copy?

[jasonmalinowski]

@jaredpar Potentially. As crazy as the feature request is, I almost wish the compiler had a feature where it'd generate a (determinstic) but effectively random version and stuff that into the assembly version. :smile:

[jaredpar]

Honestly this could be done as a simple post-build sttep. Load the binary in memory, use the metadata writer to flip the strong name bit, change the version and then load that vs. the one on disk.

[jasonmalinowski]

@jaredpar Thanks for volunteering!

[jaredpar]

The best part about being a lead, maybe the only good part, is the ability to delegate ... @chsienki

😉

[PathogenDavid]

I almost wish the compiler had a feature where it'd generate a (determinstic) but effectively random version and stuff that into the assembly version. 😄

The module version ID would be good for this. Just kludge it into the version number. (It's been a hot minute since I've looked into it, but if I remember right with determinism enabled it's the SHA1/SHA256 of all the compiler inputs or something along those lines.)

Right now specifying a wildcard for the assembly version results in CS8357 if determinism is enabled. Maybe determinism + wildcard could mean "kludge MVID into version number".

[MisinformedDNA]

Is the Roslyn team not going through the same pains as us?

[jaredpar]

We use source generators inside our main solution hence we are dogfooding the experience every day. The source generators aren't iterated on as frequently though hence we don't hit the specific reload problem.

This is a problem we are taking a look at. It's existed since Roslyn 1.0 with analyzers (so roughly five years now) hence it's not a new problem, generators has just shined a new light on it. There are some ideas on how to work around this (see my comments above). At the moment though Roslyn does most of the evaluation in process and given we are still on .NET Desktop that limits our options a bit because of the inability to load multiple copies of a DLL into the same process space unless it's strong name signed + changes versions on every build.

[tmat]

@PathogenDavid Using hash for version is not viable since it's not monotonic and versions are expected to be.

I'd suggest that instead of running source generator in VS when developing it it's better to run it in a unit test. Write a unit test that runs the generator and produces output. In that setting you can iterate fast - even using Edit and Continue to modify the generator code as you are debugging it. There is no need to mess with versions/reloading/VS complexity etc.

[tmat]

@jasonmalinowski

So the one problem @vatsan-madhavan you're probably running into is once we've loaded your assembly...the CLR doesn't give us sane ways to unload it or load a different version if the assembly version hasn't changed. @chsienki or @cartermp any chance somebody already has some MSBuild magic to work around this in some way?

The magic is called Core CLR ;-).

[PathogenDavid]

Using hash for version is not viable since it's not monotonic and versions are expected to be.

Fair enough for making that the default behavior of *, but for the purposes of this discussion the version number being monotonic doesn't matter.

I'd suggest that instead of running source generator in VS when developing it it's better to run it in a unit test.

This is essentially what I've been doing. (Except running dotnet build outside of Visual Studio with EmitCompilerGeneratedFiles enabled.)

The main frustration for me has been that once I have finished working on my source generator, convincing Visual Studio to relinquish whatever hidden cached source generator it is seemingly impossible to do reliably. At the very least I'd expect restarting Visual Studio to fix things, but even stopping all instances of Visual Studio, killing any lingering service hubs, deleting the cache folder named in the main issue, deleting all folders starting with vs in my temp folder, making a blood sacrifice, deleting my .vs folder, and finally restarting Visual Studio: My old generator still sometimes (somehow) sticks around. (But only sometimes.)

[PathogenDavid]

any chance somebody already has some MSBuild magic to work around this in some way?

I was going to sit on this a little bit longer to make sure it doesn't have problems, but since I'm here and thinking about it...

I've been using this workaround for about a week without issues. In short: I keep the emitted code around during normal builds. During design-time builds the in-solution source generators are skipped and the previously-emitted code is used instead.

This means the IDE experience does not update without manually building, which obviously undermines a lot of the benefits of source generators, but it's a much smoother experience until the Visual Studio issues are resolved.

First, enable EmitCompilerGeneratedFiles. (I do this in Directory.Biuld.props so it applies to my entire solution.)

<PropertyGroup>
  <EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
</PropertyGroup>

Then, in Directory.Build.targets: (Should work in the project file too, did not test.)

<!--
  As a workaround for https://github.com/dotnet/roslyn/issues/44093 we skip source generators for design-time builds
  and emit generated files and include them for design-time builds.
-->
<Target Name="Roslyn44093Workaround" BeforeTargets="CoreCompile;ResolveProjectReferences;$(PrepareProjectReferencesDependsOn)" Condition="'$(DesignTimeBuild)' == 'true'">
    <ItemGroup>
      <!-- This is slightly over-zealous since it removes in-solution analyzers too -->
      <_AnalyzerProjectReferencesToRemove Include="@(ProjectReference)" Condition="'%(OutputItemType)' == 'Analyzer'" />
      <ProjectReference Remove="@(_AnalyzerProjectReferencesToRemove)" />

      <Compile Include="$(CompilerGeneratedFilesOutputPath)/%(_AnalyzerProjectReferencesToRemove.Filename)/**/*.cs" Visible="false" />
    </ItemGroup>
</Target>
<!-- Workaround for https://github.com/dotnet/roslyn/issues/47966 and https://github.com/dotnet/roslyn/issues/49125 -->
<Target Name="Roslyn46966And49125Workaround" BeforeTargets="CoreCompile;Clean" Condition="'$(DesignTimeBuild)' != 'true' and '$(CompilerGeneratedFilesOutputPath)' != ''">
  <ItemGroup>
    <_GeneratedSourceFileToRemove Include="$(CompilerGeneratedFilesOutputPath)/**/*.cs" />
  </ItemGroup>
  <Delete Files="@(_GeneratedSourceFileToRemove)" />
</Target>

Since this leverages EmitCompilerGeneratedFiles, you do not need to modify your source generator in any way. This fix intentionally only applies to in-solution generators, so NuGet generators are unaffected.

(Note the included workaround for https://github.com/dotnet/roslyn/issues/47966 and https://github.com/dotnet/roslyn/issues/49125: It's pretty aggressive, so don't use your CompilerGeneratedFilesOutputPath for unrelated files for whatever reason.)

Edit: It's also worth pointing out this is slightly over-zealous and ends up removing any in-solution analyzers from design-time builds. You can tweak _AnalyzerProjectReferencesToRemove to be less aggressive if that causes you issues.

[palpha]

Pretty aggressive is accurate – trying to implement your fix, @PathogenDavid, all my .cs files on my entire C: drive were deleted. I probably did something wrong, but perhaps you want to do some sort of check on CompilerGeneratedFilesOutputPath in _GeneratedSourceFileToRemove... Now, time to dig through my latest backup and hope for the best.

[PathogenDavid]

@palpha 😱 That's horrifying! Sorry you're having to deal with that, glad to hear you have backups at least. I added a gate to skip that target if CompilerGeneratedFilesOutputPath is missing for whatever reason.

[PathogenDavid]

It might be too late by now, but you might see if Windows File Recovery can help get some of them back.

[palpha]

Yeah, I hadn't installed the May feature update required for the tool, so I'm currently waiting for that update and hoping it doesn't overwrite the deleted data in the file system.

Any important files should have been in a repository and any changes should have been committed, and I should have checked the fix more carefully – it's definitely my fault if I lose anything of value.

Hopefully the end result is just a bit of tedious work and yet another war story of how I've failed, with which I can bore my niece and nephews when I retire.