Project-system: FastUpToDate check does not respect custom Inputs/Outputs to msbuild targets

[MansVestin]

I have a target in my csproj which has a number of text files as input and one text file as output.

When building the solution after editing one of the input files, the project is not being built. If I re-build, the project and target is executed as expected.

When enabling Verbose output from the FastUpToDate checker, I noticed that it does not mention my input or output files.

This is what my csproj looks like (I have two .txt files in the same folder):

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net471</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <TXTFiles Include="*.txt" />
  </ItemGroup>

  <Target Name="ConvertTextFiles" BeforeTargets="Build" Inputs="@(TXTFiles)" Outputs="$(OutDir)files.log">
    <Message Text="Writing $(OutDir)files.log" Importance="High" />
    <WriteLinesToFile File="$(OutDir)files.log" Lines="Hello world" Overwrite="true" />
  </Target> 

</Project>

This is the output from FastUpToDateChecker:

1>FastUpToDate: Adding project file inputs: (TestProject)
1>FastUpToDate:     'C:\GIT\raycare\src\components\user\TestProject\TestProject.csproj' (TestProject)
1>FastUpToDate: Adding import inputs: (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\Sdk\Sdk.props' (TestProject)
1>FastUpToDate:     'C:\GIT\raycare\src\components\user\TestProject\obj\TestProject.csproj.nuget.g.props' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.props' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.DefaultItems.props' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.SupportedTargetFrameworks.props' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.CSharp.props' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\Sdk\Sdk.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.BeforeCommon.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.DefaultAssemblyInfo.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.DefaultOutputPaths.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.TargetFrameworkInference.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.RuntimeIdentifierInference.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.NuGetOfflineCache.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin\Microsoft.CSharp.CurrentVersion.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Microsoft\VisualStudio\Managed\Microsoft.Managed.DesignTime.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Microsoft\VisualStudio\Managed\Microsoft.CSharp.DesignTime.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin\Microsoft.NETFramework.CurrentVersion.props' (TestProject)
1>FastUpToDate:     'C:\GIT\raycare\src\components\user\TestProject\TestProject.csproj' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin\Microsoft.Common.CurrentVersion.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin\Microsoft.NETFramework.CurrentVersion.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin\Microsoft.Xaml.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\NuGet\NuGet.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Microsoft.Common.targets\ImportAfter\Microsoft.NET.Build.Extensions.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Microsoft\Microsoft.NET.Build.Extensions\Microsoft.NET.Build.Extensions.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Microsoft\Microsoft.NET.Build.Extensions\Microsoft.NET.Build.Extensions.NETFramework.targets' (TestProject)
1>FastUpToDate:     'C:\GIT\raycare\src\components\user\TestProject\obj\TestProject.csproj.nuget.g.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.Common.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.DefaultItems.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.GenerateAssemblyInfo.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.ComposeStore.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.CrossGen.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.ObsoleteReferences.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.PreserveCompilationContext.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.ConflictResolution.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.DefaultPackageConflictOverrides.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.CSharp.targets' (TestProject)
1>FastUpToDate:     'C:\Program Files\dotnet\sdk\2.1.402\Sdks\NuGet.Build.Tasks.Pack\build\NuGet.Build.Tasks.Pack.targets' (TestProject)
1>FastUpToDate: Adding Compile inputs: (TestProject)
1>FastUpToDate:     'C:\GIT\raycare\src\components\user\TestProject\Class1.cs' (TestProject)
1>FastUpToDate: Adding ResolvedCompilationReference inputs: (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.7.1\System.Numerics.dll' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.7.1\System.Core.dll' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.7.1\mscorlib.dll' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.7.1\System.IO.Compression.FileSystem.dll' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.7.1\System.Runtime.Serialization.dll' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.7.1\System.Xml.dll' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.7.1\System.Xml.Linq.dll' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.7.1\System.dll' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.7.1\System.Data.dll' (TestProject)
1>FastUpToDate:     'C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.7.1\System.Drawing.dll' (TestProject)
1>FastUpToDate: Adding UpToDateCheckInput inputs: (TestProject)
1>FastUpToDate:     'C:\GIT\raycare\src\components\user\TestProject\obj\Debug\net471\TestProject.csproj.CoreCompileInputs.cache' (TestProject)
1>FastUpToDate: Adding UpToDateCheckBuilt outputs: (TestProject)
1>FastUpToDate:     'C:\GIT\raycare\src\components\user\TestProject\bin\Debug\net471\TestProject.pdb' (TestProject)
1>FastUpToDate:     'C:\GIT\raycare\src\components\user\TestProject\obj\Debug\net471\TestProject.dll' (TestProject)
1>FastUpToDate:     'C:\GIT\raycare\src\components\user\TestProject\obj\Debug\net471\TestProject.pdb' (TestProject)
1>FastUpToDate:     'C:\GIT\raycare\src\components\user\TestProject\bin\Debug\net471\TestProject.dll' (TestProject)
1>FastUpToDate: Latest write timestamp on input is 2018-10-10 15:25:35 on 'C:\GIT\raycare\src\components\user\TestProject\obj\Debug\net471\TestProject.csproj.CoreCompileInputs.cache'. (TestProject)
1>FastUpToDate: Earliest write timestamp on output is 2018-10-10 15:25:35 on 'C:\GIT\raycare\src\components\user\TestProject\bin\Debug\net471\TestProject.pdb'. (TestProject)
1>FastUpToDate: Project is up to date. (TestProject)
========== Build: 0 succeeded, 0 failed, 1 up-to-date, 0 skipped ==========

[drewnoakes]

@davkean do you know where we would source these inputs in code?

[Pilchie]

Hmm - I'm not sure, but I don't know if we can look at that for every arbitrary target (or even know if the target will run during build). A workaround would be to add the custom item types to @ UpToDateCheckInput and @ UpToDateCheckOutput though right?

[MansVestin]

Is there a workaround I can use while waiting for version 16?

(I tried to add a content file with "CopyAlways" to the project, but that feature seems broken as well, see https://github.com/dotnet/project-system/issues/3568)

Thanks,
Måns

[drewnoakes]

@MansVestin I verified that msbuild does the right thing with your scenario and that the problem lies in the project system's check. Is that consistent with your experience?

As a workaround, try specifying custom inputs/output using:

    <UpToDateCheckInput Include="@(TXTFiles)" />
    <UpToDateCheckOutput Include="$(OutDir)files.log" />

Whether this is a bug we can fix or not will require further research. As @Pilchie suggested, we probably can't consider all inputs/outputs of all targets as part of the fast up to date check, and it's not straightforward to know whether a given target will run or not.

Please let us know if the above works.

[drewnoakes]

@MansVestin the CopyAlways fix will be in 16.0.

[Pzixel]

@drewnoakes I must say that UpToDateCheckInput doesn't work but CustomAdditionalCompileInputs seems doing the trick. Thanks.

[drewnoakes]

@Pzixel can you elaborate on why UpToDateCheckInput wasn't working for you? Are you sure you're using this project system? (If you're using SDK-style projects then you are.)

I've tested use of UpToDateCheckInput locally and it works for me. We've had to remove support for CustomAdditionalCompileInputs as it caused other bugs.

[Pzixel]

I'm actually not sure currently. There was other problems preventing from re-builds. I can check later if I won't forget about it (I mean in a month or so, I need some changes in external dotnet CLI tools to give it a try).

[drewnoakes]

@MansVestin we've improved the documentation around customising the behaviour of the up-to-date check:

https://github.com/dotnet/project-system/blob/master/docs/up-to-date-check.md

I don't see a code change that we would apply here to cover this scenario. If you want the up-to-date check to consider more files than it currently does, you'll need to add them to UpToDateCheckInput and UpToDateCheckBuilt item groups as described in the docs.

I'm going to close this for now.

[Pzixel]

Thanks for the change. Unfortunately it seems that it doesn't work properly for me. My build also runs some dotnet-cli tools. And I can see that even with build restarted they don't produce the same result as I get when I hit rebuild button. The compilation time is also different.

I can try to present a MRE if you're interested in one.

[drewnoakes]

@Pzixel could you open a new issue with a repro and I'll take a look?

[Pzixel]

@drewnoakes see #5621