Msbuild: GenerateResource task does not support non-string resources on .NET core

[nguerrera]

_From @tarekgh on June 14, 2017 15:54_

@Spongman commented on Tue Jun 13 2017

a (.csproj) project a with an embedded text file, added using the VS2017.2 project properties UI.

ConsoleApp2.zip

on windows, it works fine.

on linux, it just shows:
$ dotnet --version 1.0.4 $ uname -a Linux mybox 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2 (2017-04-30) x86_64 GNU/Linux $ dotnet run ..\Resources\New Text Document.txt;System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;Windows-1252

---

@danmosemsft commented on Tue Jun 13 2017

@tarekgh are there any known issues here?

---

@tarekgh commented on Wed Jun 14 2017

Please take a look at the issue https://github.com/dotnet/cli/issues/3695 this was CLI issue and I believe it is already fixed but I am not sure which release include the fix. I'll move this issue to CLI repo.

_Copied from original issue: dotnet/cli#6866_

[nguerrera]

MSBuild does not support currently support non-string resources on .NET Core. However, it is not checking the resx data's type attribute, but instead blindly using the value as a string. This should either work or have a clear "not supported" error at build time.

Relevant snippet from repro project resx:

  <data name="New Text Document" type="System.Resources.ResXFileRef, System.Windows.Forms">
    <value>..\Resources\New Text Document.txt;System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;Windows-1252</value>
  </data>

Code that is not checking the type attribute:
https://github.com/Microsoft/msbuild/blob/cc727f715651f305639ad8635fefc40165299f86/src/Tasks/GenerateResource.cs#L2909-L2920

It might be easy enough to actually handle this ResxFileRef-to-string case correctly. If we do that, we still need a clear error in other non-string data cases.

[eltiare]

I am eagerly awaiting the embedding of all resources via the commandline as it will make my life easier with the watch command.

[markrendle]

As a stop-gap solution while we wait for resource embedding to become a properly-supported, cross-platform, I created a dotnet tool which does something vaguely similar: https://github.com/RendleLabs/Embedder/blob/master/README.md

(I forgot to git push, will do that soon)

[jetersen]

This also applies to images even on Windows using dotnet build, msbuild 😭

.NET Command Line Tools (2.0.2)

Product Information:
 Version:            2.0.2
 Commit SHA-1 hash:  a04b4bf512

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.16299
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.0.2\

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.0
  Build    : e8b8861ac7faf042c87a5c2f9f2d04c98b69f28d

[viktors-telle]

Any ETA on fixing this?

[auxym]

I'm also getting with issue with dotnet build / dotnet msbuild, but not with the msbuild.exe packaged with VS2017, which seems to correctly embed string resources.

[AlgorithmsAreCool]

This explains a lot. I had a WinForms application that would fail to launch at runtime with dotnet build but worked fine in VS2017. I was getting an InvalidCastException when loading an image resource.

EDIT
As auxym says, dotnet msbuild does not work as of cli version 2.1.2. You need to call the msbuild visual studio installs. For me this is located at:

"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\Bin\MSBuild.exe"

For some reason this still does not work in release configuration. I have to use debug to get the resources embedded correctly. Hope this helps until a proper fix is available.

[mikebridge]

I have a ResourceManager.GetObject call on an embedded resource which succeeds on windows but fails with the following error on using dotnetcore 2.0.3 (from the dotnet-sdk-2.0.3 docker image) on Linux:

System.InvalidCastException : Unable to cast object of type 'System.String' to type 'System.Byte[]'.

Is this a related issue?

I have an embedded text file "grades.json" which is accessed via the autogenerated Resources.Designer.cs file. That error message occurs here:

internal static byte[] grades {
    get {
        object obj = ResourceManager.GetObject("grades", resourceCulture);
        return ((byte[])(obj));
    }
}

[danmoseley]

ResourceManager will support binary blobs in resources with https://github.com/dotnet/corefx/issues/26745. I am guessing this is also "Unable to cast object of type 'System.String' to type 'System.Byte[]'" .. @viktorhofer can verify.

I had a WinForms application that would fail to launch at runtime with dotnet build

Bear in mind that Winforms is not supported on .NET Core so if it's actually a Winforms app you will hit more errors after this when you try to load a Winforms type.

One other Winforms caveat: the set of binary-serializable types is smaller in .NET Core than in .NET Framework (list) and it does not include for example Image, Icon and Font - types that the Winforms designer likes to binary serialize into resources.

[mikebridge]

@danmosemsft this particular instance is a c# library called from a webapp.

[danmoseley]

@mikebridge what are the type / mimetype you see in the resx?
eg

  <data name="pictureBox1.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">

[mikebridge]

@danmosemsft The resx file shows this:

<data name="grades" type="System.Resources.ResXFileRef, System.Windows.Forms">
  <value>..\grades.json;System.Byte[], mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
</data>

I didn't realize it was specific to winforms---I can embed this file another way if necessary....

[danmoseley]

Yeah, ResxFileRef is a Winforms type for whatever reason (although presumably it doesn't need Winforms proper so it could be ported.) You'll want to embed as a string or a binary formatted string directly into the resx.

[tarekgh]

you can just embed the file directly to the resources and read it with Assembly.GetManifestResourceStream. no need to miss with the resx at all.

[mikebridge]

@danmosemfst @tarekgh, Thanks, I managed to work around it like this:

var assembly = Assembly.GetExecutingAssembly();
String resourceName = @"GradeTranslator.grades.json";
using (var stream = assembly.GetManifestResourceStream(resourceName))
{
    if (stream == null)
    {
        throw new Exception($"Resource {resourceName} not found in {assembly.FullName}.  Valid resources are: {String.Join(", ", assembly.GetManifestResourceNames())}.");
    }
    using (var reader = new StreamReader(stream))
    {
        return reader.ReadToEnd();
    }
}

This appears to work under both dotnet cli and msbuild

[amrmahdi]

When is this planned to be fixed ?

[nguerrera]

I'm pretty sure this is now a must-fix for the announced .NET Core 3.0 support for Windows Forms: https://blogs.msdn.microsoft.com/dotnet/2018/05/07/net-core-3-and-support-for-windows-desktop-applications/ :)

And we can't just error out and say only strings are supported. We have to actually make non-strings work.

[danmoseley]

fyi @ericstj

[nguerrera]

@tarekgh wrote:

you can just embed the file directly to the resources and read it with Assembly.GetManifestResourceStream. no need to miss with the resx at all.

Note that this doesn't work very well if your resources are localized. You also need the logic to get the right resource for the current UI culture. ResourceManager is supposed to be the thing that handles that.

[tarekgh]

Note that this doesn't work very well if your resources are localized. You also need the logic to get the right resource for the current UI culture. ResourceManager is supposed to be the thing that handles that.

That is right. who want language-specific resources will need this functionality fixed otherwise it will not be easy to workaround this except if you do a lot of manual workaround work.

@nguerrera I was looking at some other offline internal reported issue, and it end-up it is the same issue here but using binary files and calling ResourceManager.GetStream.

  <data name="LanguageDetection" type="System.Resources.ResXFileRef, System.Windows.Forms">
    <value>AudioFiles\pt-br\LanguageDetection.wav;System.IO.MemoryStream, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
  </data>

of course, using mscorlib need to change anyway by our tools.

I have a couple of questions here:

• does currently msbuild differ if building for Windows than building on Linux? I am asking because the claim from the user is what they are doing work on Windows but not on Linux. I'll try to take a deeper look to know how they are building anyway.
• Is there any possibility the fix of this embedded files issues be serviced to 2.1? the user is trying to ship with 2.1 and wants this functionality.

[rainersigwald]

does currently msbuild differ if building for Windows than building on Linux?

The distinction is whether the MSBuild that is building the project is the full-framework MSBuild.exe that is distributed with VS and runs only on Windows, or the .NET Core version that comes with the dotnet SDK and runs cross-platform. The latter does not support some paths through resource generation.

[jaredpar]

For a real world example of where this can come up: this bug is currently blocking the VB compiler tests from running on CoreClr. The dotnet build command embeds our resources used in tests incorrectly and as a result the tests end up failing.

https://github.com/dotnet/roslyn/pull/28063

[jaredpar]

@nguerrera gave me a work around here. Going to move to this declaration to work around it.

<EmbeddedResource Include="results\uncheckedctypeandimplicitconversionseven.txt" />

[maxrandolph]

Commenting for visibility. This is not purely an issue on Linux.

[jaredpar]

@maxrandolph definitely. If you look at all the commits I've been linking in here they are fixing problems on Windows. It's a dotnet build issue.

@nguerrera okay to change the title?

[nguerrera]

I changed the title. It was just carried over from the original user report on dotnet/sdk that got moved here.

[nguerrera]

It's a dotnet build issue.

Correct. (More pedantically, it's an issue with MSBuild running on .NET Core.)

[nguerrera]

Just bumping this comment: https://github.com/Microsoft/msbuild/issues/2221#issuecomment-389000840

This is a must-fix for .NET Core 3. @AndyGerlicher, do we have a milestone or tag we can use for that?

@dsplaisted Can you put this on the agenda for next sync up with msbuild?

cc @dasMulli

[rainersigwald]

We're not currently distinguishing between 16.0 and .NET Core 3 even though the timelines may not match in practice, so pulled this in for 16.0.

[danmoseley]

cc @dreddy-work who is on the winforms side.

[rainersigwald]

I spent some time looking into what MSBuild's dependencies are here.

GenerateResource has a few categories of thing gated behind feature flags:

• [ ] The ability to just pass its arguments to an invocation of resgen.exe.
  
  
  
  • No resgen equivalent on Core.
  
  
• [x] References to IResourceWriter and IResourceReader.
  
  
  
  • Available now, but weren't at initial port.
  
  
  • PR to unify this codepath: https://github.com/Microsoft/msbuild/pull/3816
  
  
• [ ] References to System.Resources.ResXResourceReader (from System.Windows.Forms.dll).
  
  
  
  • Not available in Core at the moment.
  
  
  • Is this tracked in a winforms effort somewhere?
  
  
• [ ] AppDomain and other assembly-loading-manipulation code.
  
  
  
  • There's code to run ProcessResourceFiles in a separate appdomain if the resx file has types that cannot be loaded from the GAC.
  
  
  • That appdomain is unloaded after processing, so MSBuild doesn't lock the (probably built in the same solution) assemblies it needed to load to process the resources.
  
  
  • In Core there are no appdomains nor assembly unloading.
  
  
  • This could just be elided for Core, but that would probably result in background MSBuild.exe processes locking user code semi-randomly.
  
  
  • Or maybe there's some future in "don't allow loading types during resource processing"? I don't know how feasible that would be.
  
  

[dsplaisted]

System.Resources.ResXResourceReader looks like it's available in the Microsoft.DesktopUI shared framework. Probably we don't want to add a dependency in MSBuild to that, so we'll have to discuss further.

[nguerrera]

no resgen equivalent on core

I would imagine it would be possible to build resgen on core now if we need it.

References to System.Resources.ResXResourceReader (from System.Windows.Forms.dll).

I see this type in System.Windows.Forms.dll of 3.0 builds now. I think we still have a layering issue, though. I'm not sure if we can make msbuild depend on the winforms shared runtime component.

In Core there are no appdomains nor assembly unloading.

AFAIK, collectible assemblies are planned for 3.0. https://github.com/dotnet/coreclr/issues/552#issuecomment-410238253

[ericstj]

System.Windows.Forms

We produce a NuGet package for that assembly as well. Again, not sure if that is the final form of this (perhaps we want to push the type(s) down) but it could work for now.

[filipnavara]

I would imagine it would be possible to build resgen on core now if we need it.

I don't think it's necessary. It is not normally used for builds anymore unless someone explicitly forces it. The only reason to bring it back in is perhaps to replace the AppDomain isolation.

[sdmaclea]

@vitek-karas @jeffschwMSFT Note @rainersigwald's comment https://github.com/Microsoft/msbuild/issues/2221#issuecomment-426028467 above regarding AppDomain dependency. Also .Net Core resgen dependency.

[ericstj]

AppDomain dependency

Another angle to take on that is: why does ResGen need to load user code for execution/relfection? ResGen should be able to get everything it needs by just reading assembly metadata and never loading types. That likely means larger changes in either this task or in the WinForms implementation of ResXResourceReader but that's the direction we've been heading with other build tools.

[nguerrera]

I've asked this before, but I'll ask it again here. Why do we need reflection at all to go from one serialized format (resx) to another (resources)?

Ignoring format details, it seems theoretically possible to defer all instantiation until runtime and just encode the resx without loss of information into the resource.

Is there a limitation of the .resources format itself at play, or is it just the API we've been using to write it?

[dasMulli]

The design of .resx for non-string resources is questionable already.
Currently it states "use this class for serialisation with this input parameter", the parameter being file references and the classes being inside the windows forms namespaces or system.drawing..
A "new" first-class format would be great (with xliff support), but for portability we'd need some sort of compatibility..

[jnm2]

Bootstrapping a net472 deployment tool via dotnet run causes this subtle issue. You don't find out until you've already executed the SQL command path\to\file.sql;System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089;Windows-1252 against your SQL Server. 😂

We're working around it by assuming there will be a Visual Studio installation on the machine. dotnet run deploy\Foo.Deploy becomes:

$visualStudioInstallation = & "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe" -latest -products * -requires Microsoft.Component.MSBuild -property installationPath
$configuration = 'Release'
$targetFramework = 'net472'

$msbuild = Join-Path $visualStudioInstallation 'MSBuild\15.0\Bin\MSBuild.exe'
& $msbuild deploy\Foo.Deploy /restore /p:Configuration=$configuration /v:quiet /nologo
$fooDeploy= "deploy\Foo.Deploy\bin\$configuration\$targetFramework\Foo.Deploy.exe"

& $fooDeploy $args

Looking forward to the .NET Core 3.0 SDK! 🎉

[wheerd]

We are currently working around this issue in our projects with the following helper function:

public static class ResourceHelper
{
    public static byte[] GetBytes<T>(string name)
    {
        var assembly = typeof(T).Assembly;
        using (var stream = assembly.GetManifestResourceStream(name))
        {
            if (stream == null)
            {
                throw new Exception(
                    $"Resource {name} not found in {assembly.FullName}.  Valid resources are: {string.Join(", ", assembly.GetManifestResourceNames())}.");
            }
            using (var ms = new MemoryStream())
            {
                stream.CopyTo(ms);
                return ms.ToArray();
            }
        }
    }
}

Then we add the files as <EmbeddedResource Include="Resources\some_file" /> and replace the old Resourceswith this:

namespace Some.Namespace.Resources
{
    public class Resources
    {
        public static byte[] SomeFile => ResourceHelper.GetBytes<Resources>("Some.Namespace.Resources.some_file");
    }
} 

[rainersigwald]

Extremely sloppy notes to myself from things I've figured out recently:

• ResourceWriter.Precanned resource exists: https://github.com/dotnet/corefx/blob/eb068ffb575e7fc561bac65b236dbbb739e4e0c7/src/System.Resources.Writer/src/System/Resources/ResourceWriter.cs#L205-L218
• call via AddResourceData
• But getting the correct fully-qualified type from the resx xml might be a hassle, see

  <assembly alias="System.Drawing" name="System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
  <data name="logoPictureBox.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">

• So the task will probably need a custom resx parser that understands that, or get a ResXResourceReader that does type-name resolution but never loads a type (somehow?)

[jnm2]

does type-name resolution but never loads a type (somehow?)

Will https://github.com/dotnet/corefx/pull/33201 possibly help?

[rainersigwald]

Will dotnet/corefx#33201 possibly help?

I don't think so, because the problem is that in the normal current flow, the type is deserialized into a live object in memory, so the assembly that contains the type definition must be really loaded, not reflection-only loaded or metadata loaded.

[rainersigwald]

I did a sloppy prototype to learn about the feasibility of using AddResourceData: https://github.com/rainersigwald/msbuild/commit/9ee5773739eb743dba4bea22cb4ad348dd5ab0da.

It's super close, but doesn't quite work.

Unhandled Exception: System.Runtime.Serialization.SerializationException: The input stream is not a valid binary format. The starting contents (in bytes) are: FF-D8-FF-E0-00-10-4A-46-49-46-00-01-01-01-00-60-00 ...
   at System.Runtime.Serialization.Formatters.Binary.SerializationHeaderRecord.Read(BinaryParser input)
   at System.Runtime.Serialization.Formatters.Binary.BinaryParser.ReadSerializationHeaderRecord()
   at System.Runtime.Serialization.Formatters.Binary.BinaryParser.Run()   at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(BinaryParser serParser, Boolean fCheck)
   at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream, Boolean check)
   at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream)   at System.Resources.ResourceReader.DeserializeObject(Int32 typeIndex)
   at System.Resources.ResourceReader.LoadObjectV2(Int32 pos, ResourceTypeCode& typeCode)   at System.Resources.RuntimeResourceSet.GetObject(String key, Boolean ignoreCase, Boolean isString)
   at System.Resources.ResourceManager.GetObject(String name, CultureInfo culture, Boolean wrapUnmanagedMemStream)   at netcore.Form1.InitializeComponent() in S:\work\netcore_resources\netcore\Form1.Designer.cs:line 38
   at netcore.Form1..ctor() in S:\work\netcore_resources\netcore\Form1.cs:line 17
   at netcore.Program.Main() in S:\work\netcore_resources\netcore\Program.cs:line 19

Diffing the .resources file, left=full-framework MSBuild, right=hardcoded AddResourceData

Mode:  Differences
Left file: S:\work\netcore_resources\examples\full\netcore.Form1.resources
Right file: S:\work\netcore_resources\netcore\obj\Debug\netcoreapp3.0\netcore.Form1.resources
0000013C 00 49 00 6D 00 61 00 67  00 65 00 00 00 00 00 40  00 01 00 00 00 FF FF FF  .I.m.a.g.e.....@.....ÿÿÿ <> 0000013C 00 49 00 6D 00 61 00 67  00 65 00 00 00 00 00 40                           .I.m.a.g.e.....@
00000154 FF 01 00 00 00 00 00 00  00 0C 02 00 00 00 51 53  79 73 74 65 6D 2E 44 72  ÿ.............QSystem.Dr
0000016C 61 77 69 6E 67 2C 20 56  65 72 73 69 6F 6E 3D 34  2E 30 2E 30 2E 30 2C 20  awing, Version=4.0.0.0,
00000184 43 75 6C 74 75 72 65 3D  6E 65 75 74 72 61 6C 2C  20 50 75 62 6C 69 63 4B  Culture=neutral, PublicK
0000019C 65 79 54 6F 6B 65 6E 3D  62 30 33 66 35 66 37 66  31 31 64 35 30 61 33 61  eyToken=b03f5f7f11d50a3a
000001B4 05 01 00 00 00 15 53 79  73 74 65 6D 2E 44 72 61  77 69 6E 67 2E 42 69 74  ......System.Drawing.Bit
000001CC 6D 61 70 01 00 00 00 04  44 61 74 61 07 02 02 00  00 00 09 03 00 00 00 0F  map.....Data............
000001E4 03 00 00 00 05 4B 00 00  02 FF D8 FF E0 00 10 4A  46 49 46 00 01 01 01 00  .....K...ÿØÿà..JFIF.....    0000014C                             FF D8 FF E0 00 10 4A  46 49 46 00 01 01 01 00           ÿØÿà..JFIF.....
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
000001FC 60                                                                         `                        =  0000015B 60                                                                         `
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
00004CE2 40 05 14 51 40 05 14 51  40 05 14 51 40 1F FF D9  0B                       @[email protected]@..Q@.ÿÙ.        <> 00004C41 40 05 14 51 40 05 14 51  40 05 14 51 40 1F FF D9                           @[email protected]@..Q@.ÿÙ
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Looks like a missing type header, which I bet is a result of binary serializing the object. I think the resx contains only the payload data, since it also has type information elsewhere in the XML.

It's probably possible to reconstruct the payload header from the resx information directly (looks like a few length-delimited fields), but that might be more coupling with implementation details of BinaryFormatter than we care for.

We could of course do what we do on full framework: use ResXResourceReader to deserialize the type and ResourceWriter to serialize it again. Edit: But note that right now ResourceWriter on core doesn't support BinaryFormatter--that was lit up for reading in https://github.com/dotnet/coreclr/pull/20907 but not writing (https://github.com/dotnet/corefx/issues/26745#issuecomment-438342828).

[nguerrera]

I think we should avoid the deserialize, reseralize route because it's basically incompatible with cross-compilation.

The lengths involved in handling different classic runtime versions for generate resource today demonstrate this.

If we're really just a few header bytes away, let's ask corefx for an api that adds this envelope. I think that would address the coupling concern.

[danmoseley]

@nguerrera I haven't (re)read the entire thread - but is the only reason we're not following .NET Framework verbatim, that dotnet/corefx#26745 is not done?

@ericstj

[nguerrera]

@danmosemsft I don't think so. Using the exact same code as full framework msbuild (is that what you mean by following .NET Framework verbatim?), actively deserializes objects in the build only to reserialize them. I really think we should not do that. At some point, I suspect that even the full framework msbuild should avoid this. With the current approach, we cannot ever have data in a .resx that could only be used at runtime on .NET Core.

Another issue with using the exact same code is that it would introduce a dependency from MSBuild -> Windows Forms, which we shouldn't have anywhere IMHO, and can't have x-plat. We would have to move the RexXResourceReader down and make it portable.

[danmoseley]

@nguerrera that makes sense. I am on the lookout for BCL work necessary for this. It sounds like we haven't identified any remaining BCL requirements here, right now.

[nguerrera]

@rainersigwald was going to set something up to discuss the next steps.

I don't think we can say that we don't need bcl changes for this. We just don't know exactly what we'll need yet. Even if avoiding deserialization works out, we may need API to write the blobs with appropriate header, etc.

And if we don't avoid it, then we do need at least the above work item for ResourceWriter and maybe moving ResxResourceReader out of winforms.

[danmoseley]

OK, let me know.

[vanbukin]

any ETA on this?

[michalss]

Aslo would like to know when this will be fixed ?

[lellid]

This issue is seriously keeping me porting my existing wpf application to dotnet core 3!

My proposal here: in a .resx file, support only i) strings (is already working) and ii) binary data.
Thus everything else but string will be treated as binary data (images, other files etc.). ResourceMananager.GetObject can then deliver the data as byte array, and GetStream() as stream.

Delivering as GetObject() also works already, but I have to rename my image files from e.g. image.png to image.xyz, in order to get them embedded as binary.

But make it work now!

[lellid]

Update: by chance I found, that the behavior that I proposed above is already implemented, but only if you have the .resx file in a net core console app (I tested it with netcoreapp2.2).
On the other hand, if I try to embed an .png image in a .resx file that is inside a netcore3.0app wpf application, the resx editor try to embed the .png file not as binary, but at System.Drawing.Bitmap.

[vanbukin]

@lellid Have you build your app from VS or from dotnet sdk cli? VS use full msbuild and can build resx, but msbuild that ships with dotnet core sdk is not.

[lellid]

In both cases (console app and wpf app) I built it with VS2019RC3.

[lellid]

But what I think is this: it is not an issue with the build process. It is an issue with the resx editor of VS2019. In the case of the console netcore app, if a add a .png file in the .resx editor (Add resource -> Add existing file), the resx file afterwards looks like this:

  <data name="Icons.16x16.ArrangeBottom" type="System.Resources.ResXFileRef, System.Windows.Forms">
    <value>Icons.16x16.ArrangeBottom.png;System.Byte[], mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
  </data>

If I add the same file in the resx editor (Add resource -> Add existing file) in a Wpf netcoreapp3.0, the .resx file looks like this:

  <data name="Icons.16x16.ArrangeBottom" type="System.Resources.ResXFileRef, System.Windows.Forms">
    <value>Icons.16x16.ArrangeBottom.png;System.Drawing.Bitmap, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
  </data>

See the difference? While in the first case the file is embedded as byte[], in the latter case it is embedded as System.Drawing.Bitmap.

[fraher]

As all roads lead here, I found no better place to add a quick set of workaround steps for those attempting to log a TRX file and find resource related tests failing in the CLI. This is not a solution to the issue though.

Open the Solution in Visual Studio
Set the Project to the desired configuration "Debug|Release"
Clean the Solution
Build the Solution
Ensure all Unit Tests Pass in Visual Studio
Execute the CLI command on the appropriate config but do not allow the project to build(e.g dotnet test --no-build --logger "trx;logfilename=filename.trx")

This ensures the resource files have been copied, and will leave them there for the CLI execution.

Hope this helps someone.

[AraHaan]

I get this when the project is of the new sdk style *.csproj format and when it only targets net472for some reason when I do (Icon)resourceMan.GetObject("icon name", localization);. It always happens even after a little while after building within Visual Studio. I am not sure but it seems GetObject sometimes returns a string when the respurce is in fact a icon.

[rainersigwald]

It always happens even after a little while after building within Visual Studio.

Can you expand on that a little? Do you _ever_ build with dotnet CLI? That behavior should not happen in Visual Studio or msbuild.exe-built bits (but they might not overwrite things if the last build was from dotnet build and things appear incrementally up to date.

(I'm working on the real fix here)

[AraHaan]

Yes, I from time to time use dotnet build --configuration Release --ignore-failed-sources when I feel too lazy to open up Visual Studio.

[rainersigwald]

@AraHaan if you can arrange to have MSBuild on your path (for example by using the Developer Command Prompt for Visual Studio 2019), you should be able to use msbuild -restore -p:Configuration=Release instead, without hitting this problem.

(Again, fixing this, just trying to help now)

[AraHaan]

Alright cool, is there also a way for me to compile or test these fixes locally for me and verify that it works? As long as it is resolved I have no issue. Also if only the installer could list the vs developer prompts like how the ps and cmd are listed on clean os installs for a given folder. Basically so I can click on the top corner of the folder and -> Open vs2019 developer command prompt.

[mark-raymond]

Is there an ETA for this?

[jnm2]

@mark-raymond I think it's known that it's essential for Windows Forms on .NET Core, so I'm expecting the ETA to be the same as the ETA for .NET Core 3.0. (@rainersigwald Is this right?)

[markusschaber]

Let's hope it won't be windows only then (as WinForms will be.)

[dasMulli]

See related https://github.com/dotnet/corefx/issues/37041 and https://github.com/microsoft/msbuild/pull/4420

initial deploy in 16.3p1/SDK 3.0.100-p7

There was live streamed discussion available at https://www.youtube.com/watch?v=5heMq4U2ek8

[rainersigwald]

That's correct, but with one wrinkle: we didn't make it into 16.3p1, so there will be a difference in your assemblies between code compiled with MSBuild.exe/Visual Studio, and that compiled with dotnet {ms}build for 16.3p1/3.0.100-p7. Both should work, but only the dotnet build-produced binaries will fully adopt the new mechanisms. This will be fixed for p2/p8.

I'm going to close this since the initial implementation is complete. After trying your scenarios with dotnet build in p7, please file new issues about any problems you encounter with non-string resources!

[voltcode]

This fix should be backported to MSBuild 15 at least and .NET Core 2.1/2.2, resource files are a very old feature, lack of support for them prevents people on .NET FX from migrating onto .NET Core.

[nguerrera]

@voltcode I'm afraid adding support for something that never worked in that combination would not be allowed for a servicing change. We keep a very high bar for that. Taking big features like this backwards adds too much risk. This was a very large change across three repositories. We have fixed this for .NET Core 3.0+ and MSBuild 16.3+, and that is all we can do without a time machine.

[voltcode]

What do you mean it never worked? It worked in Visual Studio. This is a big inconsistency, resource files are from year 2003 and you're forcing people to move to VS 2019 (paid upgrade) to fix it.
Unless NET Core 3.0 will be supported in VS 2017?

[nguerrera]

It is not supported in VS 2017. There has been a lot of work in msbuild, NuGet, the project system, and other VS components for .NET Core 3.

[jnm2]

@voltcode What I've been doing is using Visual Studio's .NET Framework msbuild which has always supported non-string resources rather than .NET Core's msbuild which had never supported them. I used this starting with VS2017:

$visualStudioInstallation = & "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe" -latest -products * -requires Microsoft.Component.MSBuild -property installationPath
$msbuild = Join-Path $visualStudioInstallation 'MSBuild\15.0\Bin\MSBuild.exe'
& $msbuild path\to\project\folder /restore /p:Configuration=Release /v:minimal /nologo

[nguerrera]

What do you mean it never worked?

I mean .NET Core 2.x projects with .NET Core SDK 2.x behave as they always have. It is regrettable that they did not have the result of this work, but it is simply too late to change that.

.NET Core 2.x filled many gaps with .NET Framework vs. .NET Core 1.x, but not this one. We didn't backport all of those gap closings to 1.x either. We literally cannot do that. We have to defend every backport rigorously, and the change will be rejected if it is closing a gap that was always there.

We will get tenfold more passionate feedback when we break something in a servicing update. We are holding this bar with the best of intentions. The risk is simply too high.

[voltcode]

thanks @jnm2 . Just to clarify by .NET FX you mean the real, old school MSBuild.exe, and by .NET Core msbuild you mean dotnet build command?

When writing build scripts, I would like exactly the same behavior between script and VS. To achieve this goal - what is better to use for scripts - MSBuild.exe or dotnet build ?

We hit the resources problem when using dotnet test in build scripts - as far as I understand - if we use MSBuild.exe to build entire solution and dotnet test with --no-build flag - we should be fine? We're using .NET Core 2.2.

[rainersigwald]

Visual Studio uses MSBuild.exe to build. Using that will be closer to the VS build.

Yes, using MSBuild.exe followed by dotnet test --no-build sounds reasonable in your use case.

[voltcode]

Thanks @rainersigwald, we've tested your suggestion and it we're definitely going to stay on MSBuild.exe and avoid dotnet * tools as much as possible to replicate VS behavior.

[jzabroski]

EDIT: * IF YOU ARE RUNNING INTO THIS ERROR WHEN THEN READ THIS - YOUR BUILD SERVER'S "BUILD TOOLS FOR VISUAL STUDIO 2019" MAY NOT BE UP TO DATE *

I am able to successfully run my app from my command line on my local dev machine, but cannot run it after it's being built in TeamCity.

Local Dev Machine

 & "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Current\Bin\M
sBuild.exe" --version
Microsoft (R) Build Engine version 16.2.37902+b5aaefc9f for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

MSBUILD : error MSB1001: Unknown switch.
Switch: --version

For switch syntax, type "MSBuild -help"

TeamCity Build Server

Visual Studio Build Tools 16.1.9

When I click "Launch", I get:

**********************************************************************
** Visual Studio 2019 Developer Command Prompt v16.0
** Copyright (c) 2019 Microsoft Corporation
**********************************************************************

Attempt to fix

1. Go to https://visualstudio.microsoft.com/downloads/#build-tools-for-visual-studio-2019
2. Download "Build Tools for Visual Studio 2019" (vs_buildtools__1704816089.1574373004.exe)
3. Get file SHA256 hash with PowerShell: 7DA4388648C92544C97407C6F052FD6BC0317DB407CADAB9FDCB328A34D3E317

PS C:\Users\john.zabroski> Get-FileHash "C:\Users\john.zabroski\Downloads\vs_buildtools__1704816089.1574373004.exe"

Algorithm       Hash                                                                   Path
---------       ----                                                                   ----
SHA256          7DA4388648C92544C97407C6F052FD6BC0317DB407CADAB9FDCB328A34D3E317       C:\Users\john.zabroski\Downlo...

1. Install the exe
2. Running the installer indicates it is 16.3.10