Msbuild: MSbuild on .NET Core can't parse properties with commas in the values

[danmoseley]

Steps to reproduce

This is on Ubuntu.

dan@danmose2:~/dotnetU$ dotnet msbuild /p:a="b,c"
Microsoft (R) Build Engine version 15.5.179.9764 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

MSBUILD : error MSB1006: Property is not valid.
Switch: c

Expected behavior

It accepts /p:a="b,c" as defining a property named a with value b,c.

Actual behavior

It errors as above . On Windows it works fine.

On Unix if I remove the comma and use a space it is fine:

For switch syntax, type "MSBuild /help"
dan@danmose2:~/dotnetU$ dotnet msbuild /p:a="b c"
Microsoft (R) Build Engine version 15.5.179.9764 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  dotnetU -> /home/dan/dotnetU/bin/Debug/netcoreapp2.0/dotnetU.dll```

Environment data

msbuild /version output:

dan@danmose2:~/dotnetU$ dotnet msbuild /version
Microsoft (R) Build Engine version 15.5.179.9764 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

15.5.179.9764dan@danmose2:~/dotnetU$

OS info:

dan@danmose2:~/dotnetU$ uname -a
Linux danmose2 4.4.0-43-Microsoft #1-Microsoft Wed Dec 31 14:42:53 PST 2014 x86_64 x86_64 x86_64 GNU/Linux
dan@danmose2:~/dotnetU$ cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.3 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.3 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial
dan@danmose2:~/dotnetU$

If applicable, version of the tool that invokes MSBuild (Visual Studio, dotnet CLI, etc):

dan@danmose2:~/dotnetU$ dotnet --version
2.1.3

cc @erozenfeld

[danmoseley]

Eugene hit this when trying to pass a parameter /p:ILLinkArgs="-h LdtokenTypeMethods,InstanceConstructors"

[ghost]

Similar issue is https://github.com/Microsoft/msbuild/issues/471

The workaround is to escape the surrounding quotes dotnet msbuild /p:a=\"b,c\"

BTW, dotnet msbuild /p:a="b,c" fails for me on Windows as well..

[danmoseley]

@kasper3 you're right, I was trying full framework MSBuild and that worked. dotnet msbuild has the same problem on Windows:

C:\dotnet7\1>..\dotnet.exe msbuild /p:a="a,b"
Microsoft (R) Build Engine version 15.6.54.9755 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

MSBUILD : error MSB1006: Property is not valid.
Switch: b

For switch syntax, type "MSBuild /help"

Updated title. My guess is that perhaps a bug has crept into the MSBuild command line parsing since the full framework MSBuild was last built, or perhaps it's something in the dotnet.exe parsing.

[rainersigwald]

Command line parsing is sadly different between .NET Core and full framework MSBuild, but there's another wrinkle in this case: shell escaping rules are very different on different platforms.

raines@raines-z220u:~$ cat show-command-line.sh 
#!/bin/sh
xargs -0 printf '%s\n' < /proc/$$/cmdline
raines@raines-z220u:~$ ./show-command-line.sh -foo "x" y="w,z" escaped=\"quotes\" 'singlequoted="doublquotes"'
/bin/sh
./show-command-line.sh
-foo
x
y=w,z
escaped="quotes"
singlequoted="doublquotes"

Since the invoked process can't see the quotes, I don't think we can do better there.

[danmoseley]

It's odd then that the same problem for some reason occurs on Windows.

In your example case, you can still assume [^=]+=.+ indicates a property and value. The quotes have done their job, and then been stripped:

dan@danmose2:~/dotnetU$ ./showcmd.sh /p:aaa=b,c  d  /foo
/bin/sh
./showcmd.sh
/p:aaa=b,c
d
/foo
dan@danmose2:~/dotnetU$ ./showcmd.sh /p:aaa="b,c  d" /foo
/bin/sh
./showcmd.sh
/p:aaa=b,c  d
/foo

[rainersigwald]

Yeah, the problem is that the comma is syntactically relevant in a /p: parameter

  /property:<n>=<v>  Set or override these project-level properties. <n> is
                     the property name, and <v> is the property value. Use a
                     semicolon or a comma to separate multiple properties, or
                     specify each property separately. (Short form: /p)
                     Example:
                       /property:WarningLevel=2;OutDir=bin\Debug\

So we're parsing the string in a reasonable way, but then splitting on the comma and complaining that you didn't provide a value for the c property. In other words we're treating /p:a="b,c" as equivalent to /p:a=b /p:c and complaining about the latter.

Also and unimportant: if you specify ```sh-session you get syntax highlighting for unix shell sessions that helps see what's command and what's output.

[danmoseley]

Ha, I never realized a comma was valid there.

So with quotes gone, you have to decide whether /p:a=b,c=d was originally /p:"a=b,c=d" (one property a with value b,c=d) or was originally /p:a="b",c="d" - or no quotes at all (meaning two properties). If the ambiguity only exists if you have two = values it's probably pretty safe to pick the first one, and the second one always has a workaround (use /p:a=b /p:c=d)

Neither are this case though -- there is only one = so presumably you can unambiguosly treat everything after that as the property value.

Note I discovered a workaround; MSBuild escaping works here. Comma is %2c so

dan@danmose2:~/dotnetU$ dotnet msbuild  /p:"aaa=bbb,ccc" /v:diag | grep "ccc"
Switch: ccc
dan@danmose2:~/dotnetU$ dotnet msbuild  /p:"aaa=bbb%2cccc" /v:diag | grep "ccc"
/usr/share/dotnet/sdk/2.1.3/MSBuild.dll /Logger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,/usr/share/dotnet/sdk/2.1.3/dotnet.dll /m /p:aaa=bbb%2cccc /v:m /v:diag ./dotnetU.csproj
                   aaa = bbb,ccc

[rainersigwald]

I didn't either; I was getting geared up to say that this is obviously somehow different from #471 since a semicolon is significant there. Glad I double checked!

Escaping the comma/semicolon is a beautiful workaround. I'm going to pull it into #471 and mark this as a duplicate.

Duplicate of #471.

[ghost]

Another fix is https://github.com/dotnet/buildtools/pull/1917. With "$@" in Unix shell, it preserves spaces as well as quotes, comma etc.

[rainersigwald]

@kasper3 I don't think that helps for comma/semicolon, because the problem isn't escaping the argument so that MSBuild gets it as a single string, but what we do with it after we have it.

[vikasillumina]

@danmosemsft Your this comment:
Note I discovered a workaround; MSBuild escaping works here. Comma is %2c
saved me time!!

[Piedone]

Could you follow up under https://github.com/NuGet/docs.microsoft.com-nuget/issues/1948, please, with the definite syntax to pass multiple values to /p on Windows with dotnet build?