Runtime: Add System.Security.Cryptography.Xml.SignedXml class

[ghost]

Execution plan

Goal: Provide APIs fully compatible with full/Desktop .NET Framework (no changes as part of this work - straight port only)

Plan:

• [x] 1. Add the source code on GH sanitized, with licenses, etc. (it won't build)
• [x] 2. Make the source code build (still excluded from overall repo build)
• [x] 3. Remove Desktop registry-compat code paths
  
  
  
  • Remove methods that have [RegistryPermission] (Utils and SignedXml classes) along with any owned methods
  
  
  • Address any other registry-related compile errors such as those using Registry, RegistryPermission, RegistryKey, etc (delete code as necessary)
  
  
• [x] 4. Add tests (We have to agree on expected code coverage and how much of the spec we have to cover)
  
  
  
  • Compare test results between Desktop and .NET Core implementations
  
  
• [x] 5. Make it part of overall repo build & ship it!

Code changes rules: Only code changes absolutely necessary to make it build and compatible with (full) .NET Framework are allowed, no additional bug fixes or changing architecture - such PRs will be rejected now.
Changes like that can be considered after the initial port is done, when we have a good test bed and when we are able to validate such change.

If larger code/architecture changes are needed from some reason, we should discuss them first here, before doing the work / submitting PR.

If you work on some parts of the plan, please say so in the discussion to avoid duplicated work. We (@steveharter @karelz) will co-assign the issue to you.

---

Original

The class to digitally sign XML needs to be added.

[rschiefer]

As referenced by Tratcher, this is a blocker for adding support for WsFederation/ADFS in ASP.NET 5. We use ADFS extensively for many enterprise ASP.NET 4 applications. We are very interested in migrating to ASP.NET 5 and using WsFederation.

[bartonjs]

@rschiefer @Tratcher Well, it's... complicated.

• ADFS doesn't use System.Security.Cryptography.Xml.SignedXml; but instead its own implementation.
  
  
  
  • (That's largely from dusting off mental cobwebs and remembering back to being on that team for version 1 of ADFS)
  
  
• System.IdentityModel definitely doesn't use System.Security.Cryptography.Xml
  
  
  
  • (That's 'cuz their source code on referencesource says so :smile:)
  
  
• People don't really like System.Security.Cryptography.Xml.SignedXml because it's based on XmlDocument, which leads to some perf issues.
  
  
  
  • The ADFS version was based on XmlReader, IIRC.
  
  
• So, likely what people want is CoreFX to create a new implementation of SignedXml.
• But a new version of SignedXml won't compatible with the old version of SignedXml
• So some people might want us to keep the old version, too.
• So in all, people really want two versions.
• Except they don't want the old version.
• Except for when they do.

So we're left with the conundrum of:

• We can port that class that no one really wants
• Or, we can stop and design a brand new thing that probably no one will use, since they can't be compatible with anything else
• Or, to use the most effort and still not benefit anyone, do both :smile:.

@terrajobst - fyi

[bartonjs]

Apparently I was a bit rambly this morning. Sorry :).

We've definitely identified that there's work to be done here, but we don't think the right answer is to bring forward the existing System.Security.Cryptography.Xml code. Instead, this represents the item on our backlog to engineer a general-purposed implementation for XmlDSig which is fast and not tied to legacy object models (e.g. XmlDocument).

That effort isn't something that we're expecting to do for .NET Core 1.0, simply because we're focused on other things.

But, letting us know that you're impacted by the missing functionality does help with ongoing prioritization.

[AndersAbel]

I'm looking at creating an ASP.NET Core SAML2 Middleware, based on https://github.com/KentorIT/authservices. Without a SignedXml port it won't be able to run on the Core framework.

I definitely do agree that not carrying over the existing one is a good idea. Would it be possible to do anything based on an XmlReader API? That way both XDocument and XmlDocument can be supported. Also providing an implementation of the enveloped reader used in System.IdentityModel would be nice (if it was improved to support XML files with whitespace...)

[bartonjs]

@AndersAbel XmlReader is where I'd start, yeah (unless the XML guys tell me there's something better).

Since XmlReader what the System.IdentityModel variant is based on, it should be doable :).

[AndersAbel]

@bartonjs The System.IdentityModel variant is quite limited though in what transforms it can handle. For SAML2/WS-Fed work it wouldn't be a problem, but as a general API it must be considered how to deal with non-enveloped signatures and xml containing nested signatures (such as a signed saml response containing signed assertions). Also I think that the System.IdentityModel.EnvelopedSignatureReader is copying the data when doing the validation. There's a lot of fun to be done there. If I had the time I'd love to work on it.

[Icestorm0141]

I appreciate the rambling @bartonjs, helps to see whats going on behind the scenes.

Currently my company is impacted by this. We have some legacy code that we're trying to port over to .NET Core that generates signed XML license files and without this set of classes we're stuck. We're open to diverging from XML files as the base for the licenses but as of this moment we haven't found a good solution that fits our needs.

Looking forward to seeing this get added in the future.

[sandersaares]

I can attest that this (and also the slightly related XML Encryption) is relevant for us. The existing form in .NET Framework would be just fine - no need for innovation here, from my perspective. Copy & paste implementation would be very welcome!

Is there currently any workaround available?

[af0l]

Would like to know what @sandersaares asked, too. There's no built in way to sign xml in the CoreFX now?

[bartonjs]

@sandersaares / @af0l : There is not, for .NET Core 1.0, any inbuilt implementation of SignedXml/XmlDSig.

Based off of comments here (and others) we'll probably just bring over the old API, but we didn't have time to make it happen for 1.0.

[af0l]

Thank you @bartonjs, that must be the reason I could not get our project to work on Core. :) It's also a great shame, because I'd love to move ahead and can't until it's done. We have to report all payments to our tax authority using signed xml's, so it's a showstopper really.

[emailtowalter]

Any progress on this? I am stuck with the SAML token validation which requires this feature. Thanks

[af0l]

Yeah, would liek to know that too. We ended up extracting the features needing the signature and putting them into a separate web API solution...

[john182]

Already have idea on which version will be implemented or solution

[bartonjs]

At this point it seems that the most straightforward answer is to port the existing .NET Framework implementation to .NET Core. So we're bucketing this in with other "making it hard to port" API omissions.

[sandersaares]

Potentially relevant to the topic: https://connect.microsoft.com/VisualStudio/feedback/details/3002812/xmldsigc14ntransform-incorrectly-strips-whitespace-and-does-it-inconsistently and https://github.com/sandersaares/xml-c14n-whitespace-defect. It would seem to me that the .NET Framework implementation of Canonical XML 1.0 is incorrect. I hope I am wrong but if so, this might introduce some hairy compatibility questions.

[AndersAbel]

@sandersaares Looked at your sample and you need to set XmlDocument.PreserveWhiteSpace = true when reading the Xml if it contains whitespace.

[sandersaares]

@AndersAbel Thanks for the hint! That changes the situation and actually enables conforming validation if an XML Schema is present. Without an XML Schema, behavior remains invalid (in a new and exciting way). I have updated the Connect issue and GitHub repo accordingly.

[AndersAbel]

@bartonjs If you port the existing .NET framework solution over, please fix the bug that prevents verification of multiple signatures: https://connect.microsoft.com/VisualStudio/Feedback/Details/2288620

[sandersaares]

FYI if this gets to implementation stage then I have a freshly minted library here, with tests, that makes use of XML signatures (both sign and verify) and other XML functionality on .NET Framework - might be useful to get some dependency-free real-world code to try out the implementation against: https://github.com/Axinom/cpix

[henkmollema]

Is there a timeline for development of this API?

//cc @bartonjs

[bartonjs]

@henkmollema Nothing specific, no. For the 1.2 release we're working to shrink the gap between .NET Framework and .NET Core; and that effort is currently where SignedXml is coming from.

[brockallen]

I was on a customer call today that needs SAML2-P support on ASP.NET Core (which would be using KentorIT's implementation). This is a blocking issue for customers that want to move to ASP.NET Core. For now my customer will have to stay on Katana.

[brockallen]

I see the milestone was changed from 1.2 to Future (by @bartonjs). Can you comment or elaborate?

[bartonjs]

Mainly it just has to do with how we're tracking milestones. We used to do more of "we hope it'll make this one", then at the end of the milestone we'd re-assign everything that wasn't done. We're now trying really hard to say "if we marked it for this milestone it should be very rare for it to get reassigned".

This is downstream of a lot of other work for the 1.2 milestone, and it was (to me, anyways) always a bit of a stretch for it to make it for 1.2. It's still pretty high up on our "what's next" list, we're just not 'committing' to it being part of the 1.2 release (which is mainly netstandard2.0 work, bugfixing, and a couple of infrastructure projects).

Marking it as Future doesn't guarantee that it won't be part of the 1.2 release, it just means (using our current interpretation of milestone) that we won't hold the release for it. Once someone is actually working on it then there's a better understanding of when it'll actually get done, and then it'll get marked as whatever milestone it'll actually release in.

@karelz If there's anything you want to add (or correct) feel welcome to chime in.

[karelz]

We won't be able to fund the work in 1.2 timeframe (there's just too much other stuff with higher impact to finish). that's why we moved it to Future milestone, to communicate our plans.
We are aware of the number of asks, that's why it is high in our Security area backlog. It is also one of the top asked corefx missing APIs in general (among DirectoryServices, SerialPort, etc.).

cc: @steveharter @danmosemsft @terrajobst

[karelz]

Our answers crossed :)
More context on Milestones is in our issue guide.

The .NET Framework code is available as reference source. So technically speaking the port can be initiated even outside of .NET Core team - if folks are interested to help us here.
From my earlier chats with @bartonjs I think the key "challenge" will be creating/porting tests.

[Jaedson33]

Hey, what's the actual situation about that problem?

[karelz]

@Jaedson33 what do you mean by 'problem'? If you mean when it will be fixed -- see the answers from last week.

[Jaedson33]

@karelz But I don't want to wait. Why you don't fix it now?

[karelz]

@Jaedson33 see my reply above - it explains why we cannot fund it now. It is about priorities. There is a line of people who want many features/APIs now, but we do not have infinite-size team, so we have to prioritize.

If you need it really badly ASAP, you can always contribute to the code base, we will be happy to help out with guidance, code reviews and feedback.

[Jaedson33]

Ok, so I'll wait.

[tintoy]

@karelz if the original tests are still available to verify the work, I'd be willing to put my hand up :)

(one of my coworkers also has relevant experience so we'd probably be working on it together)

[karelz]

Key part of the work is actually to create new test assets. The old tests are insufficient and have poor coverage. We will need someone to review the spec and add test for every interesting requirement -- that's where most of the cost is.

If you are still interested, we can try to drop in source code from full .NET Framework as is - the next step will be to get it built and add test coverage, before it can be released as part of .NET Core. Let me know if you're interested ...

[tintoy]

Ok, yes - we're still interested :)

[Jaedson33]

@tintoy I'm interested to help you because I really need that class.

[tintoy]

@tintoy I'm interested to help you because I really need that class.

Glad to hear it :)

[Jaedson33]

So... How can I help?
Obs: It's the first time I'm using GitHub.

[tintoy]

So... How can I help?

Let me first have a chat to my coworker and we'll come up with a plan of attack. @karelz - are there any guidelines or other docs we should read before wading in? To start with, I'm guessing my coworker will probably wade into the standard, I'll probably look at working out where the code needs to go (and what's involved in getting existing tests from other parts of the framework to run before we make any changes). Does that sound reasonable?

CC: @anthonylangsworth

[AndersAbel]

To keep the scope a bit limited I'd recommend to start without the features that are disabled by MS16-035 (xpath transform, xslt transform, external references). I don't know what room there is for breaking changes, but the current fallback mechanism in DefaultGetIdElement can be exploited in signature wrapping attacks. I'd prefer a more secure default version.

It would also be good if the internal API was restructured a bit to support the EnvelopedSignatureReader used by System.IdentityModel instead of having two separate implementations of XML Signature validation.

Finally I'd also like a single dot to be added as per this bug report.

[karelz]

@tintoy I don't think we have good docs. I think we should add the sources and then we can parallelize the work - let me sync with @bartonjs @steveharter @ianhays.

[AndersAbel]

I'll try to find some time to help too. If there are any questions on the spec and how it works I'll be happy to look into that - I've already spent some time reviewing the spec.

Does anyone has anything to say about the idea to consolidate SignedXml and the EnvelopedSignatureReader used by System.IdentityModel?

[karelz]

@AndersAbel

start without the features that are disabled by MS16-035 (xpath transform, xslt transform, external references)

We should start with latest .NET Framework source code, which should be secure. If you have any concerns about .NET Framework code security, please let us know offline.

I don't know what room there is for breaking changes

No room, we should start with straightforward port from .NET Framework. Any further improvements, changes, breaking changes, etc. can be considered later. Not as part of the initial work. Otherwise it will grow over our heads.

current fallback mechanism in DefaultGetIdElement can be exploited in signature wrapping attacks

That should be treated as separate problem. @bartonjs can you please comment?

It would also be good if the internal API was restructured a bit to support the EnvelopedSignatureReader used by System.IdentityModel instead of having two separate implementations of XML Signature validation.

Again, let's take it as next step, after we have fully functional port with good test coverage.

Finally I'd also like a single dot to be added as per this bug report.

Please file it as separate issue here, on GitHub. Ideally after we have the code here ported (i.e. when the bug is truly applicable to .NET Core).

Does anyone has anything to say about the idea to consolidate SignedXml and the EnvelopedSignatureReader used by System.IdentityModel?

Just want to reiterate. We should treat it as next step, post porting.

[bartonjs]

current fallback mechanism in DefaultGetIdElement can be exploited in signature wrapping attacks

That should be treated as separate problem. @bartonjs can you please comment?

@AndersAbel If you believe there is a security concern, please follow the vulnerability reporting process at https://technet.microsoft.com/en-us/security/ff852094.aspx.

Does anyone has anything to say about the idea to consolidate SignedXml and the EnvelopedSignatureReader used by System.IdentityModel?.

It likely isn't possible. SignedXml is very heavily (and public-API-ally) based on the rich-DOM XmlDocument. IdentityModel's representation is based on XmlReader. But, once the existing stuff is brought over it can be investigated.

[tintoy]

I'll try to find some time to help too. If there are any questions on the spec and how it works I'll be happy to look into that - I've already spent some time reviewing the spec.

@AndersAbel - cheers, I'm sure we could use the help :)

[AndersAbel]

@bartonjs I've reported those issues to [email protected], which resulted in MS16-035. IMO there are some remaining risky issues though, that MS decided to not fix (they would incur breaking changes). I've not yet published the details, but if you would like to discuss them privately, please mail me.

@karelz Thanks for clearing up that there's no room for breaking changes. That means that my ideas about consolidation are not relevant.

start without the features that are disabled by MS16-035 (xpath transform, xslt transform, external references)

We should start with latest .NET Framework source code, which should be secure. If you have any concerns about .NET Framework code security, please let us know offline.

The MS16-035 patch addressed a number of issues in SignedXml. It is possible though to use registry keys to revert to the old, unsecure behaviour. Should those options also be ported to .NET Core? My suggestion above was meant to prioritise getting the current default .NET Framework behaviour ported, leaving those parts that are deactivated by default behind for now. Or do you mean that those parts are crucial to be moved over too? Then there's the question how to handle the configuration as .NET Core AFAIK doesn't rely on the registry for configuration (as it's not available on all platforms).

[bartonjs]

It is possible though to use registry keys to revert to the old, unsecure behaviour. Should those options also be ported to .NET Core?

No. Registry-compat-only code will be deleted before the package is made available.

[Jaedson33]

Why you don't create a project on GitHub to implement that?

[karelz]

We synced with @bartonjs, @steveharter and @ianhays

EDIT: Execution plan moved to top most post.

[tintoy]

Sounds good to me :)

[AndersAbel]

@karelz, @steveharter Most registry lookups are located in the Utils class: AllowAmbiguousReferenceTargets, AllowDetachedSignature, RequireNCNameIdentifier. There's also a lookup in the SignedXml class where it setups the list of known transforms. Without the registry compatibility I don't think that the XmlDsigXPathTransform and XmlDsigXsltTransform can be accessed. Should they be removed completely from the source together with the registry-compat code?

That's those I know about, I've not seen any other while reading the code, but I might have missed something.

[steveharter]

@AndersAbel I did update Karel's comment above regarding registry. If there are any classes that are not accessible we need to understand what functionality is being lost. For the ones you mention, I believe CryptoConfig needs to add the name:object pair for those and thus can be created late-bound

[Jaedson33]

When do you think this class will be ready?

[karelz]

Do you mean for contributions? @steveharter plans to submit the initial "add sources" PR very soon (most likely today).

[steveharter]

The initial code has just been merged.

[Jaedson33]

@steveharter Thanks 😃

[karelz]

Thanks @steveharter! I have moved the execution plan to the top-most post for easier progress tracking. Whenever we make changes to it, we will mention the changes in another reply here.

[karelz]

If anyone wants to start working on it, please say so to avoid duplicated work. We will co-assign the issue to you.

[anthonylangsworth]

@karelz: @tintoy and I put our hands up to get started on this. Happy for you to assign it to us.

[tintoy]

If anyone wants to start working on it, please say so to avoid duplicated work. We will co-assign the issue to you.

Cheers - I'm happy to make a start on getting it to compile :)

[tintoy]

@anthonylangsworth - hah, beat me to it!

[tintoy]

Work's happening here.

[karelz]

Assigned to @tintoy. I will co-assign it also to @anthonylangsworth once he accepts the contributor status to the repo (GH won't offer you as assignee option without it).

[tintoy]

Thanks!

[tintoy]

@karelz just to confirm - from what I understand of the contributor guidelines I'm to make these changes in master?

[tintoy]

(my master obviously)

[tintoy]

Ergh, sorry, let me try again - the eventual PR to be against your master?

[karelz]

Yes, that's right. Just don't make it part of full corefx build/test run until the very last phase.

[tintoy]

I've found a couple of constants that seem to have been moved to an internal class in src/Common/src/Interop/Windows/Crypt32/Interop.certificates_types.cs. This isn't accessible from System.Security.Cryptography.Xml though. Any thoughts on the best approach here?

[karelz]

Which of them are needed? All of them?
Can you check reference source if they are public in .NET Fx? (I don't think so, but doesn't hurt to double check)
I am a bit surprised that we do special interop, instead of leveraging the rest of Crypto library ... either it needs something special, or it is from historical reasons ... @steveharter @bartonjs any thoughts?

[bartonjs]

@tintoy One of the things that needs to be done as this effort is removing the direct interfacing with CAPI, switching to using the .NET APIs.

[tintoy]

@karelz, @bartonjs - mostly CAPI HRESULT constants passed to the CryptographicException constructor.

For example:

src/System.Security.Cryptography.Xml/src/System/Security/Cryptography/Xml/KeyInfoX509Data.cs (line 63)

[tintoy]

I'll have a look at how other code in corefx works with CryptographicException.

[tintoy]

Ah, ok - so it looks like the HRESULT constructor's not used anymore - just the one that takes a message. I'll see if there are existing message resources that correspond to those E_xxx values.

[tintoy]

As for the other issues, it looks to me like a result of types not sharing a single assembly anymore. For example, X509Utils.DecodeHexString is in System.Security.Cryptography.X509Certificates but in the full framework this just lives in the System.Security assembly along with the classes that use it.

Since everything's been broken out into multiple assemblies what's your preferred mechanism for dealing with what would ordinarily be shared components? I could just make a copy of the required functions from code in other assemblies if that's what you'd prefer.

[tintoy]

Or pull in the source using something like:

<Compile Include="$(CommonPath)\Interop\Windows\Crypt32\Interop.certificates_types.cs">
    <Link>Interop\Windows\Crypt32\Interop.certificates_types.cs</Link>
</Compile>

[tintoy]

For now I've worked around the CAPI issue by simply compiling `Interop.certificates_types.cs into the assembly and referencing constants from there.

I also had to copy over some methods from X509Utils.cs in the full framework (mainly to do with Hex encoding / decoding) as there's nothing publicly available in corefx that does this.

The only remaining issues are in src/System.Security.Cryptography.Xml/src/System/Security/Cryptography/Xml/SymmetricKeyWrap.cs (line 34, among others) which result in errors such as:

error CA5350: TripleDESKeyWrapEncrypt uses a weak cryptographic algorithm TripleDES

For now, I've suppressed error. So now it all all compiles :)

Ok, well except for the test project:

corefx\Tools\PackageLibs.targets(34,5): error : Could not locate compile assets for any of the frameworks .NETStandard,Version=v1.7
corefx\Tools\PackageLibs.targets(34,5): error : Could not locate runtime assets for any of the frameworks .NETCoreApp,Version=v1.1

I'll work that out tomorrow.

[tintoy]

@karelz @bartonjs I'm going to open a PR so we can discuss the changes (the work is basically done as far as I can tell) - are you OK with that?

[karelz]

Sounds good to me. @steveharter any thoughts?

[Jaedson33]

Happy new year =D

[Jaedson33]

Do you have any idea when phase 2 will be completed?

[karelz]

dotnet/corefx#14662 is already merged - that was the phase 2. I will mark it in the top post as 'checked'.

[brockallen]

Just so I'm clear: once all 5 steps above are done, then to get ws-fed support in ASP.NET Core, the AAD team needs to do their SAML token bits, then the ASP.NET teams needs to build the ws-fed middleware on the AAD piece. Does that match your expectations?

[karelz]

Nope, this work has nothing to do with WS-Fed.
I owe a reply and explanation on https://github.com/AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet/issues/500

[karelz]

FYI: Reply posted: https://github.com/AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet/issues/500#issuecomment-275218749

[Jaedson33]

So when phase 4 will be completed?

[karelz]

If you're asking when .NET team will have time to get to it -- we don't know yet. It is high on our backlog, but there are couple of higher priorities we have to tackle first.

We are open to contributions in the space from the community in the meantime.

[tintoy]

I'm happy to keep working on this, but I'm kinda stuck at the moment; since corefx moved to the new build process, System.Security.Cryptography.Xml no longer builds (so @anthonylangsworth and I are blocked from writing any tests). If we could get just a quick pointer on what's involved in getting the project (and test project) to build, we'll be good to go :)

PS. I've spent 20 minutes or so tracing through the build process to work out why it no longer builds, but haven't worked that out yet. Any pointers would be appreciated...

[karelz]

@mellinoe @weshaggard can you please provide guidance for migrating SignedXml to the new build system?

[Jaedson33]

😭😭 I think I'll need to wait 😭😭

[weshaggard]

@tintoy if you point me at the branch you are currently working on and which project you are trying to build I can help with getting building.

[karelz]

@weshaggard there is currently not a branch - the code is in master, it just is not hooked up into root build (on purpose) - src/System.Security.Cryptography.Xml (introduced in dotnet/corefx#14628). @steveharter can provide additional info.

[tintoy]

@weshaggard @karelz I'm happy to create a branch in our fork and only get it building there to unblock us; later, we can always cherry-pick any changes we've made once it's building again in master. Let me know if this is your preferred approach :)

[weshaggard]

PR https://github.com/dotnet/corefx/pull/15491 should get the project infrastructure working. Once CI passes for it we can merge it and it should bootstrap your efforts.

[tintoy]

Thanks - I've rebased tintoy/corefx/master and will have a play with it shortly :)

[tintoy]

@weshaggard ok, so both src and test project build, but if I add a project reference from the test project to the source project (<ItemGroup><ProjectReference Include="..\src\System.Security.Cryptography.Xml.csproj" /></ItemGroup>), I get:

1>------ Build started: Project: System.Security.Cryptography.Xml, Configuration: Debug Any CPU ------
1>  D:\Development\github\tintoy\corefx\src\System.Security.Cryptography.Xml\src\System.Security.Cryptography.Xml.csproj ConfigurationErrorMessage: Could not find a value for TargetGroup from Configuration 'Debug'.
1>  System.Security.Cryptography.Xml -> D:\Development\github\tintoy\corefx\bin\AnyOS.AnyCPU.Debug\System.Security.Cryptography.Xml\netcoreapp\System.Security.Cryptography.Xml.dll
2>------ Build started: Project: System.Security.Cryptography.Xml.Tests, Configuration: Debug Any CPU ------
2>D:\Development\github\tintoy\corefx\buildvertical.targets(88,5): error MSB4018: The "FindBestConfiguration" task failed unexpectedly.
2>D:\Development\github\tintoy\corefx\buildvertical.targets(88,5): error MSB4018: System.ArgumentException: Property 'ConfigurationGroup' value 'Debug' occured at unexpected position in configuration 'Debug'
2>D:\Development\github\tintoy\corefx\buildvertical.targets(88,5): error MSB4018:    at Microsoft.DotNet.Build.Tasks.ConfigurationFactory.ParseConfiguration(String configurationString, Boolean permitUnknownValues) in D:\Development\github\tintoy\corefx\src\Tools\CoreFx.Tools\Configuration\ConfigurationFactory.cs:line 219
2>D:\Development\github\tintoy\corefx\buildvertical.targets(88,5): error MSB4018:    at Microsoft.DotNet.Build.Tasks.FindBestConfiguration.Execute() in D:\Development\github\tintoy\corefx\src\Tools\CoreFx.Tools\FindBestConfiguration.cs:line 34
2>D:\Development\github\tintoy\corefx\buildvertical.targets(88,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
2>D:\Development\github\tintoy\corefx\buildvertical.targets(88,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext()
========== Build: 1 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

I'm guessing I've missed something about how inter-project references are meant to work in corefx?

[weshaggard]

@tintoy I'm not sure what that error is specifically but we generally don't need ProjectReferences in our new engineering system. For the test build we always build and reference the full targeting pack, in this case produced in bin\ref\netcoreapp. The library that is put into that directory is what you build from your ref folder which is currently empty. So to get yourself going you need to generate a ref with the API surface area that you need and get the ref building, then your test project will automatically see the APIs and can build against them. We have a tool called genapi that can generate the ref based on another library. Let me submit another PR quickly to seed that for you guys.

[tintoy]

Ok, now I get it now; the reason I see no types in the test project is because the ref project has no types yet :-)

[tintoy]

@weshaggard if you don't have time, I can probably work out how to do that; I was just reading about that tool the other day.

[weshaggard]

I quickly ran the genapi tool and pushed that commit https://github.com/weshaggard/corefx/commit/29cf289f3e007fd4b13b191866ae848d99dec67e. Feel free to take that or regenerate it yourself. You will need to add ProjectReference's to other refs to get it to compile.

[tintoy]

Cheers - that'll save me time, much appreciated.

[tintoy]

@weshaggard success! Thanks, hopefully we can get onto writing those tests now :)

[tintoy]

(tintoy@dd834c63af4fe40faf84bc6a776b474ec9947eb1, just ignore the duplicate)

[tintoy]

Yep! Have a working test:

https://github.com/tintoy/corefx/commit/24864b3d25e665b6305f116890328527db07f1e1

Thanks for your help, @weshaggard!

PS. I had to locally override (via .user file) the executable path under the test project properties (Debug tab). Was D:\Development\github\tintoy\corefx\Tools/testdotnetcli/dotnet.exe but only worked when I changed it to D:\Development\github\tintoy\corefx\Tools\testdotnetcli\dotnet.exe.

[weshaggard]

D:\Development\github\tintoy\corefx\Tools/testdotnetcli/dotnet.exe

My VS doesn't complain about the slashes. The path separators are generally a pain because we are trying to make things work on both windows and unix, so we end up seeing a bunch of mixed slashes on windows which is generally more accepting then unix.

[tintoy]

Just out of curiosity, which version of VS are you using? 2015 or 2017? Perhaps it's been fixed in 2017 :)

[tintoy]

(I'm usually on OSX or Linux these days, so I totally appreciate the effort to make things cross-platform friendly, BTW)

[weshaggard]

I'm using VS 2015 on Windows 10 and opened the solution and could F5 to debug the tests and my debug path is D:\git\corefx\Tools/testdotnetcli\dotnet.exe

[tintoy]

Ok, I must be going crazy - now I can't reproduce it either!

Before, it complained about not being able to find dotnet.exe, but it started working when I explicitly set the executable to a path with backslashes only. I just removed the .csproj.user file and it _still_ works, so who knows :-o

[brian-pickens-web]

Hey guys, I'd love to get involved in helping to write tests. What can I do to get started contributing?

[tintoy]

Great - I think we might be at the stage now where we can start doing that :)

Let me check with a coworker to see if he's successfully run his first test...

cc: @anthonylangsworth

[tintoy]

@karelz,

@anthonylangsworth and I have started writing tests; I'm aware that we haven't reached common agreement on what needs to be tested but we're going to kick off anyway and then port the tests over to wherever we agree to do the work.

[tintoy]

And for those that have volunteered to help with the tests (which is hugely appreciated), we should be in a position to divide up the suggested work early next week :)

[karelz]

Regarding test coverage and which tests we need - @bartonjs had strong opinion on it (as he has most expertise with the type and its troubles on our side). He suggested to write tests by the spec (I mentioned it above).
He will be back from vacation at the end of next week. We should probably discuss details & expectations with him.
Also @AndersAbel mentioned spec expertise and potential help earlier in the discussion.

cc @steveharter if he has additional guidance while @bartonjs is OOF.

Thanks @tintoy @anthonylangsworth for your contributions here! We really appreciate it!
And also thanks to everyone else who plans to jump in ;-)

[jnm2]

cc @steveharter if he has additional guidance while @bartonjs is OOF.

My curiosity reached a point which demanded to be satisfied.
https://blogs.technet.microsoft.com/exchange/2004/07/12/why-is-oof-an-oof-and-not-an-ooo/
I feel better now.

[karelz]

😃 I didn't even know it's so Microsoft specific! Thanks for your funny enlightenment 😃

[tintoy]

@anthonylangsworth has started sketching out a rough test plan in tintoy/corefx#3 (I've copied his plan into an issue to make it easy to comment) so we at least have something to discuss. Feel free to have a look over it and provide feedback :)

CC: @karelz @steveharter @bartonjs

[anthonylangsworth]

@karelz @steveharter @bartonjs (or anyone it may concern) What is the policy around using the InternalsVisibleToAttribute for tests? There are a lot of internal classes within that namespace and getting adequate test coverage may be hard going solely via public methods. However, I appreciate there are other considerations.

[karelz]

Hmmm, sadly, I don't know - @weshaggard @stephentoub? (question in previous reply)

[tintoy]

From what I've seen of other code in corefx, the projects in question sometimes include the relevant source files from other projects (although I imagine this will get complicated pretty quickly due to dependency graphs).

[tintoy]

From memory, System.Collections.Immutable uses [InternalsVisibleTo], if that's any help.

[tintoy]

For example:

https://github.com/dotnet/corefx/blob/master/src/System.Collections.Immutable/src/Properties/InternalsVisibleTo.cs

[tintoy]

More generally:

https://github.com/dotnet/corefx/search?utf8=%E2%9C%93&q=%22assembly%3A+InternalsVisibleTo%22

[weshaggard]

You can see my thoughts on InternalsVisibleTo in this old issue https://github.com/dotnet/corefx/issues/1604. My general take is not to do it unless there is a real specific need to do it.

[bartonjs]

What is the policy around using the InternalsVisibleToAttribute for tests?

No.

There are a lot of internal classes within that namespace and getting adequate test coverage may be hard going solely via public methods.

If it's something other than a deep exception path it should be reachable, or deleted. If it takes a tricky test case to hit it, well, that's what we need.

projects in question sometimes include the relevant source files from other projects

Assuming you mean "the test projects include the product source to get access to internal members": No.

---

[InternalsVisibleTo] and source sharing are both inherited strategies. The most vociferous of us believe (essentially) that our tests should only be representative of things that could be encountered in the real world. If no test is possible to hit some particular block then why does it exist? Yes, there are some exception-throwing blocks that can't be hit because a redundant check higher in the stack already caught it; but that is something that can be looked at after the fact and declared OK.

So I'd recommend pulling up the spec and ensuring that there is a positive test for every feature (e.g. every algorithm it says it supports, and boundary cases for min/max values for options on those algorithms).

Negative tests like removing required elements (e.g. 4.1 says that SignedInfo is a required element for Signature... do we emit a reasonable exception if it's missing?) are also good.

Canonicalizer option tests, like <foo><!-- hi --></foo> and <foo></foo> (and maybe <foo />?) are the same under c14n, but different under c14n-withcomments. (This probably requires signing both ways, then swapping bodies, since the canonicalization algorithm should be signed).

Transform tests. All canonicalizers are transforms. Etc.

Tamper tests are also good. But if you believe you found a tamper test that successfully tampers, don't report it here or commit/push it to any repro on github (email the test/case/description to [email protected]).

---

Okay, I've thought too much for the day. Back on vacation now.

[karelz]

Thanks @bartonjs for your insights while on vacation! Now go and have some fun in real world ;-)

[tintoy]

@karelz @bartonjs (although only once you're back from vacation!) @steveharter and anybody else with an opinion:

@anthonylangsworth has created some initial tests as a discussion-starter, and we'd appreciate any feedback you may have so far.

[steveharter]

I see that Mono has some SignedXml tests. These should be evaluated and vetted against the xmldigsig spec , suggestions that @bartonjs mentioned earlier, and current code to see what\if they are worthwhile to port\bring over. Ping me for copyright (header) information if these tests make sense to bring over.

[tintoy]

Thanks - we'll have a look at that :)

[anthonylangsworth]

Thanks for that, @steveharter . Can you provide links, please? These could short cut a lot of our testing. Are there any copyright or other considerations if we expand or build on these instead of copying them verbatim?

[karelz]

@anthonylangsworth when copying source code from Mono, we have to keep & update the copyright header. I would suggest to first do just a copy (with the right headers, potentially minor tweaks). Once we have the code in CoreFX, we can modify the code as we wish.

[steveharter]

@anthonylangsworth Mono tests - https://github.com/mono/mono/blob/0bcbe39b148bb498742fc68416f8293ccd350fb6/mcs/class/System.Security/Test/System.Security.Cryptography.Xml/SignedXmlTest.cs

[anthonylangsworth]

Thanks, @steveharter. I have started converting the tests over from NUnit to Xunit.

[danmosemsft]

I posted this but realized it's in @anthonylangsworth 's repo:

Here's the magic to do to the copyright headers when we pull code from Mono:

1. Keep the existing copyright headers in place
    ○ Note: If you are porting just portions of the file over, you still need to bring over all the existing copyright headers.
2. Add the following copyright headers – NOTE this is missing the middle line of the ‘regular’ CoreFx copyright headers:
             // Licensed to the .NET Foundation under one or more agreements.
             // See the LICENSE file in the project root for more information.

[Jaedson33]

Are there any failing tests in this project?

[anthonylangsworth]

@Jaedson33 We are currently converting the mono tests. I have not found any failing tests that indicate code errors yet but we still have many tests to go.

[Jaedson33]

@anthonylangsworth What can I do to help?

[anthonylangsworth]

@Jaedson33 I have answered that question at https://github.com/tintoy/corefx/issues/6#issuecomment-280904587 . To summarize, we have 84 mono tests still failing (mainly due to changed defaults and .Net core limitations). Any help to get the remaining tests working is appreciated. Otherwise, I'm working through them.

[anthonylangsworth]

@karelz @bartonjs @steveharter The System.Security.Cryptography.CryptoConfig class states that many XML transforms are not supported on CoreFx (lines 281 to 303 at the bottom of DefaultNameHT).

These correspond to URIs used by classes in the System.Security.Cryptography.Xml namespace. As part of adding System.Security.Cryptography.Xml back into .Net Core, I assume we should reinstate these. Please let me know if I am mistaken.

CC: @tintoy

[morganbr]

@anthonylangsworth , some of those transforms, such as XmlDsigC14NTransform are fine, but others, such as XmlDsigXsltTransform are considered very dangerous. While you can enable them via registry key opt-in in the full .NET Framework, I'd rather not support them on .Net Core. Take a look at KnownCanonicalizationMethods and DefaultSafeTransformMethods at https://github.com/dotnet/corefx/blob/ac17228d823a07a15fe53069a49fb5e5f35835b7/src/System.Security.Cryptography.Xml/src/System/Security/Cryptography/Xml/SignedXml.cs#L741 for the transforms that we should support.

[morganbr]

I hadn't realized the source of dangerous ones actually got included in the port. I'd vote for removing them entirely. There's really no safe way to use them.

[anthonylangsworth]

@morganbr Thanks for the heads up. Once I get the RSA and DSA key value XML code changes done, I'll review the list of transforms included. I will likely publish a list and ask you and others to review it.

[morganbr]

@anthonylangsworth Sounds great!

[AndersAbel]

@morganbr @AnthonyDGreen Those transforms (that was disabled in the MS16-035 patch) have been discussed previously in this thread when discussing the port. @bartonjs stated on Dec 14 that reg-compat stuff should be removed. See also comment by @steveharter Dec 15 that those transforms probably can be enabled late-bound and should thus be ported.

[karelz]

@steveharter @bartonjs can you please chime in?

[steveharter]

Even with no registry support, CryptoConfig can create those transforms late-bound by string name or oid. Search for 'CryptoConfig' in the SignedXml code as it is used to create the appropriate class based on the xml content.

This means the CryptoConfig class should be extended to support these transforms, at least for the same types in netfx anyway, and ideally also cross-reference those against Mono. That is unless there is a reason (that I'm not aware of) to not include them and not hook them up to CryptoConfig..

FWIW here's the netfx version of CryptoConfig (in order to compare with what's in corefx): https://referencesource.microsoft.com/#mscorlib/system/security/cryptography/cryptoconfig.cs,20d26e036bc718bc

[bartonjs]

There's a list of "allowed transforms" which is (in .NET Framework) registry-extensible for back-compat. For .NET Core it's not extensible, but hard coded to include only the no-input transforms.

Since you can't validate a document using transforms not on the allow-list, maybe it doesn't make sense to port them. But since someone could be using the transforms outside of the context of SIgnedXml (just wanting to use them as a general purpose transform engine) maybe it's fine to have them.

Since we'd be talking about removing a whole type it wouldn't create an inconsistency with .NET Framework... so I'd probably advocate removing any transforms not already on the hard-coded allow list. "Remove before publishing the package" can be changed later. "Publish them" cannot.

[anthonylangsworth]

@bartonjs beat me to it. CryptoConfig contains a list of allowed transforms. I had to modify that class to allow the transforms in the new namespace, System.Security.Cryptography.Xml. While some of the allowed transforms use a .Net class's full name, CryptoConfig still only permits a fixed list.

I would prefer to not port transforms that have known security issues but backwards compatibility is also important. Should we be making this decision on behalf of the developer? Do we modify CryptoConfig to allow some form of extension so a developer can add new transforms if they choose? How do we reach a decision on this?

[bartonjs]

CryptoConfig isn't the limiting factor: https://github.com/dotnet/corefx/blob/ac17228d823a07a15fe53069a49fb5e5f35835b7/src/System.Security.Cryptography.Xml/src/System/Security/Cryptography/Xml/SignedXml.cs#L763-L787. (Though, if CryptoConfig is still being used as the factory for resolution then any transform would need to be in both places)

The types providing algorithms not in that list (which are not also canonicalizers) effectively have no value.

Allowing extension to the safe list would require new API, so is technically beyond the scope of this effort.

I'd, personally, leave out the transform types that can't be used in document verification. They're going to be hard to test.

[anthonylangsworth]

Actually, both CryptoConfig and DefaultSafeTransformMethods are relevant. SignatureDescription creation is done in CryptoConfig so, irrespective of the values in DefaultSafeTransformMethods, you cannot use a transform if it is not mentioned in CryptoConfig. DefaultSafeTransformMethods restricts that list so, if a transform is specified in XML but not returned by DefaultSafeTransformMethods, SignedXml.CheckSignature returns false.

In the current implementation. CryptoConfig.AddAlgorithm throws a PlatformNotSupportedException, so users cannot add their own. Beyond the scope of this porting effort but it may be worth looking at this or even adding a RemoveAlgorithm in the future.

After my previous comment, I found the property SignedXml.SignatureFormatValidator. This allows a user to specify what transforms and hash algorithms are appropriate. The caller can use this to override DefaultSafeTransformMethods, for example.

As I asked before, who makes the decision here? As a "security guy", my preference would be to exclude insecure options. However, I do not fully understand the extent of incompatibilities this could case.

[karelz]

@anthonylangsworth this discussion is part of the decision making. Area experts with the most background will make the decision here.

My recommendation: If it is questionable what is the right action, I would suggest to keep status quo - leave the code as is during the port exercise (potentially even without any test coverage) and decide later, separately from the port exercise.

[bartonjs]

Okay, so I chatted with some people internally and I think I have the scaffold of a plan:

• Leave the types as public. (They're public types, and removing things makes people sad).
• These can't (yet) be used in end-to-end SignedXml tests. (In fact, negative tests seem good)
• They do have public members which should be sufficient for unit testing, so we should do that.
  
  
  
  • And that goes for the rest of the transform types, too.
  
  
• If, for some reason, the input-accepting transforms don't work, and everything else is fine, we can figure out what to do about them. (This would have also applied to "if they had complex compilation dependencies that can't be met", but we're already past that hurdle).

And if/when API or other configuration comes along to enable these types to work, adding the opted-in E2E tests will be easy.

[krwq]

@tintoy @anthonylangsworth @peterwurzinger what's the status of your test branch? can we start merging it? I can cherry pick your tests and keep going from there.

Could you also PTAL at https://github.com/dotnet/corefx/pull/16545 ? I've enabled one of the MSDN samples

[tintoy]

Hi @krwq - I believe there are still some tests that are failing but since they aren't run as part of the regular build process yet you may be able to include them anyway.

Let me have a chat to @anthonylangsworth and get back to you.

[tintoy]

PS. dotnet/corefx#16545 -> :+1:

[tintoy]

@krwq - had a quick chat to @anthonylangsworth. He mentioned (and I agree) that given the amount of work that's been done there perhaps it'd be good to merge what we have before going any further. It builds, and most tests pass, but a few do not.

I suspect we'll need to rebase against dotnet/corefx#16545 (which I'll do) before opening a PR (which @anthonylangsworth will do).

We'll get back to you as soon as we're ready for this (hopefully not long).

[anthonylangsworth]

@krwq To expand on what @tintoy said, while there is still some work to go, a lot of work also has been done porting existing tests and expanding them. In particular, I have already changed CryptoConfig.cs to handle many of the algorithms you mention. We all want to move this forward. We also do not want to duplicate or overwrite each other's work. Therefore, we plan to merge the changes as is so others can build on top of the work we have done, find all our bugs and, hopefully, get everything into master sooner.

[krwq]

@tintoy is https://github.com/tintoy/corefx/tree/feature/xml-crypto/tests the only branch you are working on?

[tintoy]

Yep.

[peterwurzinger]

@krwq Well, there is a PR with a good amount of stuff from Mono in (which is also a sub-branch from https://github.com/tintoy/corefx/tree/feature/xml-crypto/tests ...). If you're interested in up-to-date code you should probably have a look there. As far as I can tell there are ~40/340 tests failing.

[tintoy]

Good catch, @peterwurzinger, I thought we'd already merged that one!

[krwq]

@peterwurzinger @anthonylangsworth this PR is pretty nice! I have in fact missed it. Will you be merging that to your branch, to corefx or do you want me to pick it up and do all the merges/rebases? Does it miss anything from Mono tests or is it a complete? @anthonylangsworth - for the CryptoConfig changes - I talked about that with @bartonjs - we should be not touching that file - it's already ugly enough.

My original plan was to get few samples from MSDN and make them all work so that we can start early dogfooding. After that I'll be merging and fixing tests from your branch. If you got some tests failing we can simply disable them for now and fix them later. Let's merge your stuff into corefx/master ASAP :smile:

[anthonylangsworth]

Thanks for the update, @krwq . If you can, please keep us up to date. It helps to know what we are aiming at.

[Jaedson33]

Hi =D

[Jaedson33]

Are there any failing tests?

[krwq]

@Jaedson33 @anthonylangsworth @tintoy
Here is the current list of (arbitrarily) most important issues:
https://github.com/dotnet/corefx/issues?q=is%3Aopen+is%3Aissue+label%3ASystem.Security.Cryptography.Xml+label%3A%22up+for+grabs%22

@Jaedson33 yes, they are currently disabled. Above should give you a rough idea where we are at

[Jaedson33]

Hi, do you have any idea when this will be without any error?

[krwq]

@Jaedson33 every piece of software has errors :wink: Are there any particular things which don't work for you?

[JaedsonBarbosa]

It's simple: It just don't work in my UWP project 😞

[JaedsonBarbosa]

Hi, I'm getting this error when I try to run build.cml. Can you help me?

Error in the command:
C:\Users\jaeds\Source\Tools\msbuild.cmd /nologo /verbosity:minimal /clp:Summary /maxcpucount /nodeReuse:false /l:BinClashLogger,Tools\net46\Microsoft.DotNet.Build.Tasks.dll;LogFile=binclash.log /p:ConfigurationGroup=Debug /p:BuildPackages=false /flp:v=normal /flp2:warningsonly;logfile=msbuild.wrn /flp3:errorsonly;logfile=msbuild.err C:/Users/jaeds/Source/Repos/corefx/src/Native/../../bin/obj/Windows_NT.x64.Debug/native\install.vcxproj /t:rebuild /p:Configuration=Debug /p:Platform=x64 /p:PlatformToolset=v141. => O sistema não pode encontrar o arquivo especificado
Command execution failed with exit code 1.
Failed to generate native component build project!
Command execution failed with exit code 1.

[krwq]

@JaedsonBarbosa Are you running tests from the developer command prompt? (btw this is slightly OT) for the UWP - I will be investigating if this can be done fairly cheap for 2.0 although no promises

[JaedsonBarbosa]

I'm doing that from the Developer Command Prompt for Visual Studio 2017

[krwq]

@JaedsonBarbosa have you have pulled the latest changes from github and try building after cleaning your repo (git clean -fdx)? Second thing to try is to reduce the path length (i.e. put your repo under C:\corefx). Another thing to try is to clean nuget cache (%USERPROFILE%\AppData\Local\NuGet and %USERPROFILE%\.nuget)

If still happening please create a separate issue with information:

• your OS
• your VS version
• exact steps you did
• full log (if big put it on gist)

[JaedsonBarbosa]

I think it occurs because I think the path C:/corefx/src/Native/../../bin/obj/Windows_NT.x64.Debug/native\install.vcxproj isn't valid.

OS: Windows 10
VS: 2017 Community
I just start the Developer Command Prompt for VS 2017 and put:

cd C:/corefx
build

Now the error is:

Error in the command: C:\Users\jaeds\Source\Tools\msbuild.cmd /nologo /verbosity:minimal /clp:Summary /maxcpucount /nodeReuse:false /l:BinClashLogger,Tools\net46\Microsoft.DotNet.Build.Tasks.dll;LogFile=binclash.log /p:ConfigurationGroup=Debug /p:BuildPackages=false  /flp:v=normal  /flp2:warningsonly;logfile=msbuild.wrn  /flp3:errorsonly;logfile=msbuild.err  C:/corefx/src/Native/../../bin/obj/Windows_NT.x64.Debug/native\install.vcxproj /t:rebuild /p:Configuration=Debug /p:Platform=x64 /p:PlatformToolset=v141. => O sistema não pode encontrar o arquivo especificado
Command execution failed with exit code 1.
Failed to generate native component build project!

"O sistema não pode encontrar o arquivo"="The system couldn't find the specified file"

[JaedsonBarbosa]

I give up, I can't make it work with the 2017 VS.
So do you have any idea of when this may be downloaded as a NuGet package?

[karelz]

You should be able to consume the preview from myget.org.
Official package will be part of the .NET Core 2.0 wave - see milestone 2.0.0 description, May 10 is the ZBB (#17619), so the RTW release should follow "shortly" after it (exact dates are not yet publicly disclosed)

[JaedsonBarbosa]

@karelz Can you please send me the link of the package that contains the System.Security.Cryptography.Xml?

[karelz]

https://dotnet.myget.org/feed/dotnet-core/package/nuget/System.Security.Cryptography.Xml

Here's how to dogfood 2.0: https://github.com/dotnet/corefx/blob/master/Documentation/project-docs/dogfooding.md

[JaedsonBarbosa]

@karelz OK, I want do install that in a UWP class library, but when I try that I get that error:

O pacote System.Security.Cryptography.Xml 4.4.0-preview1-25205-01 não é compatível com uap10.0 (UAP,Version=v10.0). O pacote System.Security.Cryptography.Xml
4.4.0-preview1-25205-01 dá suporte a:

• monoandroid10 (MonoAndroid,Version=v1.0)
• monotouch10 (MonoTouch,Version=v1.0)
• netcoreapp2.0 (.NETCoreApp,Version=v2.0)
• uap10.1 (UAP,Version=v10.1)
• xamarinios10 (Xamarin.iOS,Version=v1.0)
• xamarinmac20 (Xamarin.Mac,Version=v2.0)
• xamarintvos10 (Xamarin.TVOS,Version=v1.0)
• xamarinwatchos10 (Xamarin.WatchOS,Version=v1.0)

It's my actual project.json:

{
  "dependencies": {
    "Microsoft.NETCore.UniversalWindowsPlatform": "5.3.1"
  },
  "frameworks": {
    "uap10.0": {}
  },
  "runtimes": {
    "win10-arm": {},
    "win10-arm-aot": {},
    "win10-x86": {},
    "win10-x86-aot": {},
    "win10-x64": {},
    "win10-x64-aot": {}
  }
}

[krwq]

@JaedsonBarbosa, we currently do not support UAP for that library, you'll need to wait until https://github.com/dotnet/corefx/pull/17969 is merged and new package produced.

[JaedsonBarbosa]

😧 Ok, I'll continue waiting 😭
@krwq But... What's UAP10.1???

[JaedsonBarbosa]

@krwq Why you don't merge the PR dotnet/corefx#17969 now?

[krwq]

@JaedsonBarbosa It just got merged, I believe the package should be there tomorrow morning - we usually don't merge unless the PR is green on the CI - we got some OSX CI issues since yesterday so it got stale a little bit

[JaedsonBarbosa]

@krwq Can you let me know when the NuGet package with the changes can be downloaded?

[karelz]

@JaedsonBarbosa be warned that .NET Standard 2.0 support in UWP is bleeding edge -- it won't be part of .NET Core 2.0 -- it will be part of the next version, temporarily named as 2.1. We are working on some of the 2.1 stuff ahead of time and in parallel with 2.0, to stand up infrastructure, etc., so that we can then heavily parallelize (after May 10 which is our ZBB for 2.0).
Either way, you might encounter some issues in using the library in UWP. We might not be in position to help you out right away. We should be able to help more after May 10. ... just setting the expectations.

[krwq]

@JaedsonBarbosa seems like we didn't produce any new version for 2 days 😞 You can observe changes here:
https://dotnet.myget.org/feed/dotnet-core/package/nuget/System.Security.Cryptography.Xml
whenever a package appears after 4/6 it should have the change you need. I'll check today why there was no package - it could be related to networking issues we have with OSX builds

EDIT: confirming - this is related to OSX networking issues

[JaedsonBarbosa]

@krwq Do you now if today the OSX network issues will be solved?

[danmosemsft]

It is blocking almost all our PR's so its high priority but we don't have an ETA

[JaedsonBarbosa]

OK 👍
So I will keep waiting.

[JaedsonBarbosa]

When the part 4 will be ready?

[krwq]

@JaedsonBarbosa we have most significant parts already tested and proved working, With testing there is no single "finished" point - you can have 100% code coverage and code which doesn't work. Are there any particular scenarios you are interested in?

[JaedsonBarbosa]

No 😃

[karelz]

@krwq I expect we will declare things done for the library, when we are comfortable shipping it as stable. When we do that, we can check the box and close this issue. So how far are we test-wise?

[krwq]

@karelz I'm currently comfortable shipping as is since all important E2E scenarios I could think of are tested. I still would like to increase coverage so that less popular scenarios (including error handling) can be proven working correctly too although I do not expect finding any 2.0 blocking issues.

For me "done" means that no one will maintain or improve the library anymore

[karelz]

OK, if @bartonjs agrees with ready-to-ship-state, then let's be practical and close this issue.
If we want to increase test coverage (as non blocking 2.0), we should create separate work item for Future.

[bartonjs]

If we've hit everything off of the list of algorithms and transforms and canonicalizers, it's good with me.

[JaedsonBarbosa]

So I think this issue will finally be closed.

[krwq]

Ok, let's track the coverage progress with: https://github.com/dotnet/corefx/issues/16829 - I thought we treat this as a master issue

[karelz]

Yes, we treat it as master issue, but sometimes you have to declare things done, otherwise you would have at 1 open issue for every larger feature tracking "do more" - which is not truly helping anyone.

[JaedsonBarbosa]

It's now closed 😢
So... Where should I ask questions about the System.Security.Cryptography.Xml?

[karelz]

Isn't @krwq's answer above sufficient?
If you have bigger questions, file a new issue. If it is small clarificiation of previous answer, you can keep it here. If not sure, ask here and in the worst case we will ask you to file a new issue ;-)

[JaedsonBarbosa]

@krwq It continue without support for UWP 😞

[krwq]

@JaedsonBarbosa does https://dotnet.myget.org/feed/dotnet-core/package/nuget/System.Security.Cryptography.Xml/4.4.0-preview1-25210-01 not work for you? Could you send the steps of what you're doing?

[JaedsonBarbosa]

@krwq I just put that command:
Install-Package System.Security.Cryptography.Xml -Version 4.4.0-preview1-25210-01

That's the error:

O pacote System.Security.Cryptography.Xml 4.4.0-preview1-25210-01 não é compatível com uap10.0 (UAP,Version=v10.0). O pacote System.Security.Cryptography.Xml
4.4.0-preview1-25210-01 dá suporte a:

• monoandroid10 (MonoAndroid,Version=v1.0)
• monotouch10 (MonoTouch,Version=v1.0)
• netcoreapp2.0 (.NETCoreApp,Version=v2.0)
• uap10.1 (UAP,Version=v10.1)
• xamarinios10 (Xamarin.iOS,Version=v1.0)
• xamarinmac20 (Xamarin.Mac,Version=v2.0)
• xamarintvos10 (Xamarin.TVOS,Version=v1.0)
• xamarinwatchos10 (Xamarin.WatchOS,Version=v1.0)

[JaedsonBarbosa]

@krwq See that: https://dotnet.myget.org/feed/dotnet-core/package/nuget/System.Security.Cryptography.Xml/4.4.0-preview1-25210-01#targetframeworks

[karelz]

Just a reminder of what I said earlier: https://github.com/dotnet/corefx/issues/4278#issuecomment-292448824
I don't think you will get UWP support with the package. The package depends on .NET Standard 2.0 AFAIK and UWP does not yet have .NET Standard 2.0 support -- it is something we will work on for .NET Core 2.1 (some bits are done early to unblock larger team to work on it post 5/10, but it is not fully functional).
IMO to get UWP support for the package you will have to wait on 2.1.

[JaedsonBarbosa]

@karelz So do you think I will need to wait until when?
Can I create an uap10.1 project?

[karelz]

Yes, I think you will need to wait until then.
Unless you are super-motivated and can make it through all these speed bumps. As I said, until 5/10 our ability to help you with anything UWP will be very limited, so you will be mostly on your own 😦. Just setting the expectations here ...

[JaedsonBarbosa]

So I will wait because I can't build the corefx-master with Visual Studio 2017 😞

[JaedsonBarbosa]

@karelz @krwq I can't build the solution because I give an error, you can see the CMakeError here:
https://1drv.ms/u/s!AjDoJtNk3vvWjKIAYajeMPkWkI-m6Q
Please give me a help 🆘
PS: It's in portuguese, but you can use a translator to read that 😄

I think that failed isn't good:
-- The C compiler identification is MSVC 19.0.24218.2
-- The CXX compiler identification is MSVC 19.0.24218.2
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/Shared/14.0/VC/bin/amd64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/Shared/14.0/VC/bin/amd64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/Shared/14.0/VC/bin/amd64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/Shared/14.0/VC/bin/amd64/cl.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Failed
-- Performing Test COMPILER_HAS_DEPRECATED
-- Performing Test COMPILER_HAS_DEPRECATED - Success
-- Configuring done
-- Generating done

[JaedsonBarbosa]

Anybody now what can I do to make it build?

[krwq]

@JaedsonBarbosa how do you build? Regular process for me is:

1. Open cmd.exe
2. pushd corefx\repo\path
3. git pull
4. git clean -fdx - this will clean your repository of any leftover files (be careful if you're not sure what it does)
5. build or ./build.sh if you are on non-windows

For building native components you need to install CMake 2.8.12 or higher

This has always just worked for me.

[karelz]

@Jaedson33 you can also check & help us improve our new-contributor docs (linked from CoreFX main page)

[JaedsonBarbosa]

I'm using the CMake 3.8.
@krwq I did what you said, and I'm waiting about 1 hour and there is just Installing dotnet cli..., how many hours do you think I will need to wait?

[krwq]

@JaedsonBarbosa it should take up to few minutes, try restarting - it could be some connection issues, if doesn't work please file a separate issue in this repo, someone should be able to track down what has happened

[JaedsonBarbosa]

@krwq I get this error:

C:\Users\jaeds\Source\Repos\corefx>call "C:\Users\jaeds\Source\Repos\corefx\Tools\dotnetcli\dotnet.exe" restore "C:\Users\jaeds\Source\Repos\corefx\packages\microsoft.dotnet.buildtools\1.0.27-prerelease-01512-01\lib\\tool-runtime\project.csproj" --source https://dotnet.myget.org/F/dotnet-core/api/v3/index.json --source https://dotnet.myget.org/F/dotnet-buildtools/api/v3/index.json --source https://api.nuget.org/v3/index.json  
C:\Users\jaeds\Source\Repos\corefx\packages\microsoft.dotnet.buildtools\1.0.27-prerelease-01512-01\lib\\tool-runtime\project.csproj --source https://dotnet.myget.org/F/dotnet-core/api/v3/index.json --source https://dotnet.myget.org/F/dotnet-buildtools/api/v3/index.json --source https://api.nuget.org/v3/index.json
  Restoring packages for C:\Users\jaeds\Source\Repos\corefx\packages\microsoft.dotnet.buildtools\1.0.27-prerelease-01512-01\lib\tool-runtime\project.csproj...
  Retrying 'FindPackagesByIdAsync' for source 'https://dotnetmyget.blob.core.windows.net/artifacts/dotnet-buildtools/nuget/v3/flatcontainer/system.diagnostics.debug/index.json'.
  An error occurred while sending the request.
    A conexÆo com o servidor foi interrompida de modo anormal
  Retrying 'FindPackagesByIdAsync' for source 'https://dotnetmyget.blob.core.windows.net/artifacts/dotnet-buildtools/nuget/v3/flatcontainer/system.runtime/index.json'.
  An error occurred while sending the request.
    A conexÆo com o servidor foi interrompida de modo anormal
  Retrying 'FindPackagesByIdAsync' for source 'https://dotnetmyget.blob.core.windows.net/artifacts/dotnet-buildtools/nuget/v3/flatcontainer/system.text.encoding/index.json'.
  An error occurred while sending the request.
    A conexÆo com o servidor foi interrompida de modo anormal
  Retrying 'FindPackagesByIdAsync' for source 'https://dotnetmyget.blob.core.windows.net/artifacts/dotnet-buildtools/nuget/v3/flatcontainer/system.runtime/index.json'.
  An error occurred while sending the request.
    A conexÆo com o servidor foi interrompida de modo anormal
  Retrying 'FindPackagesByIdAsync' for source 'https://dotnetmyget.blob.core.windows.net/artifacts/dotnet-buildtools/nuget/v3/flatcontainer/system.text.encoding/index.json'.
  An error occurred while sending the request.
    A conexÆo com o servidor foi interrompida de modo anormal
C:\Users\jaeds\Source\Repos\corefx\Tools\dotnetcli\sdk\2.0.0-preview1-005724\NuGet.targets(97,5): error : Failed to retrieve information about 'System.Text.Encoding' from remote source 'https://dotnetmyget.blob.core.windows.net/artifacts/dotnet-buildtools/nuget/v3/flatcontainer/system.text.encoding/index.json'. [C:\Users\jaeds\Source\Repos\corefx\packages\microsoft.dotnet.buildtools\1.0.27-prerelease-01512-01\lib\tool-runtime\project.csproj]
C:\Users\jaeds\Source\Repos\corefx\Tools\dotnetcli\sdk\2.0.0-preview1-005724\NuGet.targets(97,5): error :   An error occurred while sending the request. [C:\Users\jaeds\Source\Repos\corefx\packages\microsoft.dotnet.buildtools\1.0.27-prerelease-01512-01\lib\tool-runtime\project.csproj]
C:\Users\jaeds\Source\Repos\corefx\Tools\dotnetcli\sdk\2.0.0-preview1-005724\NuGet.targets(97,5): error :   A conexÆo com o servidor foi interrompida de modo anormal [C:\Users\jaeds\Source\Repos\corefx\packages\microsoft.dotnet.buildtools\1.0.27-prerelease-01512-01\lib\tool-runtime\project.csproj]

C:\Users\jaeds\Source\Repos\corefx>set RESTORE_ERROR_LEVEL=1 
ERROR: An error occured when running: '"C:\Users\jaeds\Source\Repos\corefx\Tools\dotnetcli\dotnet.exe" restore "C:\Users\jaeds\Source\Repos\corefx\packages\microsoft.dotnet.buildtools\1.0.27-prerelease-01512-01\lib\\tool-runtime\project.csproj"'. Please check above for more details.

[karelz]

@JaedsonBarbosa please file a new issue as suggested above. Just a reminder that we may not be able to provide as much troubleshooting support to you before 5/10 as we would like to.

[JaedsonBarbosa]

@karelz It's now working 😄
The only think I did to make it build was move the files from C:\Users\jaeds\Source\Repos\corefx-master to C:\Users\jaeds\Source.
I think it should be in the README

[JaedsonBarbosa]

Hi, now you can use this package in a UWP app, here a sample app: https://github.com/JaedsonBarbosa/corefx/tree/BigOptimization/src/System.Security.Cryptography.Xml/TesteAssinatura

[karelz]

@JaedsonBarbosa great! Is there anything from your work which would be valuable to flow back into CoreFX? (i.e. things which are not temporary hacks)

[JaedsonBarbosa]

@karelz Well, I think you can use my project to see what can you simplify (or remove) in the CoreFX 😄

[leopignataro]

Hello everybody, great effort on porting this!

May I ask why is the Nuget package referencing .NET Core 2.0, instead of .NET Standard 2.0? Wouldn't that be preferred?

[danmosemsft]

That should have been done (c4650c9730861c61c648a6b7f1bbf40e5dfbffae)

I'm guessing you're looking at the official preview on Nuget
https://www.nuget.org/packages/System.Security.Cryptography.Xml/4.4.0-preview1-25305-02

and it's only got to Myget
https://dotnet.myget.org/feed/dotnet-core/package/nuget/System.Security.Cryptography.Xml/4.4.0-preview2-25316-02

[leopignataro]

Yes, that's were I was looking. Thank you, @danmosemsft!

[danmosemsft]

@leopignataro no problem, I encourage you to try bits from "head" -- you can get them from the homepage here https://github.com/dotnet/cli ... you can just download the zip if you want. Let us know if you find issues -- we have only a little time left to fix them before release.

[karelz]

FYI: Here's info about timelines: https://github.com/dotnet/corefx/issues/17619#issuecomment-301937346

[leopignataro]

Since you mention it, @danmosemsft, there's this one issue:

https://github.com/dotnet/corefx/issues/19198

I've suggested a fix on the thread but my message may have been missed in all the buzz.

[karelz]

@leopignataro fix for what where? If it is fix for dotnet/corefx#19198, then it should be tracked in the bug. If it is something else, I would prefer to see separate issue for it.
If you think your fix suggestion was overlooked somewhere, raise it again on that thread and ask for feedback.

[fletchsod-developer]

I'm confused now. I thought System.Security.Cryptography.Xml NuGet package is for .NET Framework and it is already included in Dot Net Core v2. It's not recognizing this namespace in Dot Net Core v2. Did I hear this incorrectly? Thanks.

[bartonjs]

@fletchsod-developer The package is mainly for .NET Core. But if you reference it from a library targeting .NET Standard it will unify with the .NET Framework version on .NET Framework, and run the code from within the package on .NET Core.

We don't have any plans to put SignedXml into the default install experience for .NET Core; it's niche enough that being a separate package on NuGet seems the best distribution model.