Powershell: Support only DSC class based resources

[SteveL-MSFT]

Summary of the new feature/enhancement

Based on discussion within the PowerShell/DSC, Guest Configuration, and DSC Community we're exploring only supporting class based DSC resources for DSC v3. This is a change from the original plan to move from mof to json.

Arguments For

Create a single ecosystem for DSC resource authors and for new DSC resource authors they can find samples of only one type and not need to learn about schema nor keeping the schema in sync with implementation.
Simplify the code in PowerShell engine to only expect class based resources. This will also eliminate any potential bugs in the json code path that was added, but does add initial work to remove that code.
Support for script based + schema resources can be added back later if necessary.
Feedback from DSC Community Call is that everyone would support moving to just class based resources.
Would need to eventually have a tool to help convert existing script based resources to class based (potentially by analyzing the AST).
No need for tool for converting mof schema to json schema nor Test-DscResource cmdlet to validate the implementation matches the json schema.

Arguments Against

There are existing script based DSC Resources for Windows that would need to be ported as classes.
Script resources may be easier to write for non-developers not familiar with classes (but need to learn json schema format).

[brwilkinson]

I was going to ask in #13359 about plans for updating the xDSCResourceDesigner module, however based on this change, I guess there is no need.

[rchaganti]

Great move. However, the first question that comes to my mind is unit testing for class resources. We don't yet have an effective way to test classes in PowerShell. Isn't this a blocking factor? https://github.com/pester/Pester/issues/706

[vexx32]

Looking through the comments in that issue (and having run into quite a few cases where class support is utterly insufficient, as well), I would say that this issue should be completely and utterly blocked on massive improvements in more useable and complete class support in PS, with the inclusions noted in that issue by @SeeminglyScience.

In other words, if this is the route y'all wanna go down, please be absolutely sure that the underlying infrastructure here is actually up to the task and much more complete than it is currently, so we don't have hundreds or thousands of folks depending heavily on a subsystem that is fundamentally incomplete.

Foundations are extremely important if you're trying to put all of the DSC infrastructure on top of it and it alone.

[SteveL-MSFT]

@vexx32 we should separate what is needed by classes for DSC vs generically in PowerShell. We have an issue that links to all the known limitations of classes, but I'm not aware of any limitations purely for DSC usage as there are resources written for DSC using classes already and the original design was specifically for DSC usage.

[vexx32]

As has been mentioned in the Pester issue pester/Pester#706 (ty @rchaganti for that link!), the reliability of a purely class-based DSC resource will be very difficult to ensure. They're not testable.

Judging from the comments in that thread, fixing that will by necessity involve significant reworking / improvements in how classes in PS as a whole are handled.

I don't know about y'all, but the idea that all DSC resources will be fundamentally impossible to mock and otherwise difficult at best to shore up with proper tests makes me very uneasy.

And again... if you're going to force folks to use classes for DSC, please make them usable. PS classes are so annoying and limiting to use in PS that if I find myself needing them, I usually move to C# instead anyway. I don't think DSC resource authors should be forced into a frustrating experience.

[iSazonov]

@vexx32 we should separate what is needed by classes for DSC vs generically in PowerShell. We have an issue that links to all the known limitations of classes, but I'm not aware of any limitations purely for DSC usage as there are resources written for DSC using classes already and the original design was specifically for DSC usage.

After some experiments with PowerShell classes I did a long time ago I found many limitations and I'd expect an endless stream of claims as soon as DSC community start actively using them.
I'd like to see examples of DSC resource implementations on PowerShell classes to gain confidence otherwise.

Do you consider a DSL? I remember @rjmholt's old experiment with dynamic keyword support https://github.com/PowerShell/PowerShell/pull/3169 Perhaps this is a good alternative for DCS development.

[vexx32]

That's my concern as well @iSazonov. If we put all of DSC into classes and nothing else, there simply is no alternative option when something doesn't work quite right / simply fails to work the way someone needs or expects it to.

There are so many issues around classes that, while I wholeheartedly think this is actually a really good way to go, I'm _extremely_ wary of even implying it's a good idea to do this without fixing a lot of the outstanding issues and missing features in PowerShell's class support.

This will either be a very good decision, or a really costly decision (and probably quite upsetting for the DSC community sooner or later), and it really does all depend on the foundation being in place or not.

[ryanspletzer]

After some experiments with PowerShell classes I did a long time ago I found many limitations and I'd expect an endless stream of claims as soon as DSC community start actively using them.
I'd like to see examples of DSC resource implementations on PowerShell classes to gain confidence otherwise.

I tried with ServiceBusForWindowsServerDsc at one point. (I stopped developing it when the product was discontinued.)

If you look in the tests (which I tried to closely follow the approaches with DscResource.Tests), you'll see that I wasn't mocking classes, but rather mocking cmdlets that the classes called.

Sidenote: I'm not sure I fully understand the need to mock classes in the DSC Resource context? Unless you're trying to mock a $this.Get() call from the Test() method, but still, in that case, I was able to work around it by mocking the cmdlets the Get() method called to produce the desired result.

That said, obviously, the ability to do class mocking in general, even outside of DSC, will be helpful regardless.

I bought into the promise and the hopeful simplicity of class-based DSC resources with WMF 5.1, and viewed them as the future even then (because Microsoft was saying that they were the future at that point already), and I still think they're a great idea.

That said, at that time little info was available on them and I had to dig into the bowels of how Microsoft structured their modules and their class-based DSC resources like WindowsPackageCab (take a look at C:\Windows\System32\WindowsPowerShell\v1.0\Modules\PSDesiredStateConfiguration\DSCClassResources\WindowsPackageCab).

Interestingly xWindowsPackageCab in xPSDesiredStateConfiguration is no longer a class-based resource? (Or was there ever a class-based version in the xPSDesiredStateConfiguration module? I tried looking back in history a bit and I don't think it was.)

I'm curious if there are other class-based DSC resources out there today? (I'm not aware of a way to search the PS gallery for class-based resources specifically.)

[SeeminglyScience]

Sidenote: I'm not sure I fully understand the need to mock classes in the DSC Resource context? Unless you're trying to mock a $this.Get() call from the Test() method, but still, in that case, I was able to work around it by mocking the cmdlets the Get() method called to produce the desired result.

That's a fair point. At the same time though, if the best practice becomes to use class methods as little as possible so that your resource is testable, a class only design may not be the right choice.

---

I'm also just generally uncomfortable with improving a general feature specifically so that DSC can utilize it. Historically when that happens, we get features that are just barely enough to be great in DSC but often quite broken everywhere else. I understand that feature development needs demonstrable value and DSC is a good source of that. I just ask that the history of tunnel vision taken into account by whoever takes on these improvements.

[ryanspletzer]

That's a fair point. At the same time though, if the best practice becomes to use class methods as little as possible so that your resource is testable, a class only design may not be the right choice.

Agreed!

I'm also just generally uncomfortable with improving a general feature _specifically_ so that DSC can utilize it. Historically when that happens, we get features that are just _barely_ enough to be great in DSC but often quite broken everywhere else. I understand that feature development needs demonstrable value and DSC is a good source of that. I just ask that the history of tunnel vision taken into account by whoever takes on these improvements.

I would like to see the improvement so I can mock classes in DSC and elsewhere. :) DSC would benefit but I think the language would benefit, too. The general class concept is helpful in more places than just DSC, IMO.

Though, we'd also be in this interesting state where we'll be able to mock classes on PS 7.2+ but not on down versions, so they'll be testable in, say, PS 7.2 for correctness, but your execution environment may still be Windows PS 5.1. In theory, the class should execute the same... In theory.

5.1 is slowly becoming the IE 11 of PowerShell -- have to write things for the lowest common denominator that's in the box, because if you're making a script generally available you don't know who will be running it where. These days I tend to write scripts that target the latest PS Core and 5.1 and handle the quirks between the two (and occasionally across platforms).

If you didn't go 100% class-based, I'd foresee a scenario where people are maintaining schema mof and json files both for max compat on 5.1 and 7.2+. (Perhaps with some automated translator from json to mof -- IMHO json is a friendlier format than mof and I don't anticipate people having issues with maintaining in json and using a utility script to convert to mof.)

And if you do go 100% class-based, obviously we're talking re-writes across the board, especially from a lot of resource modules maintained by Microsof themselves. (A tool to help convert script resources to class resources is one thing. However, converting unit and integration tests... much trickier.)

And regardless of 100% class-based or not, in some scenarios people are going to have to wrangle with win-specific cmdlets like Get-EventLog and explicitly using Import-WinModule from the WindowsCompatibility module when they're in PS 7.2+.

---

I want to be clear that, though I wrote a module with class-based resources, please don't consider it an endorsement of class-based DSC resources. To be honest, they were challenging to develop when I started that module in 2016. (And this is coming from someone who writes C# -- it wasn't challenging due to a lack of understanding of class concepts, in fact I liked that it was going this direction.) It made sense at the time since that seemed like "the future" (and who wants to develop a brand new module on an approach that might become deprecated), and it seemed like avoiding hand-editing schema mof files would be a nice benefit (though that will get better with JSON I'm guessing), but that did not mean it was easy by any stretch.

The challenges then mostly revolved around DSCResources.Tests not being quite caught up to the concept of class-based resources (though it seems to have gotten better, see https://github.com/PowerShell/DscResource.Tests/pull/75, https://github.com/PowerShell/DscResource.Tests/pull/76, https://github.com/PowerShell/DscResource.Tests/pull/64, https://github.com/PowerShell/DscResource.Tests/issues/118, https://github.com/PowerShell/DscResource.Tests/pull/119, https://github.com/PowerShell/DscResource.Tests/issues/149, https://github.com/PowerShell/DscResource.Tests/pull/150, https://github.com/PowerShell/DscResource.Tests/issues/173, https://github.com/PowerShell/DscResource.Tests/pull/212, https://github.com/PowerShell/DscResource.Tests/pull/200), and also sorting through new cryptic errors in DSC that classes brought into the fold. I almost considered switching the project back over away from classes at the time because it seemed like it wasn't ready.

I don't think I was alone, either: It looks based on looking at some of those referenced issues / PR's that SqlServerDsc at one point tried to move towards class-based resources and then went back to script-based resources, see this slice in time compared to current: https://github.com/dsccommunity/SqlServerDsc/tree/21411fd562d8fd2eee43942fa5f86c1d89e0071c

This exchange in particular is interesting and related to that, and interestingly also sheds more light on why WindowsPackageCab went back to script-based resource with schema mof: https://github.com/dsccommunity/SqlServerDsc/issues/673#issuecomment-314591255

I'm torn because I really want class-based to work because of the simplicity of not having to have a schema mof (or json) file, but the approach didn't seem to get quite the uptick that was hoped for when it was first introduced. I think in order for it to get uptick in usage in the future, the core issues of why it wasn't picked up in the first place have to be fully understood and addressed.

GitHub

dsccommunity/SqlServerDsc

This module contains DSC resources for deployment and configuration of Microsoft SQL Server. - dsccommunity/SqlServerDsc

[gaelcolas]

As there's loads to unpack here, and I did not want to spam this issue, so I've written a post with my opinions here: https://dsccommunity.org/blog/class-based-dsc-resources/

TL,DR:
I'm excited and I think it's the right move.
Although potentially risky, going all-in Class resource for PS7 is a good move because it help focus the effort (both for the PowerShell team and the community). Also, that does not mean it can't change later.
JSON & MOF exist because of the lack of metadata in functions, I'd prefer having metadata and being able to use AST. It also avoids different moving parts and simplifies some of the tooling.

Supporting JSON in an existing DSC resource would often be breaking and not backward compatible, while class is.
A fair few explanations about testing, why we're not using Classes in DSC Community Resources.

It's long, but I tried to provide context. Feel free to reach out (slack/twitter) for questions and comments.

Class Based DSC Resource only proposal

[ryanspletzer]

@gaelcolas Great post! You touched on a number of things more eloquently than I could. :)

To summarize my ramblings above, I want class-based to be the way, and I want it to be better than what it was in the past.

I'll go out on another limb and say that, I don't think schema mof's are that much more approachable than classes. And I know a number of PS folks out there who never even approached DSC resource authoring at all in the first place because they look at this mof format and can't reason about it -- basically adding a property in functions and then having to add the same in this other file is a strange concept and a non-starter for some.

Classes, if done right and streamlined with proper tooling, docs, etc. like you outlined, may have the possibility to be even more approachable than what we've been doing thus far.

[iSazonov]

As for testing, we have two issues which is seems a key here:

• using custom ALC (to resolve assembly conflicts)
• support assembly unloading

[mkht]

Question

Does this change mean that the composite resources will no longer work?
Composite resources are sort of like a sugar-coated syntax, and it seems inherently difficult to replace them with class-based resources.

Impression

I've created a lot of DSC resources, but they are mostly script-based. I've never created a class-based resource, so I'm sure I can do it, but it's going to be a struggle.

Another concern is what to do with old resources that are useful but have stopped development. I'll have to decide whether to find a replacement written in a class-based or create one myself, neither of which will be easy.

[gaelcolas]

This work @mkht does not touch or replace Composite resources. Composites don't use MOF anyway (it's only the compilation work, but that's not worked on either).

This work won't affect Windows PowerShell 5.1 anyway, your script/MOF-based DSC resources will keep working there.
Creating class based DSC resource isn't hard, we'll document the process in dsccommunity.org soon, and I recommend the presentation by Bartek Bielawski in the meantime.

Always interested to hear about old resources that are useful but have stopped development, drop us a line in the DSC channel on the PowerShell slack/discord.
Those might not be usable in PowerShell 7 when this work is done, but forking and updating might not be too much work.

[rchaganti]

@nohwnd -- for your thoughts on this.

[ThomasNieto]

I haven't seen any mention of C# cmdlet resources. Since they use MOF today, I'd assume they would be not supported as well. I'd like to see C# resources be supported without the need for MOF and use a similar or same implementation to PowerShell classes.