Powershell: Behavior between New-ModuleManifest, Update-ModuleManifest, Test-ModuleManifest is inconsistent

[pougetat]

Summary of the new feature/enhancement

The behavior between New-ModuleManifest, Update-ModuleManifest and Test-ModuleManifest seems inconsistent.

• New-ModuleManifest does not check whether or not the values entered by a user are correct. This means that a user can create an invalid manifest file using this cmdlet on which Test-ModuleManifest will fail.

• Update-ModuleManifest uses Test-ModuleManifest to check certain fields in the manifest file that the user is updating. These values are not necessarily values passed to Update-ModuleManifest but rather values which are already present in the manifest file. Update-ModuleManifest also adds some extra checks that Test-ModuleManifest does not do (ex : ExternalModuleDependencies need to be present under RequiredModules or NestedModules)

• Test-ModuleManifest does not check the Private data section at all.

Proposed technical implementation details (optional)

• Test-ModuleManifest should include all checks currently in this cmdlet and in Update-ModuleManifest that are not present in Test-ModuleManifest. This would allow us to remove a lot of extra checks present in Update-ModuleManifest (PowerShellGet issue but still important to mention here)

• Update-ModuleManifest solely relies on Test-ModuleManifest for its checks.

• New-ModuleManifest uses Test-ModuleManifest to check for invalid values. This would be a breaking change since users would no longer be able to create invalid manifest files via this cmdlet. However I think it would be a minor breaking change since it's hard to see the use case for automating the creation of manifest files.

[iSazonov]

Looks good.

/cc @SteveL-MSFT

[SteveL-MSFT]

One problem I see is that a standard workflow to create a new module is:

new-modulemanifest -path ./foo.psd1 -rootmodule foo.psm1
code foo.psm1

Meaning that they create the manifest before creating the root module script. If New did validation, then this would be breaking. One option would be to add a -Force type switch to allow this, but we could go with a non-breaking change and have a -Validate switch.

[jzabroski]

-Force please. No hacks. Cut the arm off and survive.

[pougetat]

Hey @SteveL-MSFT,
I'm hesitating between -Force and -Validate. On the one hand -Force does seem much more idiomatic than -Validate and it forces the user to think about what the command is doing behind the scenes. The fact that it is a breaking change though does cut down on the appeal.
Can I still go with -Force or should we stick to -Validate ?

[jzabroski]

I vote for -Force

On Mon, Apr 1, 2019 at 1:10 PM pougetat [e-mail] wrote:

Hey @SteveL-MSFT https://github.com/SteveL-MSFT,
I'm hesitating between -Force and -Validate. On the one hand -Force does
seem much more idiomatic than -Validate and it forces the user to think
about what the command is doing behind the scenes. The fact that it is a
breaking change though does cut down on the appeal.
Can I still go with -Force or should we stick to -Validate ?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/PowerShell/PowerShell/issues/9114#issuecomment-478664203,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAbT_eyNjaSrIflImEeH4SNfaAb-2Z-Pks5vcj1zgaJpZM4boyxo
.

[joeyaiello]

The base workflow seems very common to me. Might we be okay with throwing warnings for non-valid values in New-ModuleManifest?

Given the unlikelihood of New-ModuleManifest in fully non-interactive/automated scenarios, this seems like a better approach to me. Are there any automated scenarios I'm not realizing?

[SteveL-MSFT]

@PowerShell/powershell-committee reviewed this, we agree that all the checks should be in one place which is Test-ModuleManifest. Continue having Update-ModuleManifest rely on Test-ModuleManifest by default. New-ModuleManifest should have a new -Validate switch that uses Test-ModuleManifest.

[jzabroski]

How is -Validate going to be different from what -Force would do? Does -Validate have side-effects? If so, why call it validate.

[SteveL-MSFT]

@jzabroski the difference is opt-in vs opt-out. We are optimizing for backwards compatibility here so introducing new behavior we decided to have it opt-in so the user needs to explicitly use -Validate. There would not be a -Force which would only be needed for opt-out which we are not supporting for this use case.

[jzabroski]

Let me re-phrase. What specifically would potentially cause scripts to break if you changed -Force? I just don't see who is benefited by "optimizing for backwards compatibility".

Can you please re-name -Validate to -Test, since it's:

1. Shorter, less typing
2. Matches exactly the command it delegates validation to: -Test pipes to Test-ModuleManifest

[daxian-dbw]

@jzabroski When creating a new module, it's pretty common to run New-ModuleManifest -Path "myModule.psd1" -RootModule "myModule.psm1" without having myModule.psm1 created (because that would be the next step). We want that to continue to work as always without requiring a -Force.

[jzabroski]

@daxian-dbw It's funny, because that is exactly what I hate about building PowerShell modules. Have you ever used C#, especially the new .NET SDK MSBuild project system and auto-globing compile items like *.cs files? PowerShell modules are so painful to use in my experience, and the documentation is _really bad_. I was so frustrated trying to write my own modules the first time I tried - and there were SO MANY knobs to turn that I pretty much gave up and went back to C#.

tl;dr: PowerShell module development workflow sucks. New-ModuleManifest -Path "myModule.psd1" -RootModule "myModule.psm1" -Force with should just freaking create the damn files.

When I do PowerShell development, I feel like I'm back in Java 1995 world, before Ruby on Rails woke up the echoes of the entire development community and evangelized convention over configuration. .NET Core has adopted this convention over configuration, but PowerShell seems resistant and determined to waste my time.

[SteveL-MSFT]

@jzabroski just so I understand correct, you would like:

New-ModuleManifest -Path "myModule.psd1" -RootModule "myModule.psm1" -Force

To create myModule.psm1 in addition to myModule.psd1?

[jzabroski]

That's one request as part of a _full feature_. There's a couple of things I want , most inspired by @RamblingCookieMonster 's WFTools. The only drawback I have to creating Manifests the WFTools way is the stack traces are bad, because everything is dot-sourced into the module, which creates horrible stack trace line numbers.

You can see his psm1 file here: https://github.com/RamblingCookieMonster/PowerShell/blob/master/.build/WFTools.psm1

The other problem I had reading the docs on defining psd1 and psm1 files is the whole vocabulary lesson it takes to understand how to build a module. I feel like I'm back in 1999 reading CfEngine documentation written by a physicist from CERN laboratories who just finished his PhD in particle physics. For example, what is a RootModule? That took me a week to understand, and there are so few clean open source PowerShell modules to use as reference. Even you guys at Microsoft write your junk in C# instead of PowerShell, so how is somebody supposed to learn how to do it the right way? The official documentation for New-modulemanifest has RootModule commented out for example.

Going back to the "no examples" comment, I was completely perplexed that you are concerned about breaking backward compatibility for a command I could find no good examples for how to use in the first place.

Which is why I think the focus should be, rather than on maintaining backward compatibility, focusing on user stories and _adding features_ that make module development easier. Nobody will care what the interface was 5 years from now if we get the user stories right.

[KirkMunro]

@jzabroski You may be find Plaster helpful for module development. If you're not familiar with it, it's a scaffolding generator for various things like modules, Pester tests, etc.

Stucco is a Plaster template for PowerShell modules written by Brandon Olin that may also be helpful.

I'm not offering these as a solution to making module development easier out of the box -- there's work to be done there. That work aside, you should look at these resources if you haven't already.

Also, I want to focus on one paragraph from your comment for a moment to make sure I understand where you're coming from:

For example, what is a RootModule? That took me a week to understand, and there are so few clean open source PowerShell modules to use as reference.

This statement really surprised me. Personally, I think there are _many_ clean open source PowerShell modules to use as reference; however, there are many more that are not that clean, which I think is part of the point you are trying to make. One recommendation I have here is to look for Microsoft MVP modules. There are many, and they are generally great examples to follow. Where do you go looking for clean open source PowerShell modules to use as reference?

The official documentation for New-modulemanifest has RootModule commented out for example.

True, and if you invoke New-ModuleManifest .\test.psd1, the file that is generated has that commented out as well. Right above the commented out value field though is a comment explaining what it is for, and that is also shown in the New-ModuleManifest documentation, which made me surprised that it took you a week to understand. In your mind, now that you know what RootModule is for, what would have made that learning hurdle easier to get over? Is having RootModule at the top of the manifest with the comment explaining what it is for not clear enough? It mentions "Script module" and "binary module". If you google "script module", the second link takes you to the How to write a PowerShell script module docs. Have you seen those documents? Did you come to the manifest as the starting point for your module, or after you had created a script module? There are a lot of questions here, I know, but I'm hoping to get a better idea of what might be missing.

I wonder if it might be helpful if in a manifest generated by New-ModuleManifest there was a comment that said "If you haven't created a PowerShell module before, read this first: ..." with a hyperlink.

[jzabroski]

Personally, I think there are _many_ clean open source PowerShell modules to use as reference; however, there are many more that are not that clean, which I think is part of the point you are trying to make. One recommendation I have here is to look for Microsoft MVP modules.

@KirkMunro It's a bit circular to say there are examples written by Microsoft MVPs without defining thhe set of MVPs you respect. Perhaps you were indirectly referring to yourself. Until you go ahead and define it, I'll think of it as an empty set - just like PowerShell's documentation! I actually recently discussed with the Pester team the trade-offs in defining PowerShell modules, and in just a day of thinking, discovered with another programmer serious gaps and problems in the PowerShell module system: RequiredModules has no way to hook a version handler. It's just an assembly name.

You do have a number of good suggestions here. I'd suggest creating a PR rather than just discussing them. I grew up in the Linus Torvalds' ethic of the one who writes sensible, working code usually gets the most votes on how the system takes shape.

[KirkMunro]

@jzabroski: While there are many MVPs, I was referring to Cloud and Datacenter Management MVPs, and in particular, those which were previously PowerShell MVPs when that was a separate category. But I didn't have a set to share since I don't have a specifically defined set myself.

As for a PR, I have no problem submitting PRs when I have clarity on issues. For this discussion though, I think it's premature to submit a PR. I asked you a bunch of questions to get clarity on what might be missing, and I don't have answers to those questions yet. I could create a PR based on my assumptions, but having worked as a Product Manager for quite a few years my preference is to get more clarity before moving forward with recommendations or a PR.

[mklement0]

Let me try to summarize and complement the discussion:

• There _is_ conceptual guidance in the docs, namely Writing a Windows PowerShell Module (I can't speak to the quality of that guidance):

  • The about_Modules topic links to that topic, but the only help topic for module-related _cmdlets_ that links to it, via the latter, seems to be New-ModuleManifest.

  • Just consulting the comments in a manifest generated by New-ModuleManifest is not enough; pointing to an official resource via an URL to offer guidance, as @KirkMunro suggests, is a great idea.

• While there may be conceptual guidance (if you manage to find it), the official tooling is lacking.

  • New-ModuleManifest is just a small piece of the puzzle.

  • Being able to scaffold various types of modules with sensible defaults is missing.

• The community has tried to fill that void, as evidenced by projects such as Plaster (thanks, @KirkMunro), but having such tooling be an official part of the project is obviously preferable.

I actually recently discussed with the Pester team the trade-offs in defining PowerShell modules, and in just a day of thinking, discovered with another programmer serious gaps and problems in the PowerShell module system: RequiredModules has no way to hook a version handler. It's just an assembly name.

While just referring to a module (not: assembly) by _name_ only is common, the RequiredModules entry _does_ allow you to specify version requirements, namely via a "FQMN" (a Fully Qualified Module Name), which in PowerShell code is expressed as a hashtable that represents a [Microsoft.PowerShell.Commands.ModuleSpecification] instance:

For instance, the following example specifies a dependency on version 1.7 _or higher_ of module ClipboardText:

RequiredModules = @{ ModuleName='ClipboardText'; ModuleVersion='0.1.7' }

The caveats are:

• This is _inadequately_ documented in How to Write a PowerShell Module Manifest, which, again, is not the easiest topic to discover.

  • I'm not aware of official documentation explaining FQMNs comprehensively; there are parts of the puzzle in the Get-Module and Get-Command help topics, but only this third-party reference provides a decent overview: https://info.sapien.com/index.php/scripting/versioning/using-a-modulespecification-object

• The notation is cumbersome, and lacks the concision and constraint logic of npm-style semver-compatible specifiers such as ClipboardText@^0.1.7 in order to implicitly limit the _maximum_ compatible version; you can, however, lock in a specific version with RequiredVersion, and limit the max. version with MaximumVersion.

[iSazonov]

it's pretty common to run New-ModuleManifest -Path "myModule.psd1" -RootModule "myModule.psm1" without having myModule.psm1 created (because that would be the next step). We want that to continue to work as always without requiring a -Force.

We could check presence and create .psm1 file.

[jzabroski]

While just referring to a module (not: assembly) by _name_ only is common, the RequiredModules entry _does_ allow you to specify version requirements, namely via a "FQMN" (a Fully Qualified Module Name), which in PowerShell code is expressed as a hashtable that represents a [Microsoft.PowerShell.Commands.ModuleSpecification] instance:

I fixed my comment to say:

RequiredModules _RequiredAssemblies_ has no way to hook a version handler. It's just an assembly name.

Sorry about that.

[kilasuit]

Technically this issue is a mix of Core PowerShell cmdlet issues mixed with PowerShellGet which I expect will get resolved as part of the PowerShellGet 3.0 RFC - https://github.com/PowerShell/PowerShell-RFC/pull/185

Screen grab taken from psv7-preview1

[jzabroski]

As for a PR, I have no problem submitting PRs when I have clarity on issues. For this discussion though, I think it's premature to submit a PR. I asked you a bunch of questions to get clarity on what might be missing, and I don't have answers to those questions yet. I could create a PR based on my assumptions, but having worked as a Product Manager for quite a few years my preference is to get more clarity before moving forward with recommendations or a PR.

Here is one example of where I have sought expert feedback, and received little guidance: https://github.com/pester/Pester/issues/1314

We have a team of three _very_ experienced developers trying to make usable PowerShell modules, and it is very difficult.

While experimenting with figuring out the best approach, we have learned a stunning number of problems with PowerShell, such as:

1. Lack of built-in support for attaching PwSh.exe to a Visual Studio Debugger to step-through C# cmdlets
2. Lack of way to force a global "on Import-Module" hook within C#, since the C# cmdlets API does not have a hook for things to be loaded on module import.
3. RequiredAssemblies has no way to specify a version of an assembly to load
4. RequiredModules has no way to specify version constraints (in the way NodeJS and other package managers do)
5. The Windows Compatibility pack causes issues with serialization
6. No guidance on how to integrate traditional logging frameworks like log4net, NLog, Serilog, etc. or cloud-based logging frameworks like Datadog for centralized log collection. - Forget writing enterprise code where people are accountible for fixing errors in scripts, because there is no centralized way to collect such errors.

I'm sure there are more pain points, but this is just off the top of my head. And these are "brush your teeth and eat your veggies" problems that have been solved since .NET 1.1.

@jzabroski You may be find Plaster helpful for module development. If you're not familiar with it, it's a scaffolding generator for various things like modules, Pester tests, etc.

Stucco is a Plaster template for PowerShell modules written by Brandon Olin that may also be helpful.

It seems like there may be something useful here, but it is not particularly well-documented. It would be helpful if you gave me an idea on the ramp up time to learning to use this, as my best guess is it might be powerful, but might take days to learn, and might only obfuscate some of the problems I've found with PowerShell modules in general, making debugging and reporting these issues like the ones above that much harder (how do I reproduce issues to submit bug reports here, etc.)

[vexx32]

Specifically regarding point 1 there -- I've not tried with full VS myself, but Visual Studio Code's debugger seems perfectly capable of stepping through C# cmdlets as needed. I usually open the powershell console, use the dotnet attach debug config, and import the module. Then, set a breakpoint in the cmdlet and execute the cmdlet from the console.

It's pretty effective in my experience; not sure if something similar is doable in full VS? 🙂

[jzabroski]

dotnet attach

I guess, until Resharper is supported with VS Code, I will stick with Visual Studio.
For dotnet attach, are you referring to this: https://marketplace.visualstudio.com/items?itemName=DennisMaxJung.vscode-dotnet-auto-attach - this has fewer than 3,000 installs, so definitely not a standard approach or well-known.

Note: dotnet attach is not a built-in dotnet.exe command and not mentioned in NateMcMaster/dotnet-tools repository

Thanks!

[KirkMunro]

• Lack of built-in support for attaching PwSh.exe to a Visual Studio Debugger to step-through C# cmdlets

I do this using Visual Studio 2019 just about every day, or if not then every other day using Attach to Process with pwsh.exe or powershell.exe. Note that I find the Visual Studio PowerShell Tools extension has gotten in the way for me doing this in the past, to the point where I just removed it because I didn't have time to track down the issues, and resolving them wasn't as important to me as being able to debug and step through my C# code, either in pwsh if I'm working on open source PowerShell or in binary modules I have authored. I've also done this in the past using Visual Studio 2017 (and probably 2015 as well -- been a while, can't quite remember).

• Lack of way to force a global "on Import-Module" hook within C#, since the C# cmdlets API does not have a hook for things to be loaded on module import.

If you have a public class that implements IModuleAssemblyInitializer, its OnImport method will get called at module load time. PowerShell 3.0 and later.

• No guidance on how to integrate traditional logging frameworks like log4net, NLog, Serilog, etc. or cloud-based logging frameworks like Datadog for centralized log collection. - Forget writing enterprise code where people are accountible for fixing errors in scripts, because there is no centralized way to collect such errors.

See this RFC on using ScriptBlock message handlers to facilitate centralized message management.

[jzabroski]

If you have a public class that implements IModuleAssemblyInitializer, it’s OnImport method will get called at module load time. PowerShell 3.0 and later.

Thanks; we're going to give this a shot. What we resorted to doing instead was creating an abstract class CompanyNameCmdletsBase and having a static constructor that initializes the logging framework; the first time a cmdlet accesses a base class member (e.g., protected ILog Log {get; set;}), it initializes it. This works due to the fact static constructors are guaranteed to be thread-safe, but it's messy in that we have to have our plumbing code all over the place, and no way to know which Cmdlet will be the first to call the initialization logic (multiple entry points creates unnecessary bounded non-determinism).