Choco: Control enhanced exit codes with a feature switch

[ferventcoder]

By default useEnhancedExitCodes feature will be turned on. Over time we'll be adding enhanced exit codes to more and more commands. This is related to #512, but not tied to the package itself providing an exit code.

This feature switch is being provided as a compatibility for integrations (like config managers) to continue to working if there is an issue with seeing new codes prior to having a new release of those integrations.

Release Notes

We've listed the issues related to this (#1602 #1724, especially #1724) as possibles breaking changes as it may affect tools that are integrating with Chocolatey and interpreting the output of the exit code. In these cases, it would likely temporarily break those tools until they've had a chance to release new versions of their tools. If you run into this, you simply need to turn off the feature "useEnhancedExitCodes". That is as simple as choco feature disable --name="'useEnhancedExitCodes'"

[ferventcoder]

Here is the ansible fix - https://github.com/ansible/ansible/pull/53841 (thank you @jborean93 for catching that so quickly - before we sent our announce email noting the potential breaking change)
For other config managers:

• BoxStarter - @pauby
• Chef - @mwrock @btm @Vasu1105
• Otter - @MarkJohnson1974
• PowerShell DSC - @pauby
• Puppet - @michaeltlombardi @RandomNoun7
• Salt - @twangboy @jfindlay @rallytime

[jborean93]

Heh no worries, our CI was failing within an hour of the release so was pretty easy to pick up. I’ll let people know about the feature to disable the enhanced codes but a new release of Ansible with the fix should be out relatively soon.

[jborean93]

@ferventcoder I didn't want to comment on each of the new issues you've raised but I really think you should reconsider making this the default and the enhanced exit codes should be opt in instead of opt out. As you've seen there have been multiple issues, I've closed 3 in ansible/ansible this past 24 hours due to this effectively breaking the module. Typically an exit code that is non 0 means an error and because historically this was the case for many choco.exe commands I think it was wrong to change it, especially with so little notice.

The other issue is that by default the win_chocolatey module and the PS script on Chocolatey's website installs the latest making it harder for people to just say I want the older version until they've upgraded Ansible to one that supports v0.10.12+. I'm open to suggestions here, I couldn't find anything after doing a very basic search.

If you are still wanting to push ahead with this change on other commands like feature, source, etc, I'm going to have to make changes to 3 more Chocolatey modules we have to ignore an rc of 2 and hopefully get that out in an Ansible release before 0.10.14 is out. This is all well and good for people that are upgrading Ansible on each regular release but I can guarantee that there are numerous people who cannot and will not upgrade for a long time. This means that the Chocolatey modules in Ansible are effectively broken or require a new and relatively obscure feature to be set which is not very intuitive.

I can't talk about the other config management software but this definitely hit us out of the blue and was surprising that a breaking change was made. It makes us worried what other breaking changes will be made in future Chocolatey versions that effectively break the modules we have. Even having a release candidate of a newer version that we can run our integration tests against would be great.

[nitzmahone]

... or at least delay the switch to the default until after projects that rely on choco have had a chance to accommodate it. As-is, a lot of working automation in the world will be is being broken by this change until the various projects have had a chance to release fixes, and users have upgraded.

[charlieok]

I was also just hit by this. I would add that for a change like this, which you even described in the release notes as "possible breaking changes", you should bump a more significant component of the version number. For instance 0.10.11 -> 0.11.1 rather than 0.10.11 -> 0.10.12.

That would have signaled a more significant update, which may require attention, and software configured with the intent of picking up minor non-breaking fixes and not picking up significant changes could have stayed on the 0.10.* series. This would be in line with common understandings of version numbering (e.g. https://semver.org/ ).

Granted, you're technically not breaking semver due to "Major version zero (0.y.z) is for initial development. Anything may change at any time. The public API should not be considered stable." But this has been around for years and seems to be in pretty widespread use now :)

[srjennings]

All of a sudden chocolatey was breaking in Ansible plays. Took a bit to find this thread after stumbling through diagnostic logs. Definitely following this thread.

I echo @jborean93's assessment regarding breaking changes and being surprised by it.

[ferventcoder]

FYI folks - for 0.10.14, we plan to switch the default to off. So folks can opt in.

[ferventcoder]

@charlieok

Just to touch on the SemVer bits - we are semver compliant as we are still sub-v1. I think you are correct there should be a more significant bump in the sub v1 category to signify this. We do publish a changelog with how to turn things off if they do cause issues, we also have an announcements email at https://groups.google.com/group/chocolatey-announce where we publish these.

You are correct in that we should look to moving to a v1 release and it is something we have planned to do this year.

[ferventcoder]

@jborean93 @nitzmahone apologies, I feel like we made a bad decision here in a move to get Chocolatey v0.10.12 out the door. So normally we would have had a lot more notice to folks about this as we did with #512 to allow time for preparing here. After more thought, I believe you are correct and this needs to be off by default and folks should opt-in to it. I'm not sure if we would want to switch the defaults at a later date either.

The nice thing about Chocolatey features is that we can switch defaults if a user has not explicitly made a change to them.

[ferventcoder]

Folks, please follow #1784. The good news is we are going to have a fast turnaround for 0.10.14 due to this and a couple of other things that have been broken.

[nitzmahone]

@ferventcoder thanks- having been in similar situations on our own projects, we can definitely sympathize! Personally, I'm all for making the enhanced exit code behavior the default in a future Chocolatey release, once callers that are aware of the new behavior have had a reasonable amount of time to filter out into the world. Changing return values and exit codes is particularly tough, since you can't easily issue deprecation warnings on them (that aren't just noise to 99% of users, anyway).