Psscriptanalyzer: PSUseToExportFieldsInManifest throw System.InvalidOperationException: Collection was modified; enumeration operation may not execute.

[LaurentDardenne]

Steps to reproduce

1..30|% {$res=Invoke-ScriptAnalyzer -path 'C:\Program Files\WindowsPowerShell\Modules\PSScriptAnalyzer\1.16.1\PSScriptAnalyzer.psd1' -ExcludeRule PSMissingModuleManifestField }

Expected behavior

No error.

Actual behavior

Invoke-ScriptAnalyzer : La collection a été modifiée ; l'opération d'énumération peut ne pas s'exécuter.
At line:1 char:15
+ ... .30|% {$res=Invoke-ScriptAnalyzer -path 'C:\Program Files\WindowsPowe ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (C:\Program File...ptAnalyzer.psd1:String) [Invoke-ScriptAnalyzer], Cm
   dletInvocationException
    + FullyQualifiedErrorId : RULE_ERROR,Microsoft.Windows.PowerShell.ScriptAnalyzer.Commands.InvokeScriptAnalyzerComm
   and

Environment data

$psversiontable
Name                           Value
----                           -----
PSVersion                      5.1.14409.1012
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0, 5.0, 5.1.14409.1012}
BuildVersion                   10.0.14409.1012
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1


 (Get-Module -ListAvailable PSScriptAnalyzer).Version | ForEach-Object { $_.ToString() }
1.16.1
1.16.0
1.15.0
1.12.0
1.11.1

See also the issue #900

[bergmeister]

Similar to your other issue, I cannot repro. Let's take this one aside and try to boil down the issue of the other one.

[kalgiz]

I was able to reproduce, but the error does not always occur.

[LaurentDardenne]

@kalgiz
_"but the error does not always occur."_
I agree, this error is erratic.
When executing the same code several times in a row, once the first error is triggered the behavior is degraded :

Line of code executed immediately after the first (_1..30 | ..._) :

[bergmeister]

I have tried again but it still does not happen on my machine (neither with the released version or the development branch). This sounds like a race condition where a list is modified in a non-thread safe manner. Some background: When PSSA analyses one file, it spawns a Thread for each rule.
@LaurentDardenne Can you please provide more error details using e.g. $error[0] | select *
@kalgiz If you build in debug mode, then Invoke-ScriptAnalyzer will have an -AttachAndDebug switch that automatically attaches you the debugger in Visual Studio, and the default exceptions settings should cause it to break at the line where the exception happens (if not you can change the exception settings to break on any exception).

[LaurentDardenne]

With Windows Seven sp1 Fr ( Microsoft Windows [version 6.1.7601] )

$Error[0]|select *

writeErrorStream              : True
Exception                     : System.Management.Automation.CmdletInvocationException: La collection a été modifiée;
                                l'opération d'énumération peut ne pas s'exécuter. --->
                                System.InvalidOperationException: La collection a été modifiée; l'opération
                                d'énumération peut ne pas s'exécuter.
                                   à System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
                                   à System.Collections.Generic.List`1.Enumerator.MoveNextRare()
                                   à
                                Microsoft.PowerShell.Commands.ModuleCmdletBase.UpdateCommandCollection(Collection`1
                                list, List`1 patterns)
                                   à Microsoft.PowerShell.Commands.ModuleCmdletBase.LoadModuleManifest(String
                                moduleManifestPath, ExternalScriptInfo scriptInfo, Hashtable data, Hashtable
                                localizedData, ManifestProcessingFlags manifestProcessingFlags, Version
                                minimumVersion, Version maximumVersion, Version requiredVersion, Nullable`1
                                requiredModuleGuid, ImportModuleOptions& options, Boolean& containedErrors)
                                   à Microsoft.PowerShell.Commands.TestModuleManifestCommand.ProcessRecord()
                                   à System.Management.Automation.CommandProcessor.ProcessRecord()
                                   --- Fin de la trace de la pile d'exception interne ---
                                   à System.Management.Automation.Runspaces.PipelineBase.Invoke(IEnumerable input)
                                   à
                                System.Management.Automation.PowerShell.Worker.ConstructPipelineAndDoWork(Runspace rs,
                                Boolean performSyncInvoke)
                                   à System.Management.Automation.PowerShell.Worker.CreateRunspaceIfNeededAndDoWork(Run
                                space rsToUse, Boolean isSync)
                                   à System.Management.Automation.PowerShell.CoreInvokeHelper[TInput,TOutput](PSDataCol
                                lection`1 input, PSDataCollection`1 output, PSInvocationSettings settings)
                                   à
                                System.Management.Automation.PowerShell.CoreInvoke[TInput,TOutput](PSDataCollection`1
                                input, PSDataCollection`1 output, PSInvocationSettings settings)
                                   à System.Management.Automation.PowerShell.Invoke(IEnumerable input,
                                PSInvocationSettings settings)
                                   à Microsoft.Windows.PowerShell.ScriptAnalyzer.Helper.GetModuleManifest(String
                                filePath, IEnumerable`1& errorRecord)
                                   à Microsoft.Windows.PowerShell.ScriptAnalyzer.BuiltinRules.UseToExportFieldsInManife
                                st.<AnalyzeScript>d__3.MoveNext()
                                   à System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
                                   à System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
                                   à Microsoft.Windows.PowerShell.ScriptAnalyzer.ScriptAnalyzer.<>c__DisplayClass78_0.<
                                AnalyzeSyntaxTree>b__1()
TargetObject                  : C:\Program
                                Files\WindowsPowerShell\Modules\PSScriptAnalyzer\1.16.1\PSScriptAnalyzer.psd1
FullyQualifiedErrorId         : RULE_ERROR,Microsoft.Windows.PowerShell.ScriptAnalyzer.Commands.InvokeScriptAnalyzerCom
                                mand
InvocationInfo                : System.Management.Automation.InvocationInfo
ErrorCategory_Category        : 7
ErrorCategory_Activity        : Invoke-ScriptAnalyzer
ErrorCategory_Reason          : CmdletInvocationException
ErrorCategory_TargetName      : C:\Program
                                Files\WindowsPowerShell\Modules\PSScriptAnalyzer\1.16.1\PSScriptAnalyzer.psd1
ErrorCategory_TargetType      : String
ErrorCategory_Message         : InvalidOperation: (C:\Program File...ptAnalyzer.psd1:String) [Invoke-ScriptAnalyzer],
                                CmdletInvocationException
SerializeExtendedInfo         : False
ErrorDetails_ScriptStackTrace : at <ScriptBlock>, <No file>: line 1
                                at <ScriptBlock>, <No file>: line 1
PSMessageDetails              :
CategoryInfo                  : InvalidOperation: (C:\Program File...ptAnalyzer.psd1:String) [Invoke-ScriptAnalyzer],
                                CmdletInvocationException
ErrorDetails                  :
ScriptStackTrace              : at <ScriptBlock>, <No file>: line 1
                                at <ScriptBlock>, <No file>: line 1
PipelineIterationInfo         : {0, 1}


Export-Clixml -InputObject $Error -Depth 5 -Path c:\temp\Issue902.txt

Issue902.txt

[bergmeister]

By looking at the stack trace, it seems to come from here but the help on this method says that it is executed synchronously. Therefore this does not make sense to me assuming there is no bug in the Invoke() method of PowerShell. This Helper class has shown bad behaviour in the past, see this issue, therefore it could be possibly related to that.
Can you give some details about your machine spec (i.e. do you have a rather fast or slow system so that I can size an Azure machine appropriately with the hope of being able to repro)? Did this only occur on Win7 or also Win10?

[LaurentDardenne]

_Therefore this does not make sense to me assuming there is no bug in the Invoke() method of PowerShell._

May be a side effect around Dispose() ?
My machine is a Core2 Duo, we can consider it as 'slow'.

With my tests, the problem is triggered more often with this form :

1..30|% { ... call PSSA }

In use it is random.
I have not tested with Win10, I can confirm with Windows 2016.

[bergmeister]

Update: I can finally reproduce now with a Standard D2s v3 (2 vcpus, 8 GB memory) Azure VM (WinServer 2016). I get the error also for another ps.Invoke() call in AvoidUsingDeprecatedManifestFields in addition to the one that we found before in the Helper class. I get the issue also with PowerShell Core 6.0.2. Implementing the using pattern instead of the manual call to dispose does not fix it, therefore this rather looks like a problem of thread safety of the Invoke() method.

[Jaykul]

Yeah, my team has been experiencing this frequently but sporadically on our (Windows 10) laptops -- but never on our beefier (Server 2016) build boxes.

Is this purely a matter of filing a bug against PowerShell's Invoke or can we work around it by using a concurrent collection, or an optional switch to do analysis in a slower, non-parallel way?

[bergmeister]

@Jaykul There are various multi-threading bugs that I have observed/analyzed in the last months and my current understanding is that there are 2 root causes:

• PowerShell's Invoke not being thread safe although the comment on it claims that it should be. There is already an issue for it in the PSCore repo: https://github.com/PowerShell/PowerShell/issues/6422
• Static members being locked too long and also indirectly being accessed/modified in a non-thread safe manner

I observed that some of those bugs are less likely to occur on machines with more than 2 cores and if they are fast.
For some of them, it can be that there is only an exception when PSSA analyses the scripts the first time (due to caches being populated), therefore an initial 'dry' run or just re-running could help with sporadic failures. But the situation is complex and can vary depending on the scripts being analysed. PSSA runs each rule in its own thread, I am not aware of an easy way to run it single threaded.
The release of 1.17 is imminent and we decided to not block the release on this to release the many changes (that either add features or fix other bugs) made in the last 6 months. This seems to be not a regression, therefore I hope that we can fix at least 1 of the 2 underlying issues in the next months and potentially just push out a 1.17.1 patch. The release was unfortunately delayed a lot due to the change of ownership (the PowerShell team itself owns it now with Steve being responsible for it as a PM and James/Klaudia being developers/testers/maintainers) and MS internal processes regarding releases changed and required more work behind the scenes.
We were and are still fighting with the existing test suite that also exhibits some shall we say interesting behaviour...
I myself am only a community maintainer btw. just to clarify that but I am happy and interested to further look into this.
P.S. Can you roughly quantify 'frequently'?

[EklipZgit]

I've been seeing this for awhile on a variety of psd1's. Happens transiently. Only happens when running psscriptanalyzer against psd1's. I've seen it on Win10 laptops and Server 2012 R2 servers. I haven't noticed a difference in frequency between the laptops or servers, but I will pay more attention to it after reading the comment above.

All machines involved have >=4 logical cores.

'Frequently' in my case means probably between 1/10th and 1/20th of the time, roughly estimated.

[Jaykul]

The bug here involves ALIASES as far as I can tell.

At this point, I can reproduce this easily on my laptop with an empty psm1 and a three line .psd1:

@{
    ModuleVersion          = "1.0.0"
    RootModule             = "broken.psm1"
    AliasesToExport        = @("anything")
}

NOTE:

• You must have a RootModule and the file must exist (i.e. broken.psm1)
• The AliasesToExport cannot be an empty array

You can actually set AliasesToExport to "*" and then you'll get failures of the rule mixed in with the exceptions -- The rule still works

This reproduces too:

New-Item Aliaser -Type Directory
New-ModuleManifest Aliaser\Aliaser.psd1 -AliasesToExport anything -RootModule aliaser.psm1 -FunctionsToExport @() -CmdletsToExport @() -VariablesToExport @() 
Set-Content Aliaser\Aliaser.psm1 "" -Encoding UTF8
1..100 | %{ Invoke-ScriptAnalyzer .\Aliaser\Aliaser.psd1 }

It's clearly a problem specific to aliases and this rule.

[devblackops]

I have seen this on a number of my modules as well. Most recently with psake.

@Jaykul It doesn't seem to always involve aliases. I rarely define those in the manifest and even setting it to AliasesToExport = @() still causes the issue.

In the psake repo, I can consistently hit this issue <90% of the time with this:

$sut = (Get-Item -Path .\src).FullName
(1..10) | % {$_; Invoke-ScriptAnalyzer -Path $sut}

[felixfbecker]

I have this issue 100% of the time running locally but not in CI.

> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.2.1
PSEdition                      Core
GitCommitId                    6.2.1
OS                             Darwin 18.6.0 Darwin Kernel Version 18.6.0: Thu Apr 25 23:16:27 PDT 2019; root:xnu-4903.261.4~2/RELEASE_X86_64
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

> Get-Module -ListAvailable PSScriptAnalyzer


    Directory: /Users/felix/.local/share/powershell/Modules

ModuleType Version    Name                                PSEdition ExportedCommands
---------- -------    ----                                --------- ----------------
Script     1.18.0     PSScriptAnalyzer                    Desk      {Get-ScriptAnalyzerRule, Invoke-ScriptAnalyzer, Invoke-Formatter}

[bergmeister]

I had a look at this again, thanks for your examples, especially the one by @Jaykul proved to be simple and error out every 5th time. Looking at the error stack traces, it seems this is a concurrency bug of Test-ModuleManifest that is being called under the hood. I will try to see if something can be done to avoid this or try fix it the PowerShell cmdlet but cannot guarantee anything.

[bergmeister]

This also happens in PowerShell 7 btw and unfortunately once I the attach the debugger to PowerShell itself the bug does not happen any (because execution must be slowed down), therefore this is definitely a race condition.

[bergmeister]

I opened 2 PRs:
One in PSSA to retry the operation and one in PowerShell itself to get the threading problem fixed at the root in PowerShell 7 going forward (hopefully, if the PR gets merged in preview2 of ps7)

[bergmeister]

Since this got fixed in PSSA 1.18.1, I assume all your cases have been fixed but was wondering if any of you have seen the error ever again?
@Jaykul @LaurentDardenne @devblackops @felixfbecker @EklipZgit

[LaurentDardenne]

I no longer encounter this error.