Powershell: Question concerning foreach - parallel

[Schwanowski]

Hi there,

im really like the new foreach - parallel function but I need to be able to store the results in e.g. a global variable.

Look if you call a webservice in parallel, you need to process the result and save it to some e.g. global variable.

`
$items = $myGlobalVariable.Keys | Where-Object {$myGlobalVariable[$_].run -eq $false}

$items | ForEach-Object -parallel {
$response = Invoke-RestMethod xxxx
$myGlobalVariable[$_].ID = $response.ID
}
`

Any suggestions?

[Schwanowski]

Seems to be a duplicate to https://github.com/PowerShell/PowerShell/issues/12240

[vexx32]

You can reference items from the main runspace with $using:varName; $global: will not work as far as I'm aware. However, a majority of collections are not threadsafe, and you will encounter what can only be described as 'undefined behaviour' when trying to use them from more than one thread/runspace simultaneously as you are here.

You can prepare a hashtable for use in threaded operations using the static Synchronize method, like so:

$results = [hashtable]::Synchronized(@{ })

Which you can then safely use via $using:results in Foreach-Object -Parallel. There are other, more sophisticated collection types that can be used in a multithreaded context, but that's probably closest to what's familiar. Multithreading something correctly isn't easy, and the complexity of some of the threadsafe types reflects that.

Do note that $using:var doesn't permit assignment, or (I think?) indexing operations... so you may need to do something like $myResults = $using:Results at the start of the parallel scriptblock, and then reference the hashtable with that when adding entries.

[Schwanowski]

Thanks already for your response, I will try this later today.

But one question: I have read some time ago that I should use System.Collections.Generic.Dictionary instead of an hashtable is this true? Can I use a dictonary instead of a hashtable or is this not possible in this case?

Thanks,

Alex

[Schwanowski]

I have tested it with
$myResults = $using:Results
and it works all fine

Thanks

[vexx32]

You can use a Dictionary, sure, but you can't use _that_ kind of dictionary due to the aforementioned threading issues. It also doesn't have a Synchronized() method like Hashtable does.

If you want a type-safe _and_ thread-safe dictionary, you'll need to look to System.Collections.Concurrent.ConcurrentDictionary.

As you'll see if you look through some of its methods, it has to be handled a good bit differently, with things like TryAdd() which _can_ fail and may need to be retried. It's a bit more complicated, but probably a lot more efficient to work with.

[Schwanowski]

Thank you so much for your support