Powershell v2.0 Using multiple threads

multithreading, powershell

Solution

You can run each iteration of the loop in a background job which is not the same a seperate thread in that it is a whole other PowerShell.exe process. Data is passed from the background processes through serialization.

To approach it using background jobs you'll need to define a script block that will do that actual work and then call the script block with parameters in each iteration of the loop. The script block can report back status via `Write-Output` or by throwing an exception.

You'll probably want to throttle how many concurrent background jobs are running. Here's an example of how to throttle:

$jobItems = "a", "b", "c", "d", "e"
$jobMax = 2
$jobs = @()

$jobWork = {
    param ($MyInput)
    if ($MyInput -eq "d") {
        throw "an example of an error"
    } else {
        write-output "Processed $MyInput"
    }
}

foreach ($jobItem in $jobItems) {
    if ($jobs.Count -le $jobMax) {
        $jobs += Start-Job -ScriptBlock $jobWork -ArgumentList $jobItem
    } else {
        $jobs | Wait-Job -Any
    }
}
$jobs | Wait-Job

As an alternative you might try eventing. Take a look at this thread for some examples of how to implement concurrency using events.

PowerShell: Runspace problem with DownloadFileAsync

You might be able to replace DownloadFileAsync with OpenReadAsync

Problem

Basic script idea: Hello. I've created a powershell script which I use to check the filesizes of certain executables, and then keep them in a text file. Next time the script runs, if a filesize differs it will replace the one in the text file with the new one. The structure: I have a main script and a folder which contains many scripts, each for every executable of which I want to check the filesize. So the scripts in the folder will return a string containing the link to the executable, which will be fed to the main script. The code: ``` $progdir = "C:\script\programms" $items = Get-ChildItem -filter *.ps1 -Path $progdir $webclient = New-Object System.Net.WebClient $filesizes = get-content C:\updatechecker\programms\filesizes if ($filesizes.length -ne $items.length) { if ($filesizes.length -eq $null) { Write-Host ("Building filesize database...") -nonewline } else { Write-Host ("Rebuilding filesize database...") -nonewline } clear-content C:\programms\filesizes for ($i=0; $i -le $items.length-1; $i++) { $command = "c:\programms\" + $items[$i].name $link = & $command $webclient.OpenRead($link) | Out-Null $filesize = $webclient.ResponseHeaders["Content-Length"] $filesize >> C:\programms\filesizes } echo "Done." } else { ... ``` Question: This for loop is the one I want to run in parallel. I need your advice on how to do this since I'm new to powershell. I tried to implement a few things I found but they didn't work correctly (took very long to finish, output errors, multiple entries of filesizes in my filesizes file). I suspect it's a synchronization issue and somehow I need to lock the critical parts. Isn't there anything like omp parallel for in powershell? :P Any help,advice on how to achieve this would be appreciated :) edit: ``` Get-Job | Remove-Job -Force $progdir = "C:\programms" $items = Get-ChildItem -filter *.ps1 -Path $progdir $webclient = New-Object System.Net.WebClient $filesizes = get-content C:\programms\filesizes $jobWork = { param ($MyInput) $command = "c:\programms\" + $MyInput $link = & $command $webclient.OpenRead($link) | Out-Null $filesize = $webclient.ResponseHeaders["Content-Length"] $filesize >> C:\programms\filesizes } foreach ($item in $items) { Start-Job -ScriptBlock $jobWork -ArgumentList $item.name | out-null } Get-Job | Wait-Job Get-Job | Receive-Job | Out-GridView | out-null echo "Done." ``` Edit 2: Used code I found here: http://ryan.witschger.net/?p=22 ``` $mutex = new-object -TypeName System.Threading.Mutex -ArgumentList $false, “RandomGlobalMutexName”; $MaxThreads = 4 $SleepTimer = 500 $jobWork = { param ($MyInput) $webclient = New-Object System.Net.WebClient $command = "c:\programms\" + $MyInput $link = & $command $webclient.OpenRead($link) | Out-Null $result = $mutex.WaitOne(); $file = $webclient.ResponseHeaders["Content-Length"] $file >> C:\programms\filesizes $mutex.ReleaseMutex(); } $progdir = "C:\programms" $items = Get-ChildItem -filter *.ps1 -Path $progdir $webclient = New-Object System.Net.WebClient $filesizes = get-content C:\programms\filesizes Get-Job | Remove-Job -Force $i = 0 ForEach ($item in $items){ While ($(Get-Job -state running).count -ge $MaxThreads){ Start-Sleep -Milliseconds $SleepTimer } $i++ Start-Job -ScriptBlock $jobWork -ArgumentList $item.name | Out-Null } ```

Original source

Related problems