PowerShell Background Jobs

When you execute a Windows PowerShell command that takes a long time to run, you might notice that you won't get your prompt back until the command finishes. By default, PowerShell "blocks" the console until the command completes. This means you're stuck waiting. You could open up a new PowerShell session, but then you'd lose access to items you've already created, such as variables and PowerShell drives.

A better approach is to push the long-running command into the background. Like Linux and other environments, PowerShell supports turning a long-running command into a background job. Later, when the job completes, you can retrieve the results.

After I tell you how PowerShell background jobs work, I'll show you how to create them. Note that all the code samples are one-line commands, except where noted.

Understanding How Background Jobs Work

PowerShell background jobs were introduced in PowerShell 2.0. In PowerShell 3.0, the concept was extended to scheduled background jobs. PowerShell workflows, also introduced in version 3.0, is yet another job type. For this discussion, I'm going to concentrate on plain background jobs.

When you run a command as a background job, PowerShell creates a new session that you can't see and executes the command within it. Each job will have one parent or executive job and one or more child jobs. The child jobs do the actual work. Most of the time, though, you use the job cmdlets with the parent job and it handles communicating with the child jobs. You'll see this in more detail throughout this discussion.

Using the Start-Job Cmdlet to Create Background Jobs

The easiest way to create a PowerShell background job is to use the Start-Job cmdlet. You can specify a scriptblock with the cmdlet. For example, the following command runs a simple scriptblock that gets the numbers between 1 and 50 as a background job:

Start-Job {1..50}

You can pass arguments to a scriptblock with this one-line command:

Start-Job {Param($log,$newest) Get-EventLog $log  -Newest $newest} -ArgumentList "system",25

Or you can run scripts:

Start-Job -FilePath C:scriptsSend-UptimeMail.ps1  -ArgumentList (Get-Credential [email protected])

You can see the results of these three commands in Figure 1. I ran these commands in PowerShell 3.0. However, they'll also work in PowerShell 2.0, although you won't see the PSJobTypeName column.

Figure 1: Running Scriptblocks and Scripts in the Background

The output from Start-Job is a job queue object. You use the -Job cmdlets to manage jobs and their results. You can reference jobs by an ID. In Figure 1, notice how the job IDs jump from 1 to 3 to 5. That's because there are child jobs. If you want to see the child jobs for a particular job, you can use the -IncludeChildJob parameter, like this:

Get-Job 3 -IncludeChildJob

Figure 2 shows the results. When you start running jobs against remote computers, you'll start seeing more child jobs, although usually all you need is the parent job.

Figure 2: Inspecting the Child Jobs

Besides using a job's ID to retrieve information about a job, you can use the job's name:

Get-Job Job3 | Select Name,State

In this case, the results show that job 3 is named Job3 and has finished running. A name like "Job3" is probably fine in an interactive session, but you might want to assign a more meaningful name to your job. Take, for example, the command:

Start-Job { dir $env:TEMP -Recurse -File } -Name TempDir

This command creates a job named TempDir, which gets a list of all the files in the %TEMP% folder. That job might take a long time to run, so it's a great candidate for a background job. I'll show you how to retrieve the results later.

Using the Get-WMIObject Cmdlet to Create Background Jobs

Another cmdlet that supports the background job infrastructure is Get-WMIObject. If you look at this cmdlet's Help file, you'll see that it has the -AsJob parameter. This parameter lets you push long-running Windows Management Instrumentation (WMI) queries to the background. You simply add the parameter to the end of the command, like this:

Get-WmiObject Win32_Product -comp localhost,novo8,  chi-win8-01,chi-dc04 -AsJob

In this case, the WMI job (job 9) failed. To find out what went wrong, you can check the child jobs:

Get-Job 9 -IncludeChildJob

In Figure 3, you can see all the child jobs, including those that failed. I'll discuss those failures later when I cover how to retrieve job results.

Figure 3: Checking the Child Jobs of the WMI Job That Failed

Using the Invoke-Command Cmdlet to Create Background Jobs

The last way to create background jobs is with the Invoke-Command cmdlet, which also has an -AsJob parameter. Invoke-Command is typically used to execute a scriptblock or file on one or more remote computers:

Invoke-Command { gwmi win32_logicaldisk -filter  "drivetype=3"} -comp chi-dc01,chi-dc02,chi-dc04  -AsJob

Without -AsJob, you'd have to wait until the command finishes running on all the computers listed. With -AsJob, the command runs and stores the results in the background. (Note that gwmi is the alias for Get-WMIObject.)

Whether you use the Invoke-Command cmdlet's or the Get-WMIObject cmdlet's -AsJob parameter depends on the situation. In some instances, using the Invoke-Command cmdlet's -AsJob parameter might be the only option because the WMI cmdlets require remote procedure calls (RPCs) and DCOM, which might be blocked by firewalls. In contrast, Invoke-Command uses PowerShell remoting and a single port. Another consideration is that the Invoke-Command cmdlet has a -JobName parameter, whereas the Get-WMIObject cmdlet doesn't have one.

Managing Background Jobs

No matter how you start the background job, you use the same set of cmdlets to manage the job and its results. As the name implies, Get-Job gets the background job object. Without any parameters, Get-Job will display all the jobs in your session. You can also specify a job by name, ID, or event state:

Get-Job 1Get-Job TempDirGet-Job -State Failed

Figure 4 shows results from these three commands.

Figure 4: Getting Jobs by Name, ID, and Event State

If needed, you can expand the child jobs. For example, to see the child jobs for the jobs that failed, you can run the command:

Get-Job -State Failed -IncludeChildJob

The results from this command are similar to those shown in Figure 3.

Job objects remain until you close your PowerShell session. However, you can manually remove objects from the job queue with the Remove-Job cmdlet. You can identify the job to be removed by its name, ID, or state:

Remove-Job 1Remove-Job TempDirRemove-Job -State Failed

As you can see in Figure 5, the Remove-Job cmdlet supports the -WhatIf parameter. The cmdlet also accepts pipelined input.

Figure 5: Using the -WhatIf Parameter with the Remove-Job Cmdlet

Retrieving Job Results

To get job results, you use the Receive-Job cmdlet. You can specify the job by name or ID. For example, to get the results for job 1, you'd run the command:

Receive-Job 1

As Figure 6 shows, this job was just getting the numbers between 1 and 50.

Figure 6: Retrieving the Results for Job 1

But let's say you want to get the results again, so you try running the same command:

Receive-Job 1

No matter how many times you run this command, you won't get any results. To see why, let's run a different command to look at that job:

Get-Job 1

As Figure 7 shows, the job queue has no more data. (I highlighted the relevant information in this figure.)

Figure 7: Determining Why There Are No Results

When you receive a job, PowerShell writes all the data to the pipeline and clears the queue unless you use the -Keep parameter, like this:

Receive-Job 3 -Keep

Figure 8 shows the results. As long as you continue to use the -Keep parameter, PowerShell will retain the job results. What I typically do is save the results to a variable in addition to keeping them. This way, I can work with the results in the variable and retrieve them as many times as necessary.

Figure 8: Using the -Keep Parameter to Retain Job Results

For example, the following code saves the results from job 3 to a variable named $events, then uses those results:

$events = Receive-Job 3 -Keep$events.count$events | select Source -Unique

You can see the results in Figure 9.

Figure 9: Saving Job Results in a Variable

Even if a job fails, I still recommend receiving the job results. Recall that a WMI job had failed earlier. If you use Receive-Job to retrieve a failed job's results, you'll get data for any child jobs that successfully completed as well as any error messages, as Figure 10 shows. The error messages in Figure 10 are self-explanatory.

Figure 10: Retrieving the Results for the Failed WMI Job

You can work with job results just as if you had run the command interactively:

Receive-Job TempDir -Keep | measure Length -Sum

However, as I mentioned previously, you might want to save the results to a variable first by running a command such as:

$data = Receive-Job TempDir -Keep | measure Length -Sum$data.sum/1mb

Don't forget to keep the results every time you run a Receive-Job command. One step you might try in PowerShell 3.0 is to set a default parameter value for Receive-Job:

$PSDefaultParameterValues.Add("Receive-job:Keep",$True)

If you run this command, PowerShell will automatically set the -Keep parameter to $True in the current session. If you want this behavior all the time, you need to put this command into your PowerShell profile script.

Stopping and Waiting for Jobs

By now I hope you're realizing the benefit of PowerShell background jobs. You can execute a long running task on 1, 10, or 1,000 computers and retrieve the results later. Just remember to get the results before you close your PowerShell session.

There might be times when you need to stop a job. For example, suppose you started running a job like this:

Invoke-Command {dir c: -Recurse } -ComputerName  $computers -AsJob -JobName RemoteDir

You then realize you made a mistake and want to kill the job. To do that, you simply use the Stop-Job cmdlet:

Stop-Job RemoteDir

Keep in mind that there might still be results worth receiving and saving, even though you stopped the job.

Finally, although the whole point of a background job is to get your prompt back so you can keep working, there might be situations in which you need the job to finish before you can do anything else. For example, in a script, you might kick off a job, execute some other code, then need the job results before continuing. This is where the Wait-Job cmdlet comes in handy.

With Wait-Job, PowerShell will block until the job completes. By default, the cmdlet waits indefinitely, but you can set a timeout value (in seconds). Typically, I use Wait-Job interactively when I need the job results but don't want to keep typing Get-Job to check the status.

You can pipe a job object to Wait-Job:

Invoke-Command {Get-Hotfix} -ComputerName $computers  -AsJob -JobName hotfix | Wait-Job

You'll have to wait for the job to complete, but you'll receive the job results when the prompt returns. You can even include the code that retains and uses the job results by using a series of commands like this:

$hot = Invoke-Command {Get-Hotfix} -ComputerName  $computers -AsJob -JobName hotfix | Wait-Job |  Receive-Job -Keep$hot | group PSComputername -NoElement$hot | where PSComputername -eq 'chi-dc01' |  sort Description | group Description -NoElement |  sort

This way, you can keep the results in the job queue. Figure 11 shows sample results. Although you typically use background jobs for long-running commands, PowerShell doesn't really care if you use them for this purpose.

Figure 11: Including the Code that Retains and Uses the Job Results

Be More Efficient and Productive

I haven't covered every parameter of every job cmdlet in every scenario. So, you need to take some time to read the full Help file and check out the examples for all the cmdlets I demonstrated here. In addition, read the Help topics about_Jobs, about_Job_Details, and about_Remote_Jobs. (By the way, if you look at the documentation, you'll see the Suspend-Job and Resume-Job cmdlets. Those cmdlets apply only to workflow jobs and not anything I discussed here.) As PowerShell continues to dominate the world of the IT professional, it's essential that you understand how to use background jobs so that you can be more efficient and productive.