Dude, where’s my var? – Understanding scoping in Universal Dashboard

Universal Dashboard is a web framework for PowerShell.

Universal Dashboard allows users to create websites and REST APIs using just PowerShell. Users of UD expect that it behaves much like any other PowerShell script as it looks just like a PowerShell script. To ensure that the web server provides as much performance as possible, Universal Dashboard uses background runspaces to allow for concurrent execution of script blocks that define the dynamic functionality of the dashboard. Due to this, variable, module and function scoping can be a little weird when dealing with UD.

In this blog post, we will look into the intricacies of scoping with Universal Dashboard endpoints.

What’s in a runspace?

To understand how Universal Dashboard functions, we need to understand a little bit about the runspace feature of PowerShell. Runspaces are somewhat isolated containers for PowerShell execution. When you start a PowerShell console, you will be invoking your commands in the default runspace. You can view runspaces by using the Get-Runspace command.

Windows PowerShell Copyright (C) Microsoft Corporation. All rights reserved. PS C:\> Get-Runspace Id Name ComputerName Type State Availability -- ---- ------------ ---- ----- ------------ 1 Runspace1 localhost Local Opened Busy

Runspaces allow for a single pipeline to be executing at once. This means you can’t run two commands in the same runspace. Because of this, runspaces can be kind of synonymous with threads in other programming languages.

In addition to controlling the execution of a script, they also maintain the constructs that we are used to in any PowerShell environment. Variables, functions, modules, and providers are examples of artifacts that are tied to a runspace. Most of the time we are working with PowerShell, we are dealing with a single runspace so these types of artifacts seem to to be exist indefinitely, and globally, in the environment.

Note there are scopes within the runspace that control the lifetime and accessibility of these artifacts. Read about_Scopes for more information.

When we create a new runspace, that means we can now execute another PowerShell command in tandem and have a completely new set of variables, modules, and functions in that runspace’s scope.

A good way to demonstrate this is to use the ThreadJob module. This module creates new runspaces in the background so that multiple commands can be run at once. The difference between a standard PowerShell job and a runspace job is that a standard PowerShell job actually starts a new process rather than creating a new runspace.

If we start a thread job and then use the Get-Runspace command, you’ll see that we now have more than one runspace.

PS C:\> Start-ThreadJob -ScriptBlock { Start-Sleep 10 }

Id Name PSJobTypeName State HasMoreData Location Command 1 Job1 ThreadJob Running False PowerShell Start-Sleep 10

PS C:\> Get-Runspace

Id Name ComputerName Type State Availability 1 Runspace1 localhost Local Opened Busy 2 Runspace2 localhost Local Opened Busy

To demonstrate how each runspace has its own scope, let’s create a variable in the default runspace and then try to access it in the background runspace.

As you’ll see below, the Receive-Job call did not return a value. This is because $MyVar does not exist in the background runspace.

PS C:\> $MyVar = "Test" PS C:\> Start-ThreadJob -ScriptBlock { $MyVar } | Wait-Job | Receive-Job PS C:\>

One thing to note is that runspaces are different than .NET variable scoping. You can define static variables in .NET that are available across runspaces. Assemblies loaded into a PowerShell process are also not tied to a runspace but global throughout the process. Using this knowledge allows us to transfer a variable state across runspaces.

Runspace Initialization

It’s possible to create a runspace and initialize the runspace with lots of different PowerShell artifacts. We can pass in variables, functions, modules and even snap-ins (remember those!?).

This is accomplished with the InitialSessionState class. It provides the ability to define the state of the runspace when it’s created. The ThreadJob module takes care of this for us. UD also has a helper to set up the initial session state. The New- UDEndpointInitialization cmdlet actually creates an InitialSessionState object that is then used to initialize the runspaces it uses when executing endpoints.

PS C:\> New-UDEndpointInitialization -Variable MyVar | Get-Member TypeName: System.Management.Automation.Runspaces.InitialSessionState

If we look at the InitialSessionState returned by New-UDEndpointInitialization you’ll see that the variable is set.

PS C:\> $InitialSessionState = New-UDEndpointInitialization -Variable MyVar PS C:\> $InitialSessionState.Variables | Where-Object Name -eq 'MyVar' Value : Description : Options : None Attributes : {} Visibility : Public Name : MyVar PSSnapIn : Module :

Whenever you execute an endpoint in UD it now has access to this variable because it’s part of the initial session state.

The InitialSessionState class has more options that are available with New-UDEndpointInitialization . You can always call the object directly to add more artifacts to the session state.

PS C:\> $InitialSessionState = New-UDEndpointInitialization -Variable MyVar PS C:\> $InitialSessionState.Assemblies.Add("System.Windows.Forms.dll")

Runtime Variables

Although initial session state is valuable when passing in global variables you’d like to use throughout your dashboard, they aren’t good for variables that are set during runtime or change based on data brought into the script. UD has a couple of different ways of setting these variables during execution.

Auto-scoped variables

Auto-scoped variables are variables that are available in the child-endpoints. An example of this is where you have an endpoint create a control that also has an endpoint. This requires UD to pass in the variable during execution of the endpoint script block.

For example, you might have a dynamic UDColumn that creates a UDGrid. In this case, we have two different endpoints. If you read the last post on performance, you’ll know that this actually results in two HTTP requests and thus executes two script blocks independent of each other; one for each Endpoint.

New-UDColumn -Endpoint { $Data = Get-Data New-UDGrid -Title 'Data' -Endpoint { $Data | Out-UDGridData } }

When New-UDGrid is executed, the Endpoint is stored internally as a string.

Additionally, New-UDGrid (or any cmdlet with an Endpoint parameter) then look at the script block to see which variables are defined within it. In this case, it finds the $Data variable is used. It then calls Get-Variable to get the value of $Data and stores that along with the Endpoint script block string.

When the endpoint itself is actually executed, the endpoint script is then parsed back into a script block. Then the $Data variable is set as part of that execution so it is available when the endpoint is running.

This works the same for all the built-in dynamic variables such as $Response , $Request , and $User .

You can think of the resulting endpoint being executed as something like this.

Set-Variable -Name 'Data' -Value $DataFromAnotherRunspace Set-Variable -Name 'Request' -Value $RequestFromAnotherRunspace Set-Variable -Name 'Response' -Value $ResponseFromAnotherRunspace $Data | Out-UDGridData

This effectively moves the variable from one runspace to another. After a runspace is executed, ResetRunspace is called to reset the runspace back to the initial session state. This means that any variables that you set during execution of an endpoint are cleared after execution.

One of the caveats with auto-scoped variables is that they do not work with built-in variables such as $_ or $PSItem . You can see that this doesn’t work in a regular PowerShell console.

PS C:\> Set-Variable -Name _ -Value "test" PS C:\> $_ PS C:\>

Explicit-Scoped Variables

As seen with the caveat of auto-scoped variables, you can see that certain PowerShell constructs don’t behave as you expect in UD.

For example, the below ForEach-Object doesn’t work as expected. The $_ will be $null in the New-UDEndpoint endpoint. This is because the PowerShell engine is resetting the $_ variable after we set it.

1..100 | ForEach-Object { New-UDButton -OnClick { Show-UDToast -Message $_ } }

To work around this, you can use the New-UDEndpoint cmdlet to specify an argument list to be passed to the endpoint. The reason this works is that when New-UDEndpoint is called, it is still in the same runspace as the ForEach-Object . This means that we still have access to the current value of the $_ variable. We then store that variable value along with the endpoint and make it accessible via the $ArgumentList variable when it’s executed later.

1..100 | ForEach-Object { New-UDButton -OnClick (New-UDEndpoint -Endpoint { Show-UDToast -Message $ArgumentList[0] } -ArgumentList $item) }

Cache and Session Scoping

Cache and session scoping allow variables to be stored globally and per user. The Cache scope is considered global to the current instance of UD. If you set a cache variable, it’s available in any endpoint.

Cache scoping works by defining a PS Provider (like the file system provider). When you set a variable using cache scoping, it’s actually doing something like Set-Item on the cache provider. Internally, the cache provider takes advantage of the ASP.NET Core MemoryCache class to store those variables in memory.

Since the memory cache is global and available in all endpoints, accessing those variables work from any endpoint. Depending on the order of operations, you can encounter issues where setting a variable in the cache happens after trying to access it so you may need to initialize the variable ahead of time.

You can access the cache scope outside of the dashboard entirely.

Import-Module UniversalDashboard $Cache:Init = 'InitMe'

Session scope works a bit differently. It takes advantage of the current user’s session. When a user connects to UD, they are granted an ASP.NET session cookie. This cookie identifies the user’s session. Data that is stored within the session cache can only be set and retrieved by the browser that has that session cookie.

Aside from that, the session scope works much the same as the cache scope.

In Conclusion

Scoping can be a little weird in UD due to the nature of multiple runspaces. Remember that auto-scoping should work in most cases but you can resort to explicit-scoping when necessary. The cache and session scopes can also be used to avoid these types of scoping issues all together but overuse of the session scope can cause race conditions due to the threaded nature of the web server.

Best practice would be to use auto-scoping, where possible. When dealing with loops and automatic variables, such as $_ , you should use explicit-scope. When dealing with user state, use the session scope but know that there is no guarantee as to which endpoint will load first. Finally, the cache variable can be used to globally available data.