Find a bitmap on the VM screen

06/16/2016

2 minutes to read

In this article

Continuing with my series on how to do interesting automation stuff with Hyper-V – today I want to show you how to use PowerShell and the AForge.NET library to locate a bitmap on a virtual machine screen.

For example – imagine that I had a test running in a Windows Server Core 2016 virtual machine:

And I wanted to be able to tell if PowerShell was running interactively – but I did not want to interfere with the guest operating system in anyway. What could I do? Well – I could get the screen and look for a bitmap that told me that PowerShell was running. Something like this (enlarged to make it easier to view):

Fortunately – this is quite easy to do thanks to the handy AForge.NET libraries. The result looks something like this:

If I run this script with a single PowerShell Window open I get this result:

Rectangle Similarity --------- ---------- {X=381,Y=371,Width=13,Height=12} 1

One perfect match at X=381, Y=371

If I open up two more PowerShell Windows and run this script I get this result:

Rectangle Similarity --------- ---------- {X=303,Y=522,Width=13,Height=12} 1 {X=535,Y=355,Width=13,Height=12} 0.993087 {X=74,Y=700,Width=13,Height=12} 0.993087

One perfect match – and two close matches. The reason for the difference here is that the close matches are PowerShell Windows that do not have focus – so the graphic is slightly different.

One final point to make about this code sample: Hyper-V produces a 16bit RGB565 image for a screen capture. Unfortunately the AForge.NET libraries do not accept this format. So in my sample code you will see that I upsample into 24bit RGB in order to make everything work.

Cheers,

Ben