Recently I spent the good part of a weekend putting together Pester Tests (click here if you aren’t familiar with Pester) for my LabBuilder PowerShell module- a module to build a set of Virtual Machines based on an XML configuration file. In the module I have several cmdlets that take an XML configuration file (sample below) and return an array of hash tables as well as some hash table properties containing other arrays – basically a fairly complex object structure.
In the Pester Tests for these cmdlets I wanted to ensure the object that was returned exactly matched what I expected. So in the Pester Test I programmatically created an object that matched what the Pester Test should expect the output of the cmdlets would be:
$ExpectedSwtiches = @( @{ name="General Purpose External"; type="External"; vlan=$null; adapters=[System.Collections.Hashtable[]]@( @{ name="Cluster"; macaddress="00155D010701" }, @{ name="Management"; macaddress="00155D010702" }, @{ name="SMB"; macaddress="00155D010703" }, @{ name="LM"; macaddress="00155D010704" } ) }, @{ name="Pester Test Private Vlan"; type="Private"; vlan="2"; adapters=@() }, @{ name="Pester Test Private"; type="Private"; vlan=$null; adapters=@() }, @{ name="Pester Test Internal Vlan"; type="Internal"; vlan="3"; adapters=@() }, @{ name="Pester Test Internal"; type="Internal"; vlan=$null; adapters=@() } )
What I needed to do was try and make sure the objects were the same. At first I tried to use the Compare-Object cmdlet – this actually wasn’t useful in this situation as it doesn’t do any sort of deep property comparison. What was needed was to serialize the objects and then perform a simple string comparison. The ConvertTo-JSON cmdlet seemed to be just what was needed. I also decided to use the [String]::Compare() method instead of using the PowerShell -eq operator because the -eq operator seems to have issues with Unicode strings.
The Pester test that I first tried was:
Context "Valid configuration is passed" { $Switches = Get-LabSwitches -Config $Config It "Returns Switches Object that matches Expected Object" { [String]::Compare(($Switches | ConvertTo-Json),($ExpectedSwtiches | ConvertTo-Json)) | Should Be 0 } }
This initially seemed to work, but if I changed any of the object properties below the root level (e.g. the adapter name property) the comparison still reported the objects were the same when they weren’t. After reading the documentation it states that the ConvertTo-JSON cmdlet provides a Depth property that defaults to 2 – which limits the depth that an object structure would be converted to. In my case the object was actually 4 levels deep. So I needed to add a Depth parameter to the ConvertTo-JSON calls:
[String]::Compare(($Switches | ConvertTo-Json -Depth 4),($ExpectedSwtiches | ConvertTo-Json -Depth 4)) | Should Be 0
This then did pretty much exactly what I wanted. However, I also needed the comparison to be case-insensitive, so I added a boolean parameter to the [String]::Compare static call:
[String]::Compare(($Switches | ConvertTo-Json),($ExpectedSwtiches | ConvertTo-Json),$true) | Should Be 0
The end result was an deep object comparison between a reference object and the object the cmdlet being tested returned. It is by no means perfect as if the properties or contents of any arrays in the object are out of order the comparison will report that there are differences, but because we control the format of these objects this shouldn’t be a problem and should enable some very test strict cmdlet tests.
Edit: after writing a number of Pester tests using the approach I realized it could be simplified slightly by replacing the generation of the comparison object with the actual JSON output produced by the reference object embedded inline in a variable. For example:
The JSON can be generated manually by hand (before writing the function itself) to stick to the Test Driven Design methodology or it can be generated from the object the function being tested created (once it it working correctly) and then written to a file using:
Set-Content -Path "$($ENV:Temp)\Switches.json" -Value ($Switches | ConvertTo-Json -Depth 4)
The $switches variable contains the actual object that is produced by the working command being tested.
A Word of Caution about CRLF
I have noticed that when opening the JSON file in something like Notepad++ and copying the JSON to the clipboard (to paste into my Pester test) that an additional CRLF appears at the bottom. You need to ensure you don’t include this at the bottom of your variable too – otherwise the comparison will fail and the objects will appear to be different (when they aren’t).
This is what the end of the JSON variable definition should look like:
And this is what it should not look like (the arrow indicates the location of the extra CRLF that should be removed):
Note: I could have used the Export-CliXML and Import-CliXML CmdLets instead to perform the object serialization and comparison, but these cmdlets write the content to disk and also generate much larger strings which would take much longer to compare and ending up with a more complicated test.
Well, hopefully someone else will find this useful!
Invoke-Pester also generates XML files as output. Any thoughts on using that for reporting? Json is the new XML! Great post!
LikeLiked by 1 person
Rock on JSON 🙂 But to be honest I don’t use the XML out from pester much, but in most of the projects I’m working on I’m using continuous integration via AppVeyor which does use the XML to report on (because it is nunit compatible): https://ci.appveyor.com/project/PowerShell/xnetworking/build/tests
But honestly haven’t even thought about using it to report. I think using the output is much more useful for the Ops side because I’m thinking you’d be continuously validating the environment, not necessarily waiting for a trigger like we do with Pester testing code submissions. So being able to trace the logs and identify when the failure occurs is key in operations side of things.
I’m definitely looking forward to seeing where things are going with all of this! Thanks for your really interesting posts on this stuff! 🙂
LikeLiked by 1 person
Actually the honour goes to Kevin Marqutte! I saw his demo scripts and that really resonated with me as an Ops guy!.
Btw Great blogs! Love your presentation! Have you tried using gist for you code snippets? Helps keeps your code intact.
And thanks for the tip! I’ll be sure to look into that. Why reinvent the wheel eh? 😛
Rg./Irwin
LikeLiked by 1 person
Thank you! 🙂 Actually, I wanted to ask you how you got those Github gist snippets in there? Any tips – they seem like they are way better than my current method of code embedding.
LikeLiked by 1 person
If you have a Github account you can create gists. I’m using the standard version of WordPress so embedding is rather easy, just add the link and your good to go! 😁
LikeLiked by 1 person
Oh wow! That easy? Cool. I’ve got a GitHub account already – just never used Gists on it. From memory though there is also a PowerShell Package Provider for Gists as well – so you can pull Github Gists straight into PS. http://www.powershellmagazine.com/2015/01/14/powershell-oneget-gist-as-a-package/
LikeLiked by 1 person
Reblogged this on Tim Bolton – MCITP – MCTS.
LikeLike