Syntax checking your Powershell code with Pester

If you just want to syntax check your Powershell code with Pester, scroll to the bottom and grab my describe block. If you’d like you’d like to go on a little journey with me, keep reading.

I’m in the process of getting as much of team’s Powershell code through a CI pipeline using Azure DevOps. Due to some issues with accidentally pushing code with syntax and encoding errors to production, one of the first things I wanted to do was just simply validate that the code was valid. I found a post on the PowerShell.org forums from 2014 that inspired me to write this test:

Describe "General project validation: $projectname" {

    $scripts = Get-ChildItem $projectRoot -Include *.ps1, *.psm1, *.psd1 -Recurse

    # TestCases are splatted to the script so we need hashtables
    $testCase = $scripts | Foreach-Object {@{file = $_}}         
    It "Script <file&gt; should be valid powershell" -TestCases $testCase {
        param($file)

        $file.fullname | Should Exist

        $contents = Get-Content -Path $file.fullname -ErrorAction Stop
        $errors = $null
        $null = [System.Management.Automation.PSParser]::Tokenize($contents, [ref]$errors)
        $errors.Count | Should Be 0
    }
}

When I first implemented this I had to do some work on our existing code to replace weird dash characters, and the PSParse made it difficult to find the source of the errors, but it felt like a one time job and this would be good enough. It turns out I was wrong. Quite a few things came together all at once to make me fix this. First, most of our files are encoded as UTF8 but Get-Content defaults Windows 1252 which doesn’t have as many characters. Somebody pasted some code in from the internet and it had an en dash instead of a hyphen. The en dash exists in UTF8 but not Windows 1252, so the validation failed because the got replaced by â€" and threw off all the strings.

An en dash is valid Powershell, the interpreter turns it into a regular hyphen on the fly, as well as a few other funny characters, usually ones that happen when copy/pasting from the web or Word. The ISE will alert on these characters, but VSCode saving to UTF8 won’t, and in fact the code will work perfectly. But my validation would fail. I’m just OCD enough that I want to figure out a way to report errors on those characters when they appear as part of a parameter name or in quotes, but since they’re otherwise valid characters, I’m fine with them appearing inside strings generally. I could force everything to Windows 1252 to validate, or just find every occurence of those special characters, but I demand perfection, so I wanted to allow those characters inside strings, just not as part of the language syntax.

So I learned quite a bit today about Powershell parsing and abstract syntax trees (ASTs). It turns out the PSParser is old fashioned, and I should use Language.Parser instead, which creates ASTs. And with ASTs you’re able to do all sorts of things like drill down into every single token and run tests against their properties. The Language Parser can differentiate between parameter names and strings and all sorts of other things. It’s quite a bit to take in, if you want to play around you can read the documentation and install the excellent ShowPSAst module to get a graphical representation of an AST.

After all that I ended up with something that I think will work for me. I want to give a special thanks to everyone in the Powershell Slack and Chris Dent especially for their help. Chris had written a test that would find the hyphens, and I added code to check for quotes. This is actually a Pester test that tests itself, to prove the whole thing works.

Here’s the actual test I put in my code.

Describe "General project validation" {

    $scripts = Get-ChildItem $projectRoot -Include *.ps1, *.psm1, *.psd1 -Recurse
    $predicate = {
        param ( $ast )

        if ($ast -is [System.Management.Automation.Language.BinaryExpressionAst] -or
            $ast -is [System.Management.Automation.Language.CommandParameterAst] -or 
            $ast -is [System.Management.Automation.Language.AssignmentStatementAst]) {

            if ($ast.ErrorPosition.Text[0] -in 0x2013, 0x2014, 0x2015) {return $true}
            
        }
        if ($ast -is [System.Management.Automation.Language.CommandAst] -and
            $ast.GetCommandName() -match '\u2013|\u2014|\u2015') {return $true}

        if (($ast -is [System.Management.Automation.Language.StringConstantExpressionAst] -or
                $ast -is [System.Management.Automation.Language.ExpandableStringExpressionAst]) -and 
            $ast.Parent -is [System.Management.Automation.Language.CommandExpressionAst]) {
            if ($ast.Parent -match '^[\u2018-\u201e]|[\u2018-\u201e]$') {return $true}
        }
    }

    # TestCases are splatted to the script so we need hashtables
    $testCase = $scripts | Foreach-Object {@{file = $_}}         
    It "Script <file&gt; should be valid powershell" -TestCases $testCase {
        param (
            $file
        )
        $script = Get-Content -Raw -Encoding UTF8 -Path $file
        $tokens = $errors = @()
        $ast = [System.Management.Automation.Language.Parser]::ParseInput($Script, [Ref]$tokens, [Ref]$errors)
        $elements = $ast.FindAll($predicate, $true)

        $elements | Should -BeNullOrEmpty -Because $elements
    }
}

I also want to highlight the excellent Vscode extension, unicode-Substitutions, which will give you warnings about these bad characters right in the editor, which will hopefully prevent your Pester tests from failing.

One thought on “Syntax checking your Powershell code with Pester

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.