Controlling your Azure Automation runbooks with Git and Azure DevOps (My PowerShell Journey through Azure DevOps – Part 1.5)

Part 1: My PowerShell Journey through Azure DevOps – Part 1

As I said in Part 1, I started down the paths of pipelines by copying some people that were using AppVeyor. AppVeyor does appear to have a set of tasks for building, testing, releasing etc, though for better or worse, the examples I found put must of the work into a PowerShell script and just used AppVeyor to execute that script, instead of parceling out build and test and deploy tasks directly.

With that as my influence, while I was still working on deployment for our modules, I also started looking at testing and deploying our Azure Automation runbooks through Azure DevOps. The old way was an Azure Automation runbook that would deploy any file that got committed to master of our runbook repository in GitHub. The new way uses Azure DevOps to run tests, do some kind of actual building, deploy to a test environment, and finally deploy to production. None of this was possible with our old system.

If you’ve never used Azure Automation this might not all make sense. Since this isn’t strictly about PowerShell modules I’ve labelled this Part 1.5, maybe think of it as bonus content. I would someday like to write more about how we’re using Azure Automation, and hopefully that future post can put this one in a little bit of context.

I’ve also made my code available on GitHub for anyone to look at. I’ll explain the repo a bit at the end but first here’s 2500 words explaining myself.

The problem statement

Continue reading “Controlling your Azure Automation runbooks with Git and Azure DevOps (My PowerShell Journey through Azure DevOps – Part 1.5)”

My PowerShell Journey through Azure DevOps – Part 1

Back in March of this year I decided I was unhappy with our PowerShell module deployment process. We have a number of open source PowerShell modules of varying quality that we’ve written to solve some need of ours. Most of them are providing interfaces to consume REST APIs in some kind of task based way. We have no automated testing, no real build process, and our deployment process is just an Azure Automation runbook that copies files whenever there’s a commit to master (plus a push to the PowerShell Gallery). A number of things came together back in March where I started learning about Pester testing and these things called Pipelines.

Continue reading “My PowerShell Journey through Azure DevOps – Part 1”

Generating a New SCCM Client GUID

  • Stop the SCCM service in Powershell using Stop-Service ccmexec and then wait for it to fully stop.
  • Rename the C:\Windows\SMSCFG.INI file to something like C:\Windows\SMSCFG.old.INI
  • Force the computer to update it’s AD certificate:
$certs = Get-Certificate -CertStoreLocation Cert:\LocalMachine\My -Template Machine
$thumbprint = $certs.Certificate.Thumbprint
certreq.exe -enroll -q -machine -cert "*$thumbprint*" Renew
  • Delete the computer object out of SCCM.
  • Restart the SCCM service using Start-Service ccmexec and then it should start up, generate a new GUID and re-create it’s object in SCCM with the new GUID.
  • Run Machine Policy Retreval & Evaluation Cycle in the SCCM client Control Panel.

Finding a Specific Setting in your Client Settings

With ConfigMgr 1806, the Application Catalog service and site roles are no longer needed, so for me it was a good opportunity to have 2 less servers to care and feed for. But because I was a dummy, in Default Client Settings I had selected my app catalog site explicitly rather than letting the client automatically pick (I have no idea why I did this). And when you make a new client setting, it inherits the settings of the default, so anyone who made a custom policy with Computer Agent had that setting. This setting being defined prevents the removal of the application catalog roles.

I have 41 Client Settings, I’m not about to look through each of them for Computer Agent settings. So Powershell to the rescue. Get-CMClientSetting will get you a list of all your Client Settings, but there won’t be any information on them. There’s a -Setting which will return all the settings of the specified type, but as it turns out, nothing to tell you which Client Setting contains that setting. So it turns out, you have to get every Client Setting, and interrogate it if it has the Computer Agent setting. With this code I was able to find the five Settings that had Computer Agent specified, and change the Default Application Catalog website point.

$allsettings = Get-CMClientSetting | select -ExpandProperty Name
foreach ($name in $allsettings) {
    if (Get-CMClientSetting -Setting ComputerAgent -Name $name) {
        write-host $name
    }
}

I could have gone farther with this, as the object returned will have a PortalURL property, but merely listing the Settings that had Computer Agent was sufficient for this. This could be generalized to look for any Client Setting with a particular setting, and further to look for specific values if needed.

Managing Multiple ConfigMgr Sites with Powershell

We’re in the middle of migrating from a single ConfigMgr site to having two separate sites for servers and desktops.  Along with test sites, that’s a lot of sites to manage!  When you’re running Powershell on a machine that is managed by a site, you can easily cd, set-location, or push-location to that site’s drive. But what if you want to manage a site different than what’s managing your machine?  You can open a powershell terminal or ISE session directly from the console, but that can be a hassle, and also won’t work for things not run interactively.  I’ve taken to putting this code at the top of all of my scripts.

Continue reading “Managing Multiple ConfigMgr Sites with Powershell”

Getting Redirected URI’s in Powershell

I recently ran into an issue where I needed the direct resource URI in a Powershell script. This is incredibly useful if you need to parse the actual URI instead of just pulling the resource the redirecting URI is pointing at. In my case I wanted the Firefox URI which points at the executable so I could pull the version out the of URI without having to download and analyze the executable.

You need to first grab the response head from an Invoke-Webrequest:

$request = Invoke-WebRequest -Method Head -Uri $Uri

Next, we need to determine if we’re using Powershell 5 or Powershell Core and pull the Absolute URI out of the request object:

if ($request.BaseResponse.ResponseUri -ne $null) {
    # This is for Powershell 5
    $redirectUri = $request.BaseResponse.ResponseUri.AbsoluteUri
}
elseif ($request.BaseResponse.RequestMessage.RequestUri -ne $null) {
    # This is for Powershell core
    $redirectUri = $request.BaseResponse.RequestMessage.RequestUri.AbsoluteUri
}

Now, sometimes you may get another redirected URI as a response. In these cases you’ll need to determine that and handle it. This is done through error handling by catching and looking for HttpResponseException matching 302 and then running the whole thing again:

if (($_.Exception.GetType() -match "HttpResponseException") -and ($_.Exception -match "302")) {
    $Uri = $_.Exception.Response.Headers.Location.AbsoluteUri
    $retry = $true
}
else {
    throw $_
}

This is a quick and easy way to pull the redirected URI’s from a given URI. Putting it all together we get the function below:

function Get-RedirectedUri {
    <#
    .SYNOPSIS
        Gets the real download URL from the redirection.
    .DESCRIPTION
        Used to get the real URL for downloading a file, this will not work if downloading the file directly.
    .EXAMPLE
        Get-RedirectedURL -URL "https://download.mozilla.org/?product=firefox-latest&os=win&lang=en-US"
    .PARAMETER URL
        URL for the redirected URL to be un-obfuscated
    .NOTES
        Code from: Redone per issue #2896 in core https://github.com/PowerShell/PowerShell/issues/2896
    #>

    [CmdletBinding()]
    param (
        [Parameter(Mandatory = $true)]
        [string]$Uri
    )
    process {
        do {
            try {
                $request = Invoke-WebRequest -Method Head -Uri $Uri
                if ($request.BaseResponse.ResponseUri -ne $null) {
                    # This is for Powershell 5
                    $redirectUri = $request.BaseResponse.ResponseUri.AbsoluteUri
                }
                elseif ($request.BaseResponse.RequestMessage.RequestUri -ne $null) {
                    # This is for Powershell core
                    $redirectUri = $request.BaseResponse.RequestMessage.RequestUri.AbsoluteUri
                }

                $retry = $false
            }
            catch {
                if (($_.Exception.GetType() -match "HttpResponseException") -and ($_.Exception -match "302")) {
                    $Uri = $_.Exception.Response.Headers.Location.AbsoluteUri
                    $retry = $true
                }
                else {
                    throw $_
                }
            }
        } while ($retry)

        $redirectUri
    }
}

Test your Chocolatey Packages in a Windows Container

Containers are a great place to test your Chocolatey packages or even packages from another source.  You don’t have to waste time creating and tearing down vms and perhaps most importantly you can avoid those “It worked on my machine” problems.  By spinning up a clean container every time you will know all your dependencies and can specify them.  There are many approaches and this is just one example.  This article assumes you have a place to run windows containers, docker, and a minimal amount of docker experience.  You can run this on a windows 10 machine, Server 2016 with Hyper-V, or Azure.

The complete code is posted below, but lets break it up a bit and walk through small pieces and point out some of the variations on the source of the package.

In this first version we’ll look at testing a package you’ve created and assume you have the .nupkg file locally.  First, its useful to define a few variable such as the path to the nupkg file.  The $waitTime variable gives chocolatey time to install before trying to fetch the logs.  All the real work is defined in the variable $ps.  $ps contains the code that downloads and installs the the latest chocolatey client.  Next it will use that client to install the chocolatey package you defined the path to in $chocoPack.  Finally it will download and run a script from Microsoft that keeps the container running after the install so you can examine it.  (The nature of a container is to stop running after its ran some piece of code.)  We’ll get to starting the container further down.

$chocoPack = ''
$waitTime=# [int] in seconds as estimate of how long package install takes
$waitUrl='https://raw.githubusercontent.com/Microsoft/Virtualization-Documentation/master/windows-server-container-tools/Wait-Service/Wait-Service.ps1'
$ps="iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'));choco install $chocoPack -y;Invoke-WebRequest -Uri '$waitUrl' -OutFile 'c:\Wait-Service.ps1';c:\Wait-Service.ps1 -ServiceName WinRm -AllowServiceRestart"

In this next section we’ll look at installing a package from the default public package repo.  The only difference here is that $chocoPack will just contain the name of the public package.  This example installs vscode because vscode is awesome.

$chocoPack = 'vscode'
$waitTime=# [int] in seconds as estimate of how long package install takes
$waitUrl='https://raw.githubusercontent.com/Microsoft/Virtualization-Documentation/master/windows-server-container-tools/Wait-Service/Wait-Service.ps1'
$ps="iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'));choco install $chocoPack -y;Invoke-WebRequest -Uri '$waitUrl' -OutFile 'c:\Wait-Service.ps1';c:\Wait-Service.ps1 -ServiceName WinRm -AllowServiceRestart"

And the last example is using your own private chocolatey server and specifies.   In teh choco install section I’ve added the $version variable and ‘-s $privateServerURL’.  -s specifies the ‘Source’, in this case the URL to your server.

$chocoPack = 'my-package'
$version='--version 1.0.0'
$privateServerURL = 'http:///choco/nuget/'
$waitTime=# [int] in seconds as estimate of how long package install takes
$waitUrl='https://raw.githubusercontent.com/Microsoft/Virtualization-Documentation/master/windows-server-container-tools/Wait-Service/Wait-Service.ps1'
$ps="iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'));choco install $chocoPack $version -s $privateServerURL -y;Invoke-WebRequest -Uri '$waitUrl' -OutFile 'c:\Wait-Service.ps1';c:\Wait-Service.ps1 -ServiceName WinRm -AllowServiceRestart"

Now lets look at actually spinning up the container.  The first line is the actual docker command to spin up the container based on windowsservercore image and tells powershell to run the code we defined above.  It also stores the container instance ID in $cid which you will need later.  The it waits to give the package time to install.  The Invoke-command will spit out the log files about the install.  Finally you can use powershell to “remote” into the container and manually look at logs or files, etc.

($cid = docker run -d $dockerArgs microsoft/windowsservercore powershell.exe -executionpolicy bypass $ps )
Start-Sleep -Seconds $waitTime
Invoke-Command -ContainerId $cid -RunAsAdministrator -ScriptBlock{
Get-Content C:\choco\logs\choco.summary.log
#Get-Content C:\choco\logs\chocolatey.log
choco list --local-only ## list install packages
}
Enter-PSSession -ContainerId $cid -RunAsAdministrator

And here it is altogether.

$chocoPack = ''
$waitTime=# [int] in seconds as estimate of how long package install takes
$waitUrl='https://raw.githubusercontent.com/Microsoft/Virtualization-Documentation/master/windows-server-container-tools/Wait-Service/Wait-Service.ps1'
$ps="iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'));choco install $chocoPack -y;Invoke-WebRequest -Uri '$waitUrl' -OutFile 'c:\Wait-Service.ps1';c:\Wait-Service.ps1 -ServiceName WinRm -AllowServiceRestart"
($cid = docker run -d $dockerArgs microsoft/windowsservercore powershell.exe -executionpolicy bypass $ps )
Start-Sleep -Seconds $waitTime
Invoke-Command -ContainerId $cid -RunAsAdministrator -ScriptBlock{
Get-Content C:\choco\logs\choco.summary.log
#Get-Content C:\choco\logs\chocolatey.log
choco list --local-only ## list install packages
}
Enter-PSSession -ContainerId $cid -RunAsAdministrator
Its also worth noting, in these examples I’m using the -d switch in ‘docker run -d’ which tells it to run detached or in the background.  You can use -it to just jump in and watch it as well.  When I first started working with docker there were issues doing this from powersehell.  The latest version of the tools fixed this problem 🙂
UPDATE:  You can not run a full GUI version of windows server in a container so if you are trying to install an application that has dependencies on .dll files not in Server Core this may not work.
Also, since chocolately is just Powershell files to install an application, this basic process can work to install any application that can be installed on Server Core from powershell/comdmandline.