Building the SQL backed DSC Pull Server

As stated in part one, the NuGet build was for internal use of a small-scale deployment, the DSC pull server deployment was also the same, for small deployments only (up to five hundred nodes). The SQL backed DSC pull server can support up to 3500 nodes, this can be increased through traffic load balancing between more than a single pull server to support more clients if needed.

We will do this using a DSC configuration document from start to finish, just to demonstrate the power of DSC, even if I am only using it in a push configuration.

We are going to start with an empty PowerShell Configuration document; furthermore, I will build this in a PowerShell module so it can be stored on our NuGet server and provide version control on our document. All code will be contained in the module, let's start with an empty framework:

# *****************************************************************
# *                     <module name>.psm1
# *****************************************************************
# * Description: <description>.
# * 
# * Version: 1.0.0
# *
# * Meade Automation
# * Author: Mitch Meade ~ mmeade@meadeautomation.com
# *****************************************************************
# * Version * Date    * Changes
# *****************************************************************
 
# *****************************************************************
# * Initialize the <verb>-<new function name> function
# *****************************************************************
Function <verb>-<new function name> {
    <#
        .SYNOPSIS
             
        .DESCRIPTION
           
        .EXAMPLE
           
        .LINK
    #>

    Param()
   

   Configuration "<configuration name>"
    {
        param ()
    }
}

This is a PowerShell module and a function; the function contains a Configuration. The first step we need to take is to lay down some groundwork.  We are going to install SQL Express with Chocolatey; the first step is to install Chocolatey and then we can install MS SQL Express. We need to install the PowerShell DSC Resource modules before we can do anything else with a DSC configuration, this code is placed in an external script used to load prerequisites and is loaded when the module is loaded, obviously this module needs administrative rights when it is loaded.

@('cChoco','xPSDesiredStateConfiguration','NetworkingDsc') | %{$x = $_;If (!(Get-Module $x -ListAvailable)) {Install-Module $x -Force};Import-Module $x -Force}

Then we will add our first configuration statement to install Chocolatey:

cChocoInstaller installChoco
{
    InstallDir                  = 'C:\ProgramData\chocolatey'
}

Then we will install the DSC server Windows feature and the IIS Admin tool:

WindowsFeature dscService
{
    Name                        = 'DSC-Service'
    Ensure                      = 'Present'
}
 

WindowsFeature iisAdminConsole
{
    Name                        = 'Web-Mgmt-Console'
    Ensure                      = 'Present'
}

Our next configuration will instruct Chocolatey to install SQL Express 2022 if it is not already installed with Chocolatey. I override the default command line arguments for this chocolatey package to add the invoker and system account as administrators. This is for testing, provide a service account with minimum permissions for a production server (Note, with chocolatey, the idempotency level here is with chocolatey. If chocolatey does not think its installed, it will be reinstalled).

cChocoPackageInstaller sqlExpress2022
{
    Name                        = ‘sql-server-express’
    chocoParams                 = '-o -ia "/IACCEPTSQLSERVERLICENSETERMS /Q /ACTION=install /INSTANCEID=SQLEXPRESS /INSTANCENAME=SQLEXPRESS /UPDATEENABLED=FALSE /ADDCURRENTUSERASSQLADMIN /SQLSYSADMINACCOUNTS=SYSTEM" -f -y'
    Ensure                      = 'Present'
    DependsOn                   = '[cChocoInstaller]installChoco'
}

This is the first configuration using the DependsOn clause, this will force this configuration to wait until Chocolatey is installed. The format is simply the name of the DSC resource and the name of the configuration item with the name of the resource in square brackets.



Next, we will invoke the Firewall resource from the NetworkingDSC module to configure the Windows Firewall.

Firewall allowInboundSQLPorts
{
    Name                        = 'AllowInSQLPorts'
    DisplayName                 = 'Allow Inbound-SQL Ports'
    Action                      = 'Allow'
    Description                 = 'To allow the DSC pull server to function'
    Direction                   = 'Inbound'
    Enabled                     = 'True'
    LocalPort                   = ('1433')
    Profile                     = ('Domain', 'Private', 'Public')
    Protocol                    = 'TCP'
    Ensure                      = 'Present'
    DependsOn                   = '[cChocoPackageInstaller]sqlExpress2022'
}

Now that the groundwork has been placed, we will install the DSC Web Service:

xDscWebService dscPullServer
{
    EndpointName                = 'dscPullServer'
    Port                        = 443
    CertificateThumbPrint       = $certThumbprint
    PhysicalPath                = 'C:\Program Files\WindowsPowerShell\DscService\SecureWebPullServer\Website'
    ModulePath                  = 'C:\Program Files\WindowsPowerShell\DscService\SecureWebPullServer\Modules'
    ConfigurationPath           = 'C:\Program Files\WindowsPowerShell\DscService\SecureWebPullServer\Configuration'
    RegistrationKeyPath         = 'C:\Program Files\WindowsPowerShell\DscService'
    ConfigureFirewall           = $false
    AcceptSelfSignedCertificates = $true
    UseSecurityBestPractices    = $true
    SqlProvider                 = $true
    SqlConnectionString         = 'Provider=SQLOLEDB.1;Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=master;Data Source=.\SQLEXPRESS'
    State                       = 'Started'
    Ensure                      = 'Present'
    DependsOn                   = $('[WindowsFeature]dscService','[cChocoPackageInstaller]sqlExpress2022')
}

The final two configuration statements are to create a RegisterationKeys.txt file with the Grid provided during invoking and finally we will open port 443 on the Windows firewall.



A little supporting code and documentation and this is ready for use… 

Install-Module autoBuild

Then generate a new client registration key, a registration key is just a simply clear text string access key to restrict anyone from registering to your pull server. Just use New-GUID from PowerShell to generate a new random GUID to use as your registration key. You will also need a certificate; the certificate is used to encrypt and decrypt secrets within the configuration and to secure transport with TLS 1.2. You can use a Let’s Encrypt certificate and script it to automatically renew every 3 months or for small deployments with only internal clients, you can use a self-signed certificate, you will just need to distribute the certificate. Here is a PowerShell command to generate a self-signed certificate, export it and build a process to distribute it:

$rootDomain = 'abc.local'
$certPassword = '1234'
 
$cert = New-SelfSignedCertificate `
-Subject "*.$rootDomain" `
-DnsName "$rootDomain", "*.$rootDomain" `
-CertStoreLocation "cert:\LocalMachine\My" `
-FriendlyName "*.$rootDomain" `
-TextExtension "2.5.29.37={text}1.3.6.1.5.5.7.3.1" `
-KeyUsage DigitalSignature,KeyEncipherment,DataEncipherment `
-Provider "Microsoft RSA SChannel Cryptographic Provider" `
-HashAlgorithm "SHA256" `
-NotAfter $(Get-Date).AddYears(10)
$DestStore = New-Object -TypeName System.Security.Cryptography.X509Certificates.X509Store `
-ArgumentList 'root', 'LocalMachine'
$DestStore.Open([System.Security.Cryptography.X509Certificates.OpenFlags]::ReadWrite)
$DestStore.Add($cert)
$DestStore.Close()
 
New-Item -Name 'Boot'-Path "$env:systemdrive\inetpub" -ItemType Directory -Force
$certPW = ConvertTo-SecureString -String $certPassword -Force -AsPlainText
$cert | Export-PfxCertificate `
    -FilePath $($env:systemdrive + '\inetpub\Boot\' + ($rootDomain.Replace('.','DOT')) + '.pfx') -Password $certPW

It is important to note that this includes the private key in the export, the target nodes must have the private key to decrypt the MOF file from the pull server. Hence the reason you should use a secret vault to pull sensitive secrets at configuration execution time. This will be covered in a future article. 

Invoke the New-DSCPullServerConfiguration function supplying the certificate thumbnail and your registration key.

New-DSCPullServerConfiguration -certThumbprint '<cert thumbnail>' -regKey '<reg key>'

LCM configuration

The LCM is a powerful and flexible component for DSC, it can operate in many different modes, push and pull modes with variations of both. Push mode is the default mode and it only responds to the Windows Server Manager and any local configuration documents pushed locally. In Pull mode, the LCM is setup to check a DSC Pull server for a new or changed configuration document, in addition to the Server Manager and any pushed documents. The protocol used is very efficient as it doesn’t download a configuration document from the pull server to see if it has changed, instead it pulls the very small checksum from the pull server.

The specific mode we are operating in is called pull mode with partial configurations. In this mode, the LCM is setup to look for several configurations on the pull server, it then takes these partial configurations and concatenates these into a single configuration before its applied. The partial configurations I have included in the LCM registration function include:

• Global - globally for everyone, generic best practices for everyone
• Datacenter - configuration specific to a single cloud provider, client tools, provider optimization
• Partner – for a reseller with multiple customers, specific support tools, software installations, contact information.
• Customer - a customer level for specifics to a single customer, password history policies, security requirements that are not covered elsewhere.
• Server - a server configuration, for a line of business server configuration, SQL Server for example.
• Session Host – Desktop as a service configuration shared among all session hosts, software installation.

These levels of division allow for you to author infrastructure as code to address any need of a customer or partner without affecting existing configurations.  All of these partial configurations are overkill for most situations, I included here to demonstrate how you could leverage a partial configuration setup. Obviously, redesign the partial configurations that fit your needs the best.



You will need to provide identifiers for datacenter, partner, customer. More on this later as some of this is built into the future imaging and bootstrap process. 

Register-LCM -datacenterID 1 -partnerID 1700 -tenantID 1704 -certThumbprint '59bafe71f2a1bd8e35b0b75230e0482a6738edf5' -regKey 'e7e6b7f3-2f1e-4838-99a1-ae3d1488a830' -dscUrl 'https://dscPull.abc.local/PSDSCPullServer.svc'

Review the Register-LCM function in the autoBuild module, many options are hard coded and can be tweaked to fit your specific needs, for example:

[DscLocalConfigurationManager()]
Configuration PullPartialConfig {   
   Settings {
        RefreshFrequencyMins           = 30;
        RefreshMode                    = "PULL";
        ConfigurationMode              ="ApplyAndAutocorrect";
        AllowModuleOverwrite           = $true;
        RebootNodeIfNeeded             = $false;
        ConfigurationModeFrequencyMins = 15;
        CertificateID                  = $certThumbprint;
        DebugMode                      = 'ForceModuleImport';
        }
       
    ConfigurationRepositoryWeb DSCService {
        AllowUnsecureConnection        = $true;
        CertificateID                  = $certThumbprint;
        RegistrationKey                = $regKey;
        ServerURL                      = $dscUrl;           
       ConfigurationNames             = @("Global","Datacenter_$($datacenterID)","Partner_$($partnerID)","Customer_$($tenantID)","Machine_$($env:COMPUTERNAME)","SessionHost_$($tenantID)");
        }

    PartialConfiguration "Global" {
        Description                    = "Global";
        ConfigurationSource            = @("[ConfigurationRepositoryWeb]DSCService");
        }

    PartialConfiguration "Datacenter_$($datacenterID)" {
        Description                    = "Datacenter - $($datacenterID)";
        ConfigurationSource            = @("[ConfigurationRepositoryWeb]DSCService");
        }

    PartialConfiguration "Partner_$($partnerID)" {
        Description                    = "Partner - $($partnerID)";
        ConfigurationSource            = @("[ConfigurationRepositoryWeb]DSCService");
        }
               
    PartialConfiguration "Customer_$($tenantID)" {
        Description                    = "Customer - $($tenantID)";
        ConfigurationSource            = @("[ConfigurationRepositoryWeb]DSCService");
        }

    PartialConfiguration "Machine_$($env:COMPUTERNAME)" {
        Description                    = "Machine - $($env:COMPUTERNAME)";
        ConfigurationSource            = @("[ConfigurationRepositoryWeb]DSCService");
        }

    PartialConfiguration "SessionHost_$($tenantD)" {
        Description                    = "SessionHost - $($tenantD)";
        ConfigurationSource            = @("[ConfigurationRepositoryWeb]DSCService");
        }               
   }

For more information on configuring the LCM, please refer to the MS document:

https://learn.microsoft.com/en-us/powershell/dsc/managing-nodes/metaconfig?view=dsc-1.1

DSC Resources

DSC resources allow for the extension of DSC to support more products and client types. This is how the LCM interprets the configuration and instructions on how to apply the configuration. DSC resources are just PowerShell modules with specialized metadata and code, you can even have a PowerShell module with functions and DSC resources. DSC resources are vast and numerous (near 2500 resources) and I’ll only cover how to research, install, and utilize these DSC resources. There are well over 300 modules that contain resources in the public PowerShell Gallery:

Find-DscResource -Repository PSGallery | Select-Object -Property ModuleName, Version -Unique

The best way to discover a DSC resource is to simply search the web for the DSC Resource, for example if I need a way to configure a Citrix XenApp server deployment, I’d search for DSC resource XenApp. DSC resources exist for all MS products and most third-party products. Install the DSC Resource just like a PS module then you can start authoring configurations locally. To make the DSC Resource available to the DSC pull server, it must be added to the pull server. 

The DSC Resource module must be packaged and copied with its checksum to the Pull server resource path, C:\Program Files\WindowsPowerShell\DscService\Modules.

How to package and produce the checksum is located here: https://learn.microsoft.com/en-us/powershell/dsc/pull-server/pullserver?view=dsc-1.1#dsc-resource-module-package-format

Authoring a configuration

The process of producing the configuration is relatively easy, once you know the name of the resource module, you can get a list of the resources in that resource module, Get-DSCResource -module cChoco.

To get the resource sample, use Get-DSCResource <ResourceName> -syntax will return a sample for your configuration documents.

The PSDesiredStateConfiguration is already installed and available; this is the base resource from MS that contains many of the basic resources needed. These are:

• File
• Archive
• Environment
• Group
• GroupSet
• Log
• Package
• ProcessSet
• Registry
• Script
• Service
• ServiceSet
• User
• WaitForAll
• WaitForAny
• WaitForSome
• WindowsFeature
• WindowsFeatureSet
• WindowsOptionalFeature
• WindowsOptionalFeatureSet
• WindowsPackageCab
• WindowsProcess

The syntax shows you the required fields for the resource and fields in square brackets are optional. I got in the habit of always including Ensure = Present even though Ensure = Absent is obviously the opposite of what we are wanting to achieve. The DependsOn is usually needed to state dependencies. The best way to learn this syntax is to simply review others, it’s straightforward and easy to learn.

Compiling and posting the configuration

I use a different process for generating and posting configurations and it has some merits that should be covered here. Never have I produced a configuration and posted the configuration once for all time. You tweak the configuration and post again, you might use the configuration for another customer or server, it is inevitable that you will have to repost a configuration again, therefore change control is necessary. You could use git but I’m impartial to using the PowerShell module repository built in the last article I posted. I place the configurations in the module and provide version control along with the module itself.   Co-workers simply invoke the configuration to produce a new configuration or update the module and push it back to the repo for configuration changes.



This configuration will install or uninstall named chocolatey software packages on the targeted machine.

Configuration "Software_$machine"

{

   param (

       [string]$configName,

       [string[]]$installList,

       [string[]]$uninstallList

       )

   Import-DscResource -Module cChoco

   Node $configName

   {

       cChocoInstaller installChoco

       {

       InstallDir = "C:\ProgramData\chocolatey"

       }

       If ($installList[0] -ne 'none') {

           cChocoPackageInstallerSet installSet

           {

           Name       = $installList

           Ensure     = 'Present'

           DependsOn  = "[cChocoInstaller]installChoco"

           }

       }

       If ($uninstallList[0] -ne 'none') {

           cChocoPackageInstallerSet uninstallSet

           {

           Name       = $uninstallList

           Ensure     = 'Absent'

           DependsOn  = "[cChocoInstaller]installChoco"

           }

       }

   }

} 

This is an example of a configuration that is called for multiple customers as it accepts a list of applications to install or remove. The $configurationName is the same as the last configuration in the LCM registration so when this is posted, the client machine finds it on the pull server and performs the tasks necessary.



The following code then invokes the configurations and produces a checksum then posts the resulting MOF and check sum to the pull server.

$path = "E:\configuration\$customer\$machine" 
&"Software_$machine" -configName "Software_$machine" -installList $installList -uninstallList $uninstallList -OutputPath $path
New-DscChecksum -OutPath $OutputFolderPath -Path $FullName -Force
copy $FullName $OutputFolderPath -Verbose -Force

I removed some supporting commands, but this is the basic drift of the process. I’d include all of this in a function and then automation will invoke the configuration to install the software, automatically. 

PowerShell DSC commands to know.

• Get-Command -Module PSDesiredStateConfiguration
• Get-DscLocalConfigurationManager – review the configuration of the local configuration manager.
• Remove-DscConfigurationDocument -Stage Current,Pending,Previous -force – Removes the configuration on the local machine, if the pull server is still configured, it will be downloaded again.
• Set-DscLocalConfigurationManager - sets the local configuration manager
• Start-DscConfiguration -UseExisting -Verbose -Wait -Force – starts the current configuration interactively, allows for you to watch it – can aid in troubleshooting.

Conclusion

Microsoft Desired State Configurations goes underutilized and with a full understating of the product, the options to provide a better and more responsive environment of your customers is easily achievable. This is just the beginning and how you implement your system is completely up to you and your team. You can wire the automation contained in the customized module to any front end, even Active Directory.  You could deploy software to a RDS host by utilizing the notes field in Active Directory Groups, specifying the name of the Chocolatey package to deploy when a Application Security group is created, now all you would need to do is crate a trigger to evaluate the customer and generate the MOF files. This could be done in many ways.

I’m working increasingly with Linux and more specifically managing Windows environments using Linux. I’m currently working on an internal project for a Private Cloud Deployment system with a hierarchical client structure utilizing opensource Linux components, one of these components to orchestrate Linux environments (pfSense, TureNAS, XCP-NG, Ubuntu, etc.) is Ansible. Ansible can also be used to manage windows and it uses DSC through a push mechanism without relying on a pull server at all or better yet, Ansible becomes the pull server for Windows clients but in a push mode.   Expect future articles on Ansible and how to utilize it to manage Windows.

Please consider using Meade Automation if you have any need for automation in an ISV or MSP environment. We specialize in both public and private cloud, this system can also manage traditional enterprise deployments equally well, just need internet connectivity and local administrative permissions.

To schedule an appointment to discuss you requirements and how we can help, please send an e-mail to info@meadeautomation.com or call (208) 629-2449.