cDFS is dead, long live xDFS

The xDFS DSC resource module has been officially released to the PowerShell Gallery thanks to the awesome review efforts of the Microsoft PowerShell Team. The cDFS DSC Resource has now been unlisted from the PowerShell Gallery. So now is the time to update any DSC configuration scripts to use xDFS.

ss_xdfs_releasepsgallery

Important: There were some minor changes to xDFS when it was converted from cDFS. For information on what you’ll need to change to convert to xDFS see my earlier post.

 

cDFS moving to the PowerShell Team

Just a Friday afternoon heads up – if you’re using the cDFS DSC Resource I created to manage Windows Server Distributed File System (Replication and Namespaces), it has now been accepted into the PowerShell Community resources and will be under the control of the PowerShell Team.

This means that the GitHub source code repository will be moving over to the PowerShell organization in the next few days. This also means that any future releases of this resource module won’t be provided by me as cDFS, but will be released by the PowerShell team as xDFS.

So I recommend that when this happens you switch over to using the xDFS resource. I will put another post up here when the change over officially occurs. The first official release version under the new xDFS name will be 3.0.0.x. I won’t make any further changes or bug fixes to the cDFS resources.

It is also worth noting that as part of this move some minor changes were made to the DSC Resource modules. These are breaking changes and you will most likely need to update any DSC Configurations depending on this, but you would have to do this anyway because of the name change.

The changes are:

  • Resource xDFSRepGroup renamed to xDFSReplicationGroup
  • Resource xDFSRepGroupConnection renamed to xDFSReplicationGroupConnection
  • Resource xDFSRepGroupFolder renamed to xDFSReplicationGroupFolder
  • Resource xDFSRepGroupMembership renamed to xDFSReplicationGroupMembership
  • xDFSReplicationGroupConnection:
    • Changed DisableConnection parameter to EnsureEnabled.
    • Changed DisableRDC parameter to EnsureRDCEnabled.

These changes should only require minor changes to your configuration scripts to implement.

Thanks for reading and have a great Friday~

 

 

Creating Professional DSC Resources – Part 4

The purpose of this series of articles is to try and document a few of the lessons I learned while releasing new DSC resources as well as contributing to the existing Microsoft Community DSC resources. These articles are not intended to tell you how to write DSC resources from a programming perspective, but to give you some ideas on what might be expected of a DSC resource you’re releasing to the public. For example, unit and integration tests (don’t worry if you aren’t familiar with those terms).

These articles are also not intended to tell you what you must do to release your resource, but more document what will help your resource be easier to use and extend by other people. Some of these these things are obvious for people who have come from the development community, but may be quite new to operations people.

If you missed any previous articles you can find them here:

 

Automated Testing

Automated testing is something that is familiar to most developers, but for operations people it is usually a new concept. However, it is one of the most important things you can add to your DSC Resources – and most DSC resource projects won’t even accept your code contributions if they don’t contain automated tests.

So, what are automated tests? Well, they are just PowerShell scripts that you create and run that will check your DSC Resource is working correctly. Usually automated tests are run on your DSC Resources every time you commit your code – and they’ll tell you if anything has gone wrong. I could spend the next day listing reasons why automated testing is extremely important, but that is not the purpose for this post.

PowerShell contains a great automated test framework called Pester that allows you to describe your tests using special PowerShell functions.

Important: If you aren’t familiar with Pester and automated testing, you should get familiar with it before reading any further. This series is a fantastic place to start. Even if you’re familiar with Pester it is a good read.

An example of a Pester test on a DSC Resource:

The above test is a unit test of the xFirewall resource in the xNetworking module. the Don’t worry if you don’t completely understand this yet, that is the purpose of this article – although you should understand the basic structure of the Pester test – if you don’t, you’ll definitely want to go and review this series.

 

Types of Automated Tests

There are two types of automated tests you should create for your DSC Resources:

  • Unit Tests – these test that each function in your DSC Resource works correctly in isolation. This means that if you call the function with a set of parameters you get expected output.
  • Integration Tests – these tests ensure that your DSC Resource works in a real environment – e.g. work correctly when they are actually integrated into a DSC Configuration file.

For every DSC Resource in your DSC Resource Module you should ensure that there is one unit test file and one integration test file (although usually for integration tests a support file is also needed, but we’ll cover this later).

 

Test Folders

You should place all tests inside a Tests folder in the root of your DSC Module folder:

ss_dsc_testfolders

Unit tests should be placed in a Unit folder within Tests and … I’m sure you get where I’m going here.

 

Unit Tests

Unit tests are a good place to start when creating automated tests for your DSC Resources because they’re u sually quite straight forward. Each DSC Resource in your DSC Module should contain it’s own unit test file. For example, here are the unit tests that are included with the xNetworking DSC Resource module:

ss_dsc_xnetworkingunittests

A note about code coverage: A unit test script will contain lots of individual tests. The purpose of each individual unit test is usually to test a single path through a function that the computer might take when running that function. The goal therefore is to add as many tests as needed to make sure each code path is covered. This is called code coverage. The goal is to have 100% code coverage for your DSC Resource.

In these unit tests I am using a DSC Resource for creating iSCSI Virtual Disks to illustrate the process. You don’t need to know anything about iSCSI Virtual Disks to understand these articles or resources, but if you’re interested to know the cmdlets I’m using for these, see this page. I’m using the *_iSCSIVirtualDisk cmdlets in this DSC Resource.

 

Creating a New Unit Test

You could just go ahead and create a new unit from scratch, but there is a much easier way. There is a unit test template file that you can use to create new unit tests. You need to copy the unit test template file and rename it to match your DSC Resource.

The easiest way to do this is to clone the repository containing the test template files and copy the unit_template.ps1 file to your Tests/Unit folder:

ss_dsc_createnewunittestfromtemplate

You’ll now have a new unit test file that you can open in your PowerShell editor of choice:

ss_dsc_editingnewunittest

Next, customize the TODO area in the header with the your DSC Resource Name and DSC Module Name:

ss_dsc_customizeunittestheader

Feel free to remove the TODO comments if you want (I always do).

Now, onto the job of actually adding our Pester tests. You’ll want to locate the Pester Tests region:

ss_dsc_unittestspestertestsregion

This is usually the only area of code in this file you need edit. There are several areas you need to customize here:

  1. Pester Test Initialization – this is where you would define any variables or objects that you might use for testing.
  2. Function Get-TargetResource – this would contain the Pester Tests to test the Get-TargetResource function.
  3. Function Test-TargetResource – this would contain the Pester Tests to test the Test-TargetResource function.
  4. Function Set-TargetResource – this would contain the Pester Tests to test the Set-TargetResource function.
  5. Pester Tests for any Helper Cmdlets – you would add Describe blocks for any additional functions you have defined in your resource to ensure those functions are tested.

Once there sections are completed for a DSC Resource the tests can be invoked using the Invoke-Pester command.

But first I’ll provide more detail on what you should add into each section.

 

Pester Test Initialization

The purpose of this section is to initialize any variables or objects that you’re going to use for testing.

For example, if you were testing a resource that was for creating an iSCSI Virtual Disk you might define the parameters of the iSCSI Virtual Disk that you’re going to use for testing:

The $TestVirtualDisk object is used to @splat onto the Set-TargetResource and Test-TargetResource functions and for comparison. It saves us lots of typing and therefore potential mistakes.

The $MockVirtualDisk object is used as output to the Mocks that get defined on the Get-iSCSIVirtualDisk cmdlet when we want to simulate the scenario where the iSCSI Virtual Disk already exists.

You don’t of course have to do this, but you’ll find that having some objects defined up front that you can use as test objects will make your code smaller and easier to understand as well as making the tests run faster.

 

Function Get-TargetResource

This area will contain the actual Pester tests that test the Get-TargetResource function. Usually there are only a small number of tests you can perform in this function:

  • Does it work when the resource being configured does not exist?
  • Does it work and return the expected parameters when the resource being configured does exist?

For example:

In the above code we have two Context blocks – one for each of our scenarios above.

Context ‘Virtual Disk does not exist’

In this scenario we Mock the Get-iSCSIVirtualDisk cmdlet to return nothing. This is the behavior we’d expect if the resource being configured didn’t exist.

We can now check the object returned is what is expected and that the Get-iSCSIVirtualDisk was called the expected number of times – in this case once.

Context ‘Virtual Disk does exist’

In this scenario we Mock the Get-iSCSIVirtualDisk cmdlet to return the object we defined in the Pester Test Initialization section. This is the behavior we’d expect if the resource being configured does exist.

We can therefore check the object returned is what is expected and all the returned object properties match and that the Get-iSCSIVirtualDisk was called the expected number of times – again this is once.

 

Unit Tests to be Continued…

I wanted to put all the unit test info in a single part of this series, but as you can see it is getting quite long. I’ll complete the rest of the information on unit tests tomorrow.

So in the next article, I’ll cover the unit tests for Set-TargetResource and Get-TargetResource as well as unit testing any additional functions. Thanks again for reading and I hope it is useful.

Further parts in this series:

 

Distributed File System DSC Resource Update

After releasing the DFS DSC Resource Module yesterday, I had an idea of how to simplify it if you’re deploying a DFS folder that contains the same path content path for all members. I added a ContentPaths parameter (an array of strings) to the cDFSRepGroup resource so that if the folder exists in the same location on every member, you won’t need to use the cDFSRepGroupMembership resource to individually set the Content Path for each member.

For example:

configuration Sample_cDFSRepGroup_Simple
{
    Import-DscResource -Module cDFS

    Node $NodeName
    {
        [PSCredential]$Credential = New-Object System.Management.Automation.PSCredential ("CONTOSO.COM\Administrator", (ConvertTo-SecureString $"MyP@ssw0rd!1" -AsPlainText -Force))

        # Install the Prerequisite features first
        # Requires Windows Server 2012 R2 Full install
        WindowsFeature RSATDFSMgmtConInstall 
        { 
            Ensure = "Present" 
            Name = "RSAT-DFS-Mgmt-Con" 
        }

        # Configure the Replication Group
        cDFSRepGroup RGPublic
        {
            GroupName = 'Public'
            Description = 'Public files for use by all departments'
            Ensure = 'Present'
            Members = 'FileServer1','FileServer2'
            Folders = 'Software','Misc'
            Topology = 'Fullmesh'
            ContentPaths = 'd:\public\software','d:\public\misc'
            PSDSCRunAsCredential = $Credential
            DependsOn = "[WindowsFeature]RSATDFSMgmtConInstall"
        } # End of RGPublic Resource
    } # End of Node
} # End of Configuration

The above example creates a DFS Replication Group called Public containing two folders, Software and Misc. The DFS Replication Group replicates to two members, FileServer1 and FileServer2. It is maintaining a Fullmesh connection topology.

The thing to note is that the ContentPaths array should have the elements in a matching order to the Folders parameter. So this:

            Folders = 'Misc','Software'
            ContentPaths = 'd:\public\software','d:\public\misc'

Would result in the Misc folder being set with the Content Path d:\public\software’ and the Public folder being set with the Content Path  d:\public\misc‘ – which is probably not ideal.

The Primary Member

Every Resource Group Folder needs a Primary Member set for initial replication to take place. If you use this automatic assigning of content paths the Primary Member will automatically be set to the computer listed first in the Members parameter. If you want to change this you’ll need to use the manual cDFSRepGroupMembership resource instead.

Partially Setting Content Paths

It is actually possible to only automatically configure some of the content paths in a DFS Replication Group by leaving the appropriate ContentPaths array entry blank. This would allow you to automatically configure some folders but leave other folders to be manually configured.

For example:

        cDFSRepGroup RGPublic
        {
            GroupName = 'Public'
            Description = 'Public files for use by all departments'
            Ensure = 'Present'
            Members = 'FileServer1','FileServer2'
            Folders = 'Software','Misc','Video'
            Topology = 'Fullmesh'
            ContentPaths = 'd:\public\software','','e:\video'
            PSDSCRunAsCredential = $Credential
            DependsOn = "[WindowsFeature]RSATDFSMgmtConInstall"
        } # End of RGPublic Resource

This would create a Replication Group called Public, with three folders Software, Misc and Video. The Software and Video folders will be automatically configured with Content Paths but the Misc folder will be left unconfigured so that it can be configured manually.

Optional Use

Using the ContentPaths or Topology parameters is optional. You can still define the folder Content Paths manually using the cDFSRepGroupMembership resource and/or configure the connection topology manually using the cDFSRepGroupConnection resource if you want to.

Important: It is not recommended that you define a ContentPath for a folder in the cDFSRepGroup ContentPaths parameter if you are also setting it in a cDFSRepGroupMembership resource. The same applies to defining and automatic Topology and using the cDFSRepGroupConnection resource.

And again, in case you missed it, the post covering the original resource is here.

Windows Distributed File System DSC Resource

Introduction

While studying for my MS 70.411 exam, I found that one way of getting a good understanding of a feature is to perform as many feature tasks as possible using PowerShell. One especially useful way of doing this for me was to implement a DSC resource for the feature. So, this week the feature was Distributed File System Replication Groups. I’ll refer to Distributed File Systems as DFS in future to save typing.

Note: I am going to implement DFS Namespaces as well, but that will be left to next week.

Update: After releasing this version I had an idea for some improvements to simplify this resource. See the details here.

Node vs. Active Directory

The first thing to note with implementing a DSC resource for Windows Distributed File System is that the resource is actually setting the Desired State of Active Directory elements rather than that of a Node. What that means is that when you use the PowerShell (or Management Console) to manage Windows DFS Replication Groups or DFS Namespaces you’re actually configuring items in the Active Directory database – you’re not changing anything on the actual node/computer you’re running the commands on.

This means that a DSC Resource for configuring Windows DFS could be run on any computer within the AD Domain. This is actually very handy as it turns out. At first though, I wasn’t sure DSC should be used for configuring elements that aren’t actually on a Node/Computer, but I couldn’t see why not, and then I remembered that there are other resources that do this (xActiveDirectory for example).

Server Core Not Supported

The first problem I ran into when implementing this DSC Resource is that you can’t install the DFS Replication (DFSR) PowerShell module onto a Windows Server Core installation. This is because the PowerShell DFSR module is only installed with the DFS Management Tools feature, which requires a Full Server install (or at least the Graphical Management Tools and Infrastructure feature).

This feature is required to enable the DFSR PowerShell Module.

This feature is required to enable the DFSR PowerShell Module.

This isn’t the end of the world, but it is annoying because all my file servers are Server Core. Therefore I’d need to run this resource on a node with a Full Server install that is also part of the AD Domain. So it is great that this resource can be run on any Full Server install (or even a Desktop with RSAT).

Setting AD Credentials

Because this resource calls PowerShell CmdLets that interact with the AD Database, AD credentials need to be supplied that can have the appropriate permissions. This means that the PSDSCRunAsCredential property must be set for each resource entry, which in turn means this Resource can only be used on nodes with Windows Management Framework 5.0 (WMF 5.0) or greater installed. If you’re not familiar with this property, see this link.

Installing the Resource

Because this resource requires WMF 5.0 you can just download this directly from the PowerShell Gallery by running this command:

Install-Module -Name cDFS

Using the Resource

The following example creates a DFS Replication Group called Public containing two members, FileServer1 and FileServer2. The Replication Group contains a single folder called Software. A description will be set on the Software folder and it will be set to exclude the directory Temp from replication.

configuration Sample_cDFSRepGroup
{
    Import-DscResource -Module cDFS

    Node $NodeName
    {
        [PSCredential]$Credential = New-Object System.Management.Automation.PSCredential ("CONTOSO.COM\Administrator", (ConvertTo-SecureString $"MyP@ssw0rd!1" -AsPlainText -Force))

        # Install the Prerequisite features first
        # Requires Windows Server 2012 R2 Full install
        WindowsFeature RSATDFSMgmtConInstall 
        { 
            Ensure = "Present" 
            Name = "RSAT-DFS-Mgmt-Con" 
        }

        # Configure the Replication Group
        cDFSRepGroup RGPublic
        {
            GroupName = 'Public'
            Description = 'Public files for use by all departments'
            Ensure = 'Present'
            Members = 'FileServer1','FileServer2'
            Folders = 'Software'
            PSDSCRunAsCredential = $Credential
            DependsOn = "[WindowsFeature]RSATDFSMgmtConInstall"
        } # End of RGPublic Resource

        cDFSRepGroupConnection RGPublicC1
        {
            GroupName = 'Public'
            Ensure = 'Present'
            SourceComputerName = 'FileServer1'
            DestinationComputerName = 'FileServer2'
            PSDSCRunAsCredential = $Credential
        } # End of cDFSRepGroupConnection Resource

        cDFSRepGroupConnection RGPublicC2
        {
            GroupName = 'Public'
            Ensure = 'Present'
            SourceComputerName = 'FileServer2'
            DestinationComputerName = 'FileServer1'
            PSDSCRunAsCredential = $Credential
        } # End of cDFSRepGroupConnection Resource

        cDFSRepGroupFolder RGSoftwareFolder
        {
            GroupName = 'Public'
            FolderName = 'Software'
            Description = 'DFS Share for storing software installers'
            DirectoryNameToExclude = 'Temp'
            PSDSCRunAsCredential = $Credential
            DependsOn = '[cDFSRepGroup]RGPublic'
        } # End of RGSoftwareFolder Resource

        cDFSRepGroupMembership RGPublicSoftwareFS1
        {
            GroupName = 'Public'
            FolderName = 'Software'
            ComputerName = 'FileServer1'
            ContentPath = 'd:\Public\Software'
            PrimaryMember = $true
            PSDSCRunAsCredential = $Credential
            DependsOn = '[cDFSRepGroupFolder]RGSoftwareFolder'
        } # End of RGPublicSoftwareFS1 Resource

        cDFSRepGroupMembership RGPublicSoftwareFS2
        {
            GroupName = 'Public'
            FolderName = 'Software'
            ComputerName = 'FileServer2'
            ContentPath = 'e:\Data\Public\Software'
            PSDSCRunAsCredential = $Credential
            DependsOn = '[cDFSRepGroupFolder]RGPublicSoftwareFS1'
        } # End of RGPublicSoftwareFS2 Resource

    } # End of Node
} # End of Configuration

Example Breakdown

The resource usage hopefully is fairly straight forward and the Module itself contains documentation in the Readme.md (you can also see it here). But I’ll provide a quick breakdown of the resources just in case.

WindowsFeature RSATDFSMgmtConInstall

Install the Windows Feature that is required to use this DSC Resource. It installs the Windows DFSR/DFSN PowerShell Modules.

cDFSRepGroup

This resource creates, configures or removes a DFS Replication Group. You should specify both the Members and the Folders that are in this Replication Group. Both of these properties take an array of strings so you can specify more than one member (not much of a Distributed File System without that right?) and more than one folder. You of course also need to specify a DFS Replication Group Name.

This resource also contains an optional Topology parameter that defaults to Manual. If this parameter is set to Fullmesh then a Full Mesh connection topology will be configured automatically for this Replication Group, based on the members specified in the resource.

cDFSRepGroupConnection

This is an optional resource that allows the Replication Group Connections to be defined manually. I used the above example, so that it was obvious how they should be used. It allows a Replication Group Connection to be defined for a Replication Group between two members. A description can also be set on each connection. The connections can be disabled and also have RDC (Remote Differential Compression) disabled.

Note: this resource should only be used if the Topology parameter of the cDFSRepGroup resource is set to Manual (which is the default). If you set the Topology parameter to Fullmesh, a set of Replication Group Connections will automatically be created in a Full Mesh structure. The Hub and Spoke structure is not currently supported but may be in the future.

cDFSRepGroupFolder

This is an optional resource that can be used to configure specific properties of any of the folders in a DFS Replication Group. It is not used to create a folder within the Replication Group, that is the job of the cDFSRepGroup resource. This job of this resource is to configure the following properties of a Replication Group Folder:

  • Description
  • FilenameToExclude – if this is not specified the default value that DFS assigns is automatically used.
  • DirectoryNameToExclude – if this is not specified the default value that DFS assigns is automatically used.

cDFSRepGroupMembership

This resource is used to configure the actual content folders on each member of the Replication Group Folder. An instance of this resource should be used for each combination of member and folder in a Replication Group to set the Content Folder. It can also be used to set the following optional properties:

  • StagingPath – this can be used to override the default staging path. Usually this should be left to the default.
  • ReadOnly – this property can be used to make this content folder read only.
  • PrimaryMember – this property allows a Primary Member of the replication group to be set. At least one member of each Replication Group folder must set as the Primary Member otherwise initial replication will never take place.

Common Parameters

There are a couple of parameters that are common to each resource:

  •  GroupName – this is the name of the Replication Group.
  • Domain – this is the name of the AD Domain this Replication Group is part of. If not specified then the AD Domain that the computer that is running the config is part of is used. Usually it should not be specified.

Summary

Well, there is not much more to say about this. Hopefully someone finds it useful. I intend to add DFS Namespace support over the next week or so, so if you’re needing that, keep an eye out.

Feedback

If you’re interested in contributing to this resource, providing feedback or raising issues or requesting features, please feel free (anything is appreciated). You’ll find the resource GitHub repository here where you can fork, issue pull requests and raise issues/feature requests.