Stop, Start or Restart all Web Apps in Azure using PowerShell

Here is a short (and sometimes handy) single line of PowerShell code that can be used to restart all the Azure Web Apps in a subscription:

Note: Use this with care if you’re working with production systems because this _will_ restart these Web Apps without confirming first.

This would be a handy snippet to be able to run in the Azure Cloud Shell. It could also be adjusted to perform different actions on other types of resources.

To stop all Web Apps in a subscription use:

To start them all again:

The key part of this command is the GetEnumerator() method because most Azure Cmdlets don’t return an array of individual objects into the pipeline like typical PowerShell cmdlets. Instead returning a System.Collections.Generic.List object, which requires a slight adjustment to the code. This procedure can be used for most Azure Cmdlets to allow the results to be iterated through.

Thanks for reading.

Advertisements

Install Nightly Build of Azure CLI 2.0 on Windows

The Azure PowerShell cmdlets are really first class if you’re wanting to manage Azure with PowerShell. However, they don’t always support the very latest Azure components and features. For example, at the time of writing this there is no Azure PowerShell module for managing Azure Container Instances.

The solution to this is to install the Nightly Build of Azure CLI 2.0. However, on Windows it is not entirely clear the easiest way to do this. So, in this post I’ll provide a PowerShell script that will:

1. Install Python 3.x using Chocolatey
2. Use PIP (Python package manager) to install the latest nightly build packages
3. Update the Environment Path variable so that you can use Azure CLI 2.0.

Note: If you have the stable build of Azure CLI 2.0 installed using the MSI then you’ll need to configure your Environment Path variable to find the Az command that you’d like to use by default. I personally removed the stable build of Azure CLI 2.0 to make it easier.

Performing the Install

Make sure you’ve got Chocolatey installed. If you aren’t sure what Chocolatey is, it is a package management system for Windows – not unlike Apt-Get or Yum for Linux. It is free and awesome. In this process we’ll use Chocolatey to install Python for us. If you haven’t got Chocolatey installed, see this page for instructions.

Next, download and run this PowerShell script in a PowerShell Administrator Console:

You could save the content of this script into a PS1 file and then execute it like this:

It will then download and install Python, then use PIP to install the current nightly build packages. After a few minutes the installation will complete:

You can then run:

Az Login

To get started.

If you’re a bit new to Azure CLI 2.0, then another great way is to use Azure CLI Interactive:

Az Interactive

If you need to update to a newer nightly build, just run the script again and it will update your packages.

Easy as that! Now you can experiment with all the latest automation features in Azure without needing to wait for a new version of Azure CLI 2.0 or for latest Azure PowerShell cmdlets.

Edge Builds

If you want to install even more “bleeding edge” builds (built straight off the master branch on every merge to master) then you can make a small adjustment to the script above:

On line 34 change the URL of the feed from:

https://azureclinightly.blob.core.windows.net/packages

To:

https://azurecliprod.blob.core.windows.net/edge

Thanks for reading!

 

 

Sonatype Nexus Containers with Persistent Storage in Azure Container Instances

On the back of yesterdays post on running Azure Container Instance containers with persistent storage, I thought I’d try a couple of other containers with my script.

Note: I don’t actually plan on running any of these apps, I just wanted to test out the process and my scripts to identify any problems.

I tried:

• Sonatype Nexus 2 – sonatype/nexus:oss
• Sonatype Nexus 3 – sonatype/nexus3:latest
• Jenkins – jenkins/jenkins

And here are the results of my tests:

Sonatype Nexus 2

Works perfectly and the container starts up quickly (under 10 seconds):

I passed the following parameters to the script:

Note: The Nexus 2 server is only accessible on the path /nexus/.

Sonatype Nexus 3

Works perfectly but after takes at least a minute to be accessible after the container starts. But this is normal behavior for Nexus 3.

I passed the following parameters to the script:

Jenkins

Unfortunately Jenkins does not work with a persistent storage volume from an Azure Share. It seems to be trying to set the timestamp of the file that will contain the InitialAdminPassword, which is failing:

I passed the following parameters to the script:

So, this is still a little bit hit and miss, but in general Azure Container Instances look like a very promising way to run different types of services in containers without a lot of overhead. With a bit of automation, this could turn out to be a cost effective way to quickly and easily run some common services.

Persistent Storage in Azure Container Instances

Update 2017-08-06: I have improved the script so that it is idempotent (can be run more than once and will only create anything that is missing). The Azure Container Instance resource group can be deleted once you’ve finished with the container and then recreated again with this same script when you next need it. The storage will be preserved in the separate storage account resource group. The script can now be run with the -verbose parameter and will produce much better progress information.

Azure Container Instances (ACI) is a new resource type in Azure that allows you to quickly and easily create containers without the complexity or overhead of Azure Service Fabric, Azure Container Services or provisioning a Windows Server 2016 VM.

It allows you to quickly create containers that are billed by the second from container images stored in Docker Hub or your own Azure Container Registry (ACR). Even though this feature is still in preview, it is very easy to get up and running with it.

But this post isn’t about creating basic container instances, it is about running container instances where some of the storage must persist. This is a basic function of a container host, but if you don’t have access to the host storage then things get more difficult. That said, Azure Container Instances do support mounting Azure File Shares into the container as volumes. It is fairly easy to do, but requires quite a number of steps.

There is some provided documentation for persisting storage in a container instance, but it is quite a manual process and the example ARM templates are currently broken: there are some typos and missing properties. So this post aims to make the whole thing a lot simpler and automatable.

So in this post, I’m going to share a PowerShell function and Azure Resource Manager (ARM) template that will allow you to easily provision an Azure Container Instance with an Azure File Share mounted. The process defaults to installing a GoCD Server container (version 17.8.0 if you’re interested), but you could use it to install any other Linux Container that needs persistent storage. The script is parameterized so other containers and mount points can be specified – e.g. it should be fairly easy to use this for other servers like Sonatype Nexus or Jenkins Server.

Update 2017-08-06: I documented my findings trying out these other servers in my following blog post.

Requirements

To perform this process you will need the following:

• PowerShell 5.0+ (PowerShell 4.0 may work, but I haven’t tested it).
• The Azure PowerShell module installed.
• Created an Application Service Principal – see below.

Azure Service Principal

Before you start this process you will need to have created an Application Service Principal in Azure that will be used to perform the deployment. Follow the instructions on this page to create an application and then get the Service Principal from it.

You will need to record these values as they will be provided to the script later on:

• Application Id
• Application Key
• Tenant Id
• Subscription Name

The Process

The process will perform the following tasks:

1. The Service Principal is used to login to Azure to perform the deployment.
2. An Azure Resource Group is created to contain a Azure Storage Account and Azure Key Vault.
3. An Azure Storage Account is created and an Azure File Share is created in it.
4. An Azure Key Vault is created to store the Storage Account Key and make it accessible to the Azure Container Instance.
5. The Service Principal is granted permission to the Azure Key Vault to read and write secrets.
6. The key to the Storage Account Key is added as a secret to the Azure Key Vault.
7. The parameters are set in an ARM Template parameter file.
8. An Azure Resource Group is created to contain the Azure Container Instance.

The Script

This is the content of the script:

The script requires a four parameters to be provided:

• ServicePrincipalUsername – the Application Id obtained when creating the Service Principal.
• ServicePrincipalPassword – the Application Key we got (or set) when creating the Service Principal.
• TenancyId – The Tenancy Id we got during the Service Principal creation process.
• SubscriptionName – the name of the subscription to install the ACI and other resources into.

There are also some other optional parameters that can be provided that allow the container image that is used, the TCP port the container listens on and mount point for the Auzre File Share. If you don’t provide these parameters will be used which will create a GoCD Server.

 

• AppCode – A short code to identify this application. It gets added to the resource names and resource group names. Defaults to ‘gocd’.
• UniqueCode – this string is just used to ensure that globally unique names for the resources can be created. Defaults to ‘zzz‘.
• ContainerImage – this is the name and version of the container image to be deployed to the ACI. Defaults to ‘gocd/gocd-server:v17.8.0‘.
• CPU – The number of cores to assign to the container instance. Defaults to 1.
• MemoryInGB – The amount of memory (in GB) to assign to the container instance. Defaults to 1.5.
• ContainerPort – The port that the container listens on. Go CD Server defaults to 8153.
• VolumeName – this is a volume name that is used to represent the volume in the ARM template. It can really be set to anything. Defaults to ‘gocd‘.
• MountPoint – this is the folder in the Container that the Azure File Share is mounted to. Defaults to ‘/godata/‘.

ARM Template Files

There are two other files that are required for this process:

1. ARM template – the ARM template file that will be used to install the ACI.
2. ARM template parameters – this file will be used to pass in the settings to the ARM Template.

ARM Template

This file is called aci-azuredeploy.json and should be downloaded to the same folder as the script above.

ARM Template Parameters

This file is called aci-azuredeploy.parameters.json and should be downloaded to the same folder as the script above.

Steps

To use the script the following steps need to be followed:

1. Download the three files above (the script and the two ARM template files) and put them into the same folder:
2. Open a PowerShell window.
3. Change directory to the folder you place the files into by executing:
4. CD <folder location>
5. Execute the script like this (passing in the variables):
6. The process will then begin and make take a few minutes to complete:Note: I’ve changed the keys to this service principal and deleted this storage account, so I using these Service Principal or Storage Account keys won’t work!
7. Once completed you will be able to log in to the Azure Portal and find the newly created Resource Groups:
8. Open the resource group *gocdacirg and then select the container group *gocdaci:
9. The IP Address of the container is displayed. You can copy this and paste it into a browser window along with the port the container exposed. In the case of Go CD it is 8153:
10. The process is now completed.

The Azure Container Instance can now be deleted and recreated at will, to reduce cost or simply upgrade to a new version. The Azure File Share will persist the data stored by the container into the mounted volume:

Hopefully this process will help you implement persisted storage containers in Azure Container Instances more easily and quickly.

Thanks for reading!

Using Azure Key Vault with PowerShell – Part 1

Azure Key Vault is used to safeguard and manage cryptographic keys, certificates and secrets used by cloud applications and services (you can still consume these on-premise though). This allows other application, services or users in an Azure subscription to store and retrieve these cryptographic keys, certificates and secrets.

Once cryptographic keys, certificates and secrets have been stored in a Azure Key Vault access policies can be configured to provide access to them by other users or applications.

Azure Key Vault also stores all past versions of a cryptographic key, certificate or secret when they are updated. So this allows easily rolling back if anything breaks.

This post is going to show how:

1. Set up an Azure Key Vault using the PowerShell Azure Module.
2. Set administration access policies on the Azure Key Vault.
3. Grant other users or applications access to cryptographic keys, certificates or secrets.
4. Add, retrieve and remove a cryptographic key from the Azure Key Vault.
5. Add, retrieve and remove a secret from the Azure Key Vault.

Requirements

Before getting started there is a few things that will be needed:

1. An Azure account. I’m sure you’ve already got one, but if not create a free one here.
2. The Azure PowerShell module needs to be installed. Click here for instructions on how install it.

Install the Key Vault

The first task is to customize and install the Azure Key Vault using the following PowerShell script.

But first, the variables in the PowerShell script need to be customized to suit. The variables in the PowerShell script that needs to be set are:

• $subscriptionName – the name of the Azure subscription to install the Key Vault into.
• $resourceGroupName – the name of the Resource Group to create to contain the Key Vault.
• $keyVaultName – the name of the Key Vault to create.
• $location – the Azure data center to install the Key Vault to (use Get-AzureRMLocation to get a list of available Azure data centers).
  
• $keyVaultAdminUsers – an array of users that will be given administrator (full control over cryptographic keys, certificates and secrets). The user names specified must match the full name of users found in the Azure AD assigned to the Azure tenancy.

It will take about 30 seconds for the Azure Key Vault to be installed. It will then show up in the Azure Subscription:

Assigning Permissions

Once the Azure Key Vault is setup and an administrator or two have been assigned, other access policies will usually need to be assigned to users and/or application or service principal.

To create an access policy to allow a user to get and list cryptographic keys, certificates and secrets if you know the User Principal Name:

Note: the above code assumes you still have the variables set from the ‘Install the Key Vault’ section.

If you only have the full name of the user then you’ll need to look up the Object Id for the user in the Azure AD:

Note: the above code assumes you still have the variables set from the ‘Install the Key Vault’ section.

To create an access policy to allow a service principal or application to get and list cryptographic keys if you know the Application Id (a GUID):

Note: the above code assumes you still have the variables set from the ‘Install the Key Vault’ section.

Changing the values of the PermissionsToKeys, PermissionsToCertificates and PermissionsToSecrets parameters in the cmdlets above allow different permissions to be set for each policy.

The available permissions for certificates, keys and secrets are:

An access policy can be removed from users or service principals using the Remove-AzureRmKeyVaultAccessPolicy cmdet:

Note: the above code assumes you still have the variables set from the ‘Install the Key Vault’ section.

Working with Secrets

Secrets can be created, updated, retrieved and deleted by users or applications that have been assigned with the appropriate policy.

Creating/Updating Secrets

To create a new secret, use the Set-AzureKeyVaultSecret cmdlet:

Note: the above code assumes you still have the variables set from the ‘Install the Key Vault’ section.

This will create a secret called MyAdminPassword with the value P@ssword!1 in the Azure Key Vault.

The secret can be updated to a new value using the same cmdlet:

Additional parameters can also be assigned to each version of a secret to control how it can be used:

• ContentType – the type of content the secret contains (e.g. ‘txt’)
• NotBefore – the date that the secret is valid after.
• Expires – the date the secret is valid until.
• Disable – marks the secret as disabled.
• Tag – assigns tags to the secret.

For example:

Retrieving Secrets

To retrieve the latest (current) version of a secret, use the Get-AzureKeyVaultSecret cmdlet:

This will assign the stored secret to the variable $secretText as a SecureString. This can then be passed to any other cmdlets that require a SecureString.

To list all the versions of a secret, add the IncludeVersions parameter:

To retrieve a specific version of a secret, use the Get-AzureKeyVaultSecret cmdlet with the Version parameter specified:

Removing Secrets

Finally, to remove a secret use the Remove-AzureKeyVaultSecret cmdlet:

That pretty much covers managing and using secrets in Azure Key Vault using PowerShell.

Cryptographic keys and Certificates

In the next part of this series I’ll cover using Azure Key Vault to use and manage cryptographic keys and certificates. Thanks for sticking with me this far.

 

 

 

Easily Create a Hyper-V Windows Server 2016 AD & Nano Server Lab

Introduction

One of the PowerShell Modules I’ve been working on for the last year is called LabBuilder.The goal of this module is:

To automatically build a multiple machine Hyper-V Lab environment from an XML configuration file and other optional installation scripts.

What this essentially does is allow you to easily build Lab environments using a specification file. All you need to do is provide the Hyper-V environment and the Operating System disk ISO files that will be used to build the lab. This is great for getting a Lab environment spun up for testing or training purposes.

Note: Building a new Lab can take a little while, depending on the number of VM’s in the Lab as well as the number of different Operating Systems used. For example, a Lab with 10 VMs could take an hour or two to spin up, depending on your hardware.

The LabBuilder module comes with a set of sample Labs that you can build “as is” or modify for your own purpose. There are samples for simple one or two machine Labs as well as more complex scenarios such as failover clusters and two tier PKI environments. Plus, if you’re feeling adventurous you can easily create your own LabBuilder configurations from scratch or by modifying an existing LabBuilder configuration.

In this article I’ll show how to use a configuration sample that will build a lab containing the following servers:

• 1 x Windows Server 2016 RTM Domain Controller (with DNS)
• 1 x Windows Server 2016 RTM DHCP Server
• 1 x Windows Server 2016 RTM Certificate Authority Server
• 1 x Windows Server 2016 RTM Edge Node (Routing and Remote Access server)
• 8 x Windows Server 2016 RTM Nano Servers (not yet automatically Domain Joined – but I’m working on it).

This is a great environment for experimenting with both Windows Server 2016 as well as Nano Server.

So, lets get started.

Requirements

To follow along with this guide your Lab host (the machine that will host your Lab) will need to have the following:

Be running Windows Server 2012 R2, Windows Server 2016 or Windows 10

I strongly recommend using Windows 10 Anniversary Edition.

If you are using Windows Server 2012 R2 you will need to install WMF 5.0 or above. Although WMF 4.0 should work, I haven’t tested it.

Have enough RAM, Disk and CPU available for your Lab

Running a lot of VMs at once can be fairly taxing on your hardware. For most Sample Lab I’d recommend at least a quad core CPU, 16 GB RAM and a fast SSD with at least 10 GB per VM free (although for Nano Server VMs only 800MB is required).

The amount of disk used is minimized by using differencing disks, but Labs can still get pretty big.

Hyper-V Enabled

If you’re using Windows 10, see this guide.

If you’re using Windows Server 2012 R2 or Windows Server 2016, you probably already know how to do this, so I won’t cover this here.

Copies of any Windows install media that is used by the Lab

In our case this is just a copy of the Windows Server 2016 Evaluation ISO. You can download this ISO from here for free.

You can use non-evaluation ISOs instead if you have access to them, but at the time of writing this the Windows Server 2016 non-evaluation ISO wasn’t yet available on my MSDN subscription.

An Internet Connection

Most Labs use DSC to configure each VM once it has been provisioned, so the ability to download any required DSC Resources from the PowerShell Gallery is required. Some sample Labs also download MSI packages and other installers that will be deployed to the Lab Virtual Machines during installation – for example RSAT is often installed onto Windows 10 Lab machines automatically.

The Process

Step 1 – Install the Module

The first thing you’ll need to do is install the LabBuilder Module. Execute this PowerShell command at an Administrator PowerShell prompt:

Note: If you have an older version of LabBuilder installed, I’d recommend you update it to at least 0.8.3.1081 because this was the version I was using to write this guide.

Step 2 – Create the ISOs and VHDs Folders

Most labs are built using Windows Install media contained in ISO files. These are converted to VHD files that are then used by one or more Labs. We need a location to store these files.

By default all sample Labs expect these folders to be D:\ISOs and D:\VHDs. If you don’t have a D: Drive on your computer, you’ll need to adjust the LabBuilder configuration file in Step 4.

Execute the following PowerShell commands at an Administrator PowerShell prompt:

Step 3 – Create a Folder to Contain the Lab

When building a Lab with LabBuilder it will create all VMs, VHDs and other related files in a single folder.

For all sample LabBuilder configurations, this folder defaults to a folder in C:\vm. For the sample Lab we’re building in this guide it will install the Lab into c:\vm\NANOTEST.COM. This can be changed by editing the configuration in Step 4.

Note: Make sure you have enough space on your chosen drive to store the Lab. 10GB per VM is a good rough guide to the amount of space required (although it usually works out as a lot less because of the use of differencing disks).

Execute the following PowerShell commands at an Administrator PowerShell prompt:

Step 4 – Customize the Sample Lab file

We’re going to build the Lab using the sample Lab found in the samples folder in the LabBuilder module folder. The sample we’re using is called Sample_WS2016_NanoDomain.xml. I’d suggest editing this file in an editor like Notepad++.

If you changed the paths in Step 2 or Step 3 then you’ll need to change the paths shown in this screenshot:

You may also change other items in the Settings section, but be aware that some changes (such as changing the domain name) will also need to be changed elsewhere in the file.

If you already have an External Switch configured in Hyper-V that you’d like to use for this Lab to communicate externally, then you should set the name of the switch here:

If you don’t already have an External Switch defined in Hyper-V then one called General Purpose External will be created for you. It will use the first Network Adapter (physical or team) that is not already assigned to an External Switch. You can control this behavior in the LabBuilder configuration file but it is beyond the scope of this guide.

Save the Sample_WS2016_NanoDomain.xml once you’ve finished changing it.

Step 5 – Copy the Windows Media ISOs

Now that the ISOs folder is ready, you will need to copy the Windows Install media ISO files into it. In this case we need to copy in the ISO for Windows Server 2016 (an evaluation copy can be downloaded from here).

The ISO file must be name:

14393.0.160715-1616.RS1_RELEASE_SERVER_EVAL_X64FRE_EN-US.ISO

If it is named anything else then you will either need to rename it or go back to Step 4 and adjust the sample Lab configuration file.

Step 6 – Build the Lab

We’re now ready to build the lab from the sample configuration.

Execute the following PowerShell commands at an Administrator PowerShell prompt:

This will begin the task of building out your Lab. The commands just determine the location of your LabBuilder sample file and then call the Install-Lab cmdlet. I could have specified the path to the sample file manually, and you can if you prefer.

So sit back and grab a tea or coffee (or beer), because this will take a little while.

Note: The individual virtual machines are configured using PowerShell DSC after they are first started up. This means that it might actually take some time for things like domain joins and other post configuration tasks to complete. So if you find a Lab VM hasn’t yet joined the domain, it is most likely that the DSC configuration is still being applied.

Using the Lab

Once you’ve built the Lab, you can log into the VMs like any other Hyper-V VM. Just double click the Virtual Machine and enter your login details:

For the sample Lab the Domain Administrator account password is configured as P@ssword!1. This is set in the Lab Sample configuration and you can change it if you like.

Note: Nano Server is not designed to have an interactive GUI. You interact with Nano Server via PowerShell Remoting. You’ll want to have a basic knowledge of PowerShell and PowerShell Remoting before attempting to administer Nano Servers.

Shutting Down the Lab

Once the Lab has been completely built, you can shut it down with the Stop-Lab command. You need to pass the path to the Lab Configuration file to shut it down:

The Virtual Machines in the Lab will be shut down in an order defined in the Lab Configuration file. This will ensure that the VMs are shut down in the correct order (e.g. shut down the domain controllers last).

Starting the Lab Up

If you need to start up a previously created Lab, use the Start-Lab command. You will again need to provide the path to the Lab Configuration file of the Lab you want to shut down:

The Virtual Machines in the Lab will be started up in an order defined in the Lab Configuration file. This will ensure that the VMs are started up in the correct order.

Uninstalling the Lab

If you want to completely remove a Lab, use the Uninstall-Lab command. You will again need to provide the path to the Lab Configuration file of the Lab you want to unisntall:

Note: You will be asked to confirm the removals.

Wrapping Up

This article has hopefully given you a basic understanding of how to use LabBuilder to stand up a Hyper-V Lab in relatively short order and without a lot of commands and clicks. This project is still in Beta and so there may be bugs as well as some incomplete features. If you want to raise an issue with this project (or even submit a PR), head on over to the GitHub repository.

Nano Server Packages in Windows Server 2016 TP5

Happy TP5 day!

Here is just a quick snapshot of the package list for Nano Server in Windows Server 2016 TP5:

Right off the bat I notice a few new ones are included:

• BootFromWim
• SecureStartup
• ShieldedVM

I can guess at the purpose of these packages, but will be interested to learn a bit more about them. That is all I have time for as I now need to go and update the LabBuilder sample projects to use this new ISO.