Understanding Azure Virtual Machine Scale Sets (Part 2)

Scale set refresher

In part 1 of this article, I described what Azure scale sets are, how they work, and a bit about what you can do with them. Part 1 covered creating scale sets manually through the Azure portal, which is a good way to start if you just want an army of VMs to run some arbitrary task. In this article, we’ll build on what you learned in part 1 by describing how to build scale sets from the command line and how to customize your scale set worker VMs with PowerShell and templates.

Scale set command line basics

Microsoft has done a superb job of making Azure compute services accessible across multiple platforms. You can use the Azure command-line interface (Azure CLI) package or Azure PowerShell to create, modify, operate, and remove scale sets. Which tool you use is subject to two factors: your personal preference and whether Azure PowerShell runs on your desired target platform. I prefer PowerShell because I can mix Azure, Windows, and Office 365 PowerShell in a single session, and I prefer it as a scripting language to bash, so normally I would use it. However, the az vmss command family has a much higher level of abstraction—you can create a VM scale set with a single az vmss command because it can create a load balancer, default network, and other required objects for you. By comparison, the example PowerShell code Microsoft posts here requires 17 lines of code. The tradeoff is that you have much more control when using Azure PowerShell because you can specify every aspect of what you actually want to create. The same is generally true for all the other things you might do with a scale set: Azure PowerShell gives you the most control but az vmss is simplest to use.  Whichever version of the command-line tools you want, you’ll get them from the Azure SDK download page.

Instead of digging into the minutiae of creating scale sets from the command line, let me share a secret with you: it’s easier and faster to use template files that specify exactly how you want your scale set built. The best way to work with scale sets day-to-day is to build a template that does what you want, then use the command line to manage the existing scale set—so that’s what we’ll do in this article.

Template basics

Scale sets use the standard Azure Resource Manager template system. That means you’ll create a JSON file that specifies three main things:

While you can certainly build your own templates completely from scratch, there’s a far easier way. The Azure team maintains a library of quickstart templates at https://github.com/Azure/azure-quickstart-templates that you can use to quickly dive in to working with various Azure features through the Azure Resource Manager (AzureRM) interface. While the template library is well worth investigating for fun (there’s a ton of neat stuff there to poke around in), we’re most interested in the VM scale set templates. For this article, I chose the 201-vmss-windows-jumpbox template because it will allow you to set up a scale set that has a single Azure VM that you can RDP into and configure. This is useful because it makes it easier to fine-tune your VM configuration; once you have the template and scale set built exactly the way you want, you may not need to access the individual VMs. In fact, that should probably be your goal: have a scale set that you can deploy in a completely automated manner, with no post-install tweaking required.

Building the template

Start by going to https://github.com/Azure/azure-quickstart-templates/tree/master/201-vmss-windows-jumpbox and saving the template file, then open it in your favorite text editor (I am a big fan of Notepad++ but any tool that can edit JSON files will do). If you were to create a scale set with this template, with no modification, you’d get up to 100 Windows 2008 R2, Windows 2012, or Windows 2012 R2 machines, plus a separate jumpbox. Let’s start by adding support for Windows 2016. Find the section in the Parameters block labeled windowsOSVersion and add a new element to the allowedValues element for Windows 2016, as shown in the screenshot below. Notice that I also edited the defaultValue element so that we’ll get Windows 2016 by default.

Next, consider what size of VM instance we actually need. Microsoft offers a wide range of VM instances, including instances with multiple CPU cores, powerful GPUs, and other added-cost features that we don’t need for testing. There’s a helpful guide to sizing at https://docs.microsoft.com/en-us/azure/cloud-services/cloud-services-sizes-specs, but for our purposes, we want something cheap and small—after all, right now all we’re doing is testing. The VM instance size created for the scale set in this template is controlled by the value of the vmSku parameter. For now, we’ll stick with Standard_A1.

Elsewhere in the Parameters block, you can see parameters that you’ll set at create time for the scale set name, the number of VMs you want in it, and the admin username and password that will be applied to the VMs in the scale set. The default template will assign the same user name and password to all the machines, including the jumpbox. Let’s add a new parameter set so that the jumpbox can have its own unique set of credentials. The simplest way to do this is to copy and paste the adminUsername and adminPassword blocks and rename the copies to jumpAdminUsername and jumpAdminPassword, as shown below.

Next, scroll down to the Variables block. You’ll notice that many of the variables use parameter as input—for example, the jumpboxName variable is set to the combination of the scale set name with “jbox” tacked on the end, like this: "jumpBoxName": "[concat(variables('namingInfix'), 'jbox')]".

For now, that’s enough customization to get a basic scale set up—one of the nice things about using templates is that you can iteratively change your template, try it out, and make changes based on the results. This process is much more difficult if you rely solely on the command line.

Creating a templated scale set with PowerShell

Once you have Azure PowerShell installed, launch it and you’ll be ready to work with scale sets. The first required step is to connect to Azure and select the subscription you want to use for managing your scale sets. To log in, use the Login-AzureRmAccount cmdlet. You can provide credentials with the familiar -Credential parameter, but if you don’t you’ll get an ADAL-based login dialog to authenticate to the service.

Once you’ve logged in to the service, you’ll need to select the Azure subscription in which you want to create your new scale set. If you only have one subscription in your Azure account, you can skip this; otherwise, use Get-AzureRmSubscription and Select-AzureRmSubscription to control where the new resources you create are accounted for.

For testing, I normally recommend creating a new Azure resource group for your test objects. This makes it easy to clean up the results of your testing without contaminating any of your production resource groups. Thankfully, Azure makes this simple; create a new resource group with New-AzureRmResourceGroup. You’ll need to specify a name and the Azure region where you want the resources to live, as shown below:

Once you know what correct subscription and resource group, you want, the New-AzureRmResourceGroupDeployment cmdlet is what you’ll actually use to create the scale set. This cmdlet has a really large parameter set, but the only things you need to specify are the resource group where you want the scale set created and the template file you want to use. You can specify the location of the template file as a URI (which is handy if you have it in Azure Storage or on a web site). The parameters that the template file needs, if any, can be specified in a file (using the -TemplateParameterFile or -TemplateParameterUri switches) or just via the command line. PowerShell will prompt you for any parameter values you leave out, or if you don’t specify the template parameter file. For example, here’s what you’d see when running New-AzureRmResourceGroupDeployment with some, but not all, of the template parameters specified on the command line—note that I was prompted for values for the adminPassword and jumpAdminPassword parameters.

As Azure processes the template, it will give you diagnostic updates on its progress. If an error occurs, you’ll see the familiar red console text that PowerShell cmdlets use to tell you what’s wrong. For example, if you enter an insufficiently strong password for either of the password parameters, you’ll see an error message like the one below, which makes the problem immediately obvious:

Sometimes the problem may be more subtle, especially as you start writing your own templates. These problems happen often enough that Microsoft has produced an entire troubleshooting guide for Azure Resource Manager at http://aka.ms/arm-debug. The guide contains some practical tips for troubleshooting ARM deployment, whether you do it through the portal, PowerShell, or the command line, and whether manually or with templates. The most common errors you are likely to commit include simple typos in the template file (this is where an editor that understands JSON syntax really helps) and using the wrong name for a parameter or variable (e.g. defining a parameter named “adminusername” and then referring to it later as “adminuser”). Basically, if you’ve ever written scripts or code in other programming languages, you’ll be familiar with the most likely types of errors. Less often, you’ll run into issues caused by constraints in the service, such as trying to create a resource you can’t use in the selected region or taking an action blocked by the ARM policy active on your resource group. Most of these are self-explanatory and easy to fix by following the troubleshooting guide.

Adding customizations

The basic scale set you just created is a flock of Windows VMs, with no software or applications on them other than what comes with the base OS. This isn’t a very interesting configuration. As I mentioned earlier, you have a few options for customizing your scale set worker VMs. The easiest to start with is to run a custom PowerShell script. ARM supports this option through the extensionProfile construct in a template file; by specifying that you want the CustomScriptExtension run, you can download and execute a script of your choosing. CustomScriptExtension is documented at https://docs.microsoft.com/en-us/azure/virtual-machines/windows/extensions-customscript but here are the basic things you need to know:

Here’s an example CustomScriptExtension definition:

"extensionProfile": {
"extensions": [
{
"name": "customScript",
"properties": {
"publisher": "Microsoft.Compute",
"settings": {
"fileUris": [
"https://contoso.blob.core.windows.net/config.ps1"
]
},
"typeHandlerVersion": "1.8",
"autoUpgradeMinorVersion": true,
"protectedSettings": {
"commandToExecute": "powershell -ExecutionPolicy Unrestricted- File config.ps1"
},
"type": "CustomScriptExtension"
}
}
]
}

In this case, when the extension runs, it will fetch the config.ps1 script from the specified fileUris value, then execute it with PowerShell. Microsoft’s documentation points to a sample script at https://raw.githubusercontent.com/Microsoft/dotnet-core-sample-templates/master/dotnet-core-music-windows/scripts/configure-music-app.ps1 that illustrates some more practical configuration tasks, including setting up firewall rules, installing Windows components, fetching an application package, and installing it. One of the best things about this approach is that you can create and debug your script on any ordinary machine, fine-tune it until it does exactly what you want, and then deploy it with a CustomScriptExtension. This will be a familiar process to anyone who’s used PowerShell DSC or other automation techniques in the past.

When the extension runs, the files it downloads are stored in a dynamically created directory, and the working directory used for commandToExecute is set to that directory. It’s easiest if you just use this shortcut to access your files, rather than hardcoding paths into your configuration scripts. Its results are logged on each machine under the C:WindowsAzureLogsPluginsMicrosoft.Compute.CustomScriptExtension directory. This is very valuable for troubleshooting, as it gives you detailed information on what happened when your extension ran.

Testing your new scale set

To work with your scale set, you’ll need some information that is most easily obtained from the portal at https://portal.azure.com. Log in to the portal and choose “Virtual machine scale sets” from the left-side navigation bar. (You may need to use the “More services” command to make scale sets visible.) As soon as the creation operation finishes, Azure starts the VMs in your scale set, so you can immediately access them. The example template provided with this article includes a jumpbox that you can use to get RDP access from the Internet to the VMs in your scale set. Depending on your application, you may not want or need individual scale set VMs to be accessible externally, so the exact mechanism you use for testing may be as simple as validating that your application is running by loading a URL through a load balancer or as complex as logging into each scale set member and running test procedures.

Summary

Scale sets driven by templates provide a flexible, easily configured way to quickly build large flocks of VMs that you can customize to run applications of any time. The most critical restriction is that the machines have to start in an identical state (unless you want to manually configure each of them) but with a bit of up-front work you can build a reliable and robust scale set and put it to whatever use you need. This is a great example of how scalable cloud computing is providing capabilities that just didn’t exist before hand!