Peering into the future of windows automation testing with Chef, Vagrant and Test-Kitchen – Look mom, no SSH! by Matt Wrock


Linux automation testing has been supported for a while now using many great tools like chef, puppet, Test-KitchenServerSpec, MiniTest, Bats, Vagrant, etc. If you were willing to install an SSH server on Windows, you could get most of these tools to work but if you wanted to stay “native” you were on your own.

Pictured above: Testing node convergence on an 8 inch tablet.

I’m not at all morally opposed to installing SSH on windows. I love SSH. We spoon regularly. But while SSH is “just there” on linux, it incurs an extra install step for windows that must either be done manually or included in initial provisioning or image creation. Also, for some windows-only shops, the unfamiliarity of SSH may add a layer of unwanted friction in an automation ecosystem where windows is often an after thought.

Well recent efforts to make Windows testing a first class experience are beginning to take shape. Its still early days and some of the bits are not yet “officially” released but are available to use by pulling the latest bits from source control. I know…I know…to many that will still spell “friction” in bold. However, I want to share that one can today test windows machine builds via winrm with no SSH server installed, and I also want to offer a glimpse to those who prefer to wait until everything is fully baked of what is to come, and inform you that the wheels are in motion so please keep abreast of these developments.

Note: I presented much of this material and several Boxstarter demos to the Philadelphia PowerShell User Group last week, the video is available here.

Its not automated until it is tested

I work for CenturyLink Cloud and infrastructure automation is front and center to our business. Like many shops, we have a mixed environment and central to our principals is the belief that testing our automation is just as important as building our automation. In fact they are not even two separate concepts. Untested automation is not finished being built. So I am going to share with you here how we test our Windows server infrastructure along with some other bits I have been working with on the side.


If you have not heard of Vagrant, just stop reading right now and mosey on over to Vagrant is a hypervisor agnostic way of spinning up and provisioning servers that is particularly suited for developing and testing. It completely abstracts both the VM infrastructure as well as many possible provisioning systems (chef, puppet, plain shell scripts, docker and many many more) so that one can provision and share the same machine among a team using different platforms.

To illustrate the usefulness here, where I work we have a diverse team where some prefer MACs, others work on Windows and others (like myself) run a Linux desktop. We use Chef to automate our infrastructure and anyone who needs to create or edit chef artifacts needs all sorts of dependencies installed with specific versions in order to be successful. Vagrant plays a key role here. Anyone can download our Ubuntu 12.04 base image via VirtualBox, VMWare or Hyper-V and then use its Chef provisioner plugin to build that image to a state that mirrors the one used by the entire team. all this is done by including a small file of metadata that serves as a pointer to here the base images can be found as well as the chef recipes. If this sounds interesting, again I refer you to Vagrant’s documentation for the details What I want to point out here is its windows support.

Added support for WinRM and Hyper-V

Until fairly recently, Vagrant only supported SSH as a transport mechanism to provision a VM. It also lacked official Hyper-V support as a VM provider. This changed in version 1.6 with a WinRM “Communicator” and a Hyper-V provider plugin included in the box. While I don’t really use Hyper-V at work, I have some windows based personal projects at home and I prefer to use Hyper-V. So I quickly tested out this new plugin and was happy to see it available. There are still some kinks in the current version but work is underway to improve the experience. I’m trying to to personally contribute to issues that are blocking my own work and a couple have been accepted into Vagrant Master. Overall that has been a lot of fun. Here are the issues that have come up for me:

  • Only .vhdx image files are supported and .vhd files cannot be imported. I hit a wall with this when trying to use the .vhd files freely available for testing here on Technet. I have since added a patch which has been accepted to fix this.
  • Generation 2 Hyper-V VMs are imported as Generation 1 VMs and fail to boot. Oddly, most .vhdx images tend to be generation 2. My PR for this issue was just accepted yesterday.
  • Synced folders over SMB (this is the norm for a windows host/windows guest setup) fail. I’m hoping my PR for this issue is accepted.

If these same issues become blockers for you, the first two can be immediately fixed by pulling the latest copy of Vagrant’s master branch and copying the lib and plugin directories onto the installed version and you are welcome to pull my smb_sync branch which includes all of the fixes:

git clone -b smb_sync
copy-item -path vagrant\lib `
  C:\HashiCorp\Vagrant\embedded\gems\gems\vagrant-1.6.3 `
  -recurse -force
copy-item -path vagrant\plugins `
  C:\HashiCorp\Vagrant\embedded\gems\gems\vagrant-1.6.3 `
  -recurse -force

Having worked with Vagrant for the past few months, I’ve been finding myself wishing there was a remote powershell equivilent to the vagrant SSH command which drops you into an ssh session on the guest box. So today I banged out a first draft of a vagrant ps command that does just that and will submit once it is more polished. You can expect it to look like this:

C:\dev\vagrant\win8.1x64> vagrant ps
default: Creating powershell session to
default: Username: vagrant
[]: PS C:\Users\vagrant\Documents>

A base box for testing

I’ve been playing with creating windows vagrant boxes. Unfortunately for Hyper-V, the vagrant package command is not yet implemented so I have to “manually” create the base box. Perhaps I’ll work on an implementation for my next contribution. My Windows 2012R2 Hyper-V box requires all the above fixes to install without error. You could use this Vagrantfile to test:

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| = "mwrock/Windows2012R2"
  config.vm.box_url = ""
  # Change "." below with your own folder you would like to sync
  config.vm.synced_folder ".", "/chocolateypackages", disabled: true
  config.vm.guest = :windows 
  config.vm.communicator = "winrm"
  config.winrm.username = "administrator"
  config.winrm.password = "Pass@word1"

Note here that you need to specify :windows as the guest. Vagrant will not infer that on its own nor will it assume you are using winrm if you are using a windows guest so make sure to add that to your boxes as well if you intend to use winrm.


Test-Kitchen is a testing framework most often used for testing Chef recipes (hence – kitchen). However I understand it is also compatible with Puppet as well. Like many tools in this space such as Vagrant above, it is highly plugin driven. Test-Kitchen by itself doesn’t really do much. What Test-Kitchen brings to the table (Ha Ha! I said table. get it?) is the ability to bring together a provisioning configuration management system like Chef and Puppet, a myriad of different cloud and hypervisor platforms and several testing frameworks. In the end it will spin up a machine, run your provisioning code and then run your tests. Further you can integrate this in your builds providing quick feedback as to the quality of your automation upon committing changes.

“Official” support for windows guests coming soon

Currently the “official” release of Test-Kitchen does not support winrm and must go through SSH on windows. However, Salim Afiune (@afiune), a developer with Chef has been working on adding winrm support. I have plumbed this into our Windows testing at CenturyLink cloud and have also used it developing my Boxstarter cookbook, which allows one to embed boxstarter based powershell in a recipe and provides all the reboot resiliency and windows config functions available in Boxstarter core. Salim has also contributed corresponding changes to the vagrant and EC2 Test-Kitchen drivers.

At CenturyLink, we use vmware and a customized vsphere driver to test with Test-Kitchen. It was trivial to add support for Salim’s branch.. With the Boxstarter cookbook, I use his vagrant plugin without issue. According to this Chef blog post, all of this windows work will likely be pulled into the next release of Test-Kitchen.

But I just cant wait. I must try this today!

So for those interested in “kicking the tires” today, here is how you can install all the bits needed:

cinst chefdk
cinst vagrant

git clone -b transport
git clone -b transport 

copy-item test-kitchen\lib `
  C:\opscode\chefdk\embedded\apps\test-kitchen ` 
  -recurse -force
copy-item test-kitchen\support `
  C:\opscode\chefdk\embedded\apps\test-kitchen `
  -recurse -force
copy-item -Path kitchen-vagrant\lib ` 
  -recurse -force 

cd test-kitchen
gem build .\Test-kitchen.gemspec
chef gem install test-kitchen-1.3.0.gem

This will install the Chef Development Kit and vagrant via Chocolatey and I’m assuming you have chocolatey installed. Otherwise you can download these from their respective download pages here and here. Then it clones the winrm based test-kitchen and kitchen-vagrant projects and copies them over the current bits.

Note that my instructions here are assuming you are testing on Windows. However, the winrm functionality is most certainly capable of running on Linux as I do at work. If you were doing this on Linux, I’d suggest running bundle install and bundle exec instead of copying over the chef directories. However this has caused me too many problems on Windows to recommend to others and purely copying the bits has not caused me any problems.


Now you can pull down the boxstarter cookbook to test from If you run Hyper-V, you will want to install my vagrant fixes according to the instructions above since the box inside the boxstarter cookbook’s kitchen config is on a vhd file. You can then simply navigate to the boxstarer cookbook directory and run:

kitchen test

This will build a win 2012 R2 box and install and test a very simple cookbook via Test-Kitchen.

Virtual Box

If you run VirtualBox, you will need to make a couple changes. Replace the VagrantfileWinrm.erb content with this:

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| = "<%= config[:box] %>"
  config.vm.box_url = "<%= config[:box_url] %>"
  config.vm.guest = :windows 
  config.winrm.username = "vagrant"
  config.winrm.password = "vagrant"
  config.winrm.port = 55985

You would also replace the .kitchen.yml content with:

The test included in the boxstarter cookbook is not very interesting but illustrates that you can indeed run kitchen tests against windows machines with no ssh installed.

  name: vagrant 

  name: chef_zero 

  - name: windows-81 
      name: winrm 
      max_threads: 1 
      port: 55985 
      username: vagrant 
      password: vagrant 
      guest: :windows 
      box: mwrock/Windows8.1-amd64 vagrantfile_erb: VagrantfileWinrm.erb 

  - name: default
      - recipe[boxstarter_test::simple] 

Looking at a more interesting ServerSpec test

For those reading who might want to see what a more interesting test would look like, lets take a look at this Chef recipe:

include_recipe 'boxstarter::default'

boxstarter "boxstarter run" do
  password 'Pass@word1'
  code <<-EOH
    Update-ExecutionPolicy Unrestricted
    Set-WindowsExplorerOptions -EnableShowHiddenFilesFoldersDrives `
      -EnableShowProtectedOSFiles -EnableShowFileExtensions 
    cinst console2
    cinst IIS-WebServerRole -source windowsfeatures

    #Install-WindowsUpdate -acceptEula

This is a sample recipe I include with the Boxstarter cookbook but I have commented out the call that runs windows updates. This recipe will run the included Boxstarter resource and perform the following:

  • Update the powershell execution policy

  • Adjust the windows explorer settings

  • enable remote desktop

  • install the console2 command line console

  • Install IIS

Here is a test file that will check most of the items changed by the recipe:

require 'serverspec'

include Serverspec::Helper::Cmd
include Serverspec::Helper::Windows 

describe file('C:\\programdata\\chocolatey\\bin\\console.exe') do
  it { should be_file }

describe windows_feature('Web-Server') do
  it{ should"powershell") }

describe windows_registry_key(
  'HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\advanced') do
  it { should have_property_value('Hidden', :type_dword,'1') }

describe command('Get-ExecutionPolicy') do
  it { should return_stdout 'Unrestricted'}

Serverspec provides a nice Ruby DSL for testing the state of a server. Although the test is pure ruby code, in most cases you don’t really need to know ruby. Familiarity with the cut and paste features will be very helpful so please review those as necessary.

The documentation on the page does a decent job of describing the different resources that can be tested. Above are just a few:  a file resource, windows feature resource, a windows registry resource and a command resource that you can use to issue any powershell necessary to test your server.

All of these tests, as we do at CenturyLink can be fed into a Continuous Integration server (Jenkins, TeamCity, TFS, etc.) to give your team speedy feedback on the state of your automation codebase.

I hope you find this helpful and I look forward to these features making it into the official vagrant and test-kitchen installs soon.

Leaving Microsoft and Building a ‘DataCenterStarter’ for CenturyLink Cloud by Matt Wrock

As of last week I am no longer working at Microsoft. I worked at Microsoft for the last four and a half years and it was an amazing experience where I learned a lot from many very smart people. I am now a Software Engineer focusing on data center automation at CenturyLink Cloud.

What the heck did I do at Microsoft?

I came to Microsoft and the Pacific Northwest from Southern California where I had spent the previous 9 years working for an online advertising company starting as a front line web developer and eventually becoming VP of technology. I reached a point where I wanted a change and a return to hands on engineering. Having no formal computer engineering training and being almost exclusively exposed to “startup” shops, I really wanted to work for a major technology company to witness how a well established organization runs things. Well I definitely got what I was looking for and received exposure to some amazing people and practices.

At Microsoft, I started working on the Visual Studio Gallery and several other similar sites like the Technet script gallery and the MSDN Code Sample gallery as well as some of the “goo” that provided a unified experience for the Microsoft Forums, the galleries, search and profile pages on MSDN and Technet. Some of the greatest things I walked away with here was an engrained devotion to the practice of Test Driven Development and a great appreciation for not only the consumption but participation in Open Source Software projects.

In my free time I created an open source library that significantly improved our page load performance across the above sites as well as the msdn/technet blog and wiki platform. Later I worked on environment setup and deployment automation for these properties which inspired The last 2 years were spent in the Visual Studio Cloud Services org within DevDiv where I worked on “Feature Flags” allowing us to deploy “hidden” features while they were in the middle of development, the back end for the new Charting features inside TFS Work Item Tracking and most recently deployment automation for Visual Studio Online.

A new chapter

Over the past couple of years, my “side project” Boxstarter has consumed a lot of my passion and has led me to develop some relationships in the DevOps community and learn of many disciplines and technologies that fascinate me. I love and have become somewhat consumed by automation.

So a little over a month ago I received a twitter DM from a previous colleague, Tim Shakarian, asking me if I would be interested in building a “DataCenterStarter” for CenturyLink’s recent cloud acquisition at Tier3. I read this having just returned from dinner with my friend Rob Reynolds and some other guys from Puppet Labs and Peter Pouliot who heads up Microsoft community development of Hyper-V integration in OpenStack. Everyone present shared the same passions for automation and I was especially inspired hearing Peter’s automation stories from his Novell days and recent work with organizations like CERN. So with these conversations fresh in my mind, I wondered what are these DataCenters Tim speaks of?

I really was not “on the market” looking to move from Microsoft. In fact my role had recently changed and there were some great opportunities ahead to bring my organization to an exciting new level of engineering efficiency, but I thought it would be foolish not to at least listen to what my friend Tim had to say. Well six weeks later here I am and I am totally excited to be working on data center automation for CenturyLink Cloud. I feel like a kid in a candy store beginning work on projects that give me the opportunity to build out automation at vast scale and help my team deliver an awesome cloud solution to our customers.

Its not easy leaving behind an organization like Microsoft. There are a lot of great people there and I will truly miss my free MSDN Ultimate subscription where I managed to consistently milk about 148 of my 150 dollar monthly spending allowance on Azure services (the same granted to any MSDN Ultimate subscriber). I think it may be a couple years before I fully process my experiences at Microsoft so please be on the lookout for my forthcoming graphic novel series, Razzle Dragon, that portrays in Japanese Manga style my stint as a Microsoft software engineer.

Released Boxstarter v2.4: Test Runner for Chocolatey Packages and many more Windows GUI configuration functions by Matt Wrock

boxLogoCheckThis week I released Boxstarter version 2.4. This release introduces a new feature for testing packages, the addition of many more Windows GUI configuration functions and several bug fixes and minor enhancements.

Windows GUI Configuration Functions

Boxstarter gives great thanks to Gary Park (@gep13) who provided the pull requests delivering these functions. These are a great addition to the value provided by Boxstarter to script your box so that it not only has what you want but also looks just the way you want it. Its not always easy to remember where to right click or how to “swipe” to find the settings that make your environment the most productive place for you to get things done. Here is a list of the new functions:

  • Enable/Disable showing charms when mouse is in the upper right corner
  • Enable/Disable switching apps when pointing in the upper left corner
  • Enable/Disable the option to launch powershell from win-x
  • Enable/Disable boot to desktop
  • Enable/Disable desktop background on the start screen
  • Enable/Disable showing the start screen on the active display
  • Enable/Disable showing the Apps View by default on the start screen
  • Enable/Disable searching everywhere in apps view. Not just apps.
  • Enable/Disable showing desktop apps first in results
  • Lock/Unlock task bar
  • Change taskbar icon size
  • Change location of taskbar docking

You can find the exact function names and syntax examples here.

Testing Packages and Continuous Package Delivery

A new Boxstarter module has been added, Boxstarter.TestRunner, that can test either specific packages or all packages in a repository that have versions greater than what is published. Boxstarter can be configured to test these packages on one or more “deployment targets” to determine if the package installs are successful. Boxstarter can also be configured to publish a package to its feed if it installed successfully on all deployment targets.

The Boxstarter.TestRunner module includes some powershell scripts and an MSBuild project file that can be used to integrate with modern build servers enabling scenarios where newly committed packages and package changes are automatically tested and published. They can be tested on multiple targets perhaps with different versions of windows and all of this can be done in the cloud. If testing on Azure VMs, Boxstarter can shutdown the VMs when testing is complete so you only incur costs while tests are being performed.

Details on how to use the test runner both interactively and with different build server scenarios are documented here.

Personal Case Study using Visual Studio Online Build Services to detect changes and deploy to Azure VMs and finally publish to the public Chocolatey feed

I have a git repository of almost 50 Chocolatey packages that I keep on Github. I have installed the Boxstarter.TestRunner module which is a separate install from the core set of Boxstarter modules. I have configured Boxstarter to point to my local copy of this repository:


I have also configured the Boxstarter.TestRunner’s deployment options to use both a windows 2012 and a 2008 R2 Azure VM for testing. Before each test, Boxstarter will snap the VMs to a preset checkpoint labeled TestReady that I created with the Boxstarter.Azure module.


Furthermore I took an extra step of adding the Boxstarter.TestRunner build scripts to my repo using the Install-BoxstarterBuildScripts command which created a new folder in my repository to hold these scripts and persist my settings:


These files contain scripts that can bootstrap Boxstarter on a build server, test changed packages and publish successful packages. The xml files hold my options. Some of these, Boxstarter adds to my .gitignore file because they have VM credentials and Nuget API keys that I would not want kept in my public github repo.

Earlier today I received a great Pull Request on three Visual Studio 2013 packages that allow for one to pass additional install arguments and a product key through the Chocolatey installer. I merged those in but before pushing to my github repository I pushed to a special Remote I keep at my account. Anyone can sign up for a account for free that supports a team of 5 or fewer members. This gets you private git or tfsvc repositories, work item tracking and a hosted build server. You can also pay for more team members and services. I have a CI build definition setup to build commits to this repository named ChocolateyCI. Almost immediately after my push, a build begins. This calls the Boxstarter.proj MSBuild files which runs BoxstarterBuild.ps1 to orchestrate the testing and publishing using the Boxstarter modules.image

I have hidden it away but I have configured my Build Definition with all of the information it needs to establish a connection with my VMs using the Boxstarter.Azure module and run my packages on my VMs. I can see that my Azure VMs, wrocktest2012 and wrocktest2k8r2, have been fired up to test the committed changes:


Lets quietly sneak in to one of the VMs and just make sure they really are testing:

imageYep. They are diligently testing my packages.

Now unfortunately I can tell you right now that this build is doomed for failure. This is why: the Visual Studio Online hosted build controllers limit builds to an hour. I have 3 changed Visual Studio packages running sequentially on two small VM instances that have to restore their image before each test. That will certainly take longer than an hour. The hosted build solution is ideal for most packages but not multiple visual studio tests. That’s OK. I have other options. I have another VM that is a dedicated build server. I could push my repo to the remote watched by that server and my build can take as long as it needs. There are different configuration issues to take into consideration when working with a hosted build server like Visual Studio Online or your own private build server. These are documented on the doc pages.

For a faster and simpler option, I can test and publish locally on my local Hyper-V instances. That will be a lot faster because these VMs are beefier and the Hyper-V checkpoint restores are much faster than an Azure VM image blob restore. Even simpler, and Boxstarter’s default behavior, is to have one deployment target, localhost, and the package will be tested locally with reboots disabled. However, I already have Visual Studio 2013 installed locally and would rather test in a pristine, controlled environment.

So I’ll adjust my Test Runner settings:

imageNow I’ll run the test interactively from my current shell. As the results run and complete, status is reported to my console:


Once the package tests have completed I see the results:

imageWhoa. What happened here? Well lets just say my daughter who makes a great Chaos Monkey decided to close the lid of my laptop (the VM host) cutting off network connectivity to the VMs. Fortunately, Boxstarter is able to recover upon opening the lid:

imageThe test that was interrupted likely succeeded in completing the installation. We can inspect the error details of the failure and do not see anything representing an error from the installer. If the installer returned a non 0 exit code, we would see it in the details.

imageI’m going to go ahead and publish the passed packages now. This publishes the two packages that passed on both test VMs.

imageAlso to be on the safe side, I’m going to go ahead and resubmit the failed package for testing. I just need to rerun the Test-BoxstarterPackage command. Since the other two packages were published, my remaining one package will be the only package with a version that is greater than its published version.


Other Changes in 2.4

There were several bug fixes in this release. Thanks to everyone who filed an issue! Also, these two enhancements were added:

  1. All calls to the Chocolatey installer (cinst) inside of a package now includes the local repository as a package source. This is handy when working with private feeds or debugging packages an you want the package to be installed from your local repository instead of the public Chocolatey feed.
  2. When invoking a package install targeting a remote machine, the current user does not need to ba an admin user on the deployment box. They do need to be admin on the deployment target. This was done to support deploying from a Visual Studio Online build server where the ientity controllin the build is not admin on the build server.

Please file issues if you observe any thing that does not seem right or if you would like to suggest new features. I’d love to know what you think would be a worthy addition to Boxstarter functionality.

Automate the Install and setup of a Team Foundation 2013 Server with Build services on a Azure VM with Boxstarter by Matt Wrock

tfsBoxLast week I released version 2.3 of Boxstarter which includes some Azure VM integration features to Boxstarter’s Chocolatey/Nuget Package management approach to Windows environment automation tools. I blogged about how one can use it to deploy and configure a publicly accessible Minecraft server. Well Minecraft might not be for everyone. Others might prefer a TFS 2013 server.

In this post we will install Boxstarter and the Windows Azure PowerShell tools, create a Windows Azure VM, and with a single command deploy a chocolatey package that will connect to our VM and install Sql Server 2012 Express with SP1, Team Foundation Server 2013 Express, configure the TFS server to connect to the database and create a default collection and also configure and start build services. You will then be able to launch a browser or Visual Studio and connect to your VM on port 8080 to access these services. Because the entire install and configuration is encapsulated in a Chocolatey package, you can repeat this on as many servers as you like again and again and again.

Azure is cool, but what about Hyper-V or “On Prem”

Definition: “On-Prem” is what the cool kids say when referring to On Premise installs or installations that runs on hardware which resides in your own data center.

In case you are not interested in deploying TFS on an Azure VM, I will also show you how you can apply the install package to an on prem server or a Hyper-V VM toward the later end of this post.

Chocolatey? Sounds yummy. What is it?

You are correct. It is yummy, but its not the kind of chocolate you are probably thinking of. Unless of course you are thinking of the chocolate that can install Visual Studio, Office 365, ITunes and over 1500 other applications in a single command. Next…next…finish? C’mon, what are you? a Farmer? Uhh…well if you are the guy who runs the entire TFS team (AKA Brian Harry), then maybe you are.

Chocolatey leverages Nuget packaging technology and standards to automate the installation of machine applications. While typical Nuget packages componentize code libraries that can easily be consumed by Visual Studio, Chocolatey packages make the command line installation of applications a simple and repeatable operation. Furthermore you can compose these packages to build out everything a server needs. The packages are entirely Powershell based so anything that you can do in Powershell (in other words, pretty much anything) can be captured in a package.

What does Boxstarter add?

Boxstarter provides an environment for running Chocolatey packages that can handle reboots, remote installations, windows specific settings and Windows Update control and several other features. Boxstarter takes Chocolatey and targets its use specifically for scenarios involving the setup of a Windows environment from scratch. You can check out this page on the site for details on Boxstarter specific features.

Preparing your deployment environment

Before you can begin actually deploying your Chocolatey package to build your TFS Server, we will install the Boxstarter core modules and its Boxstarter.Azure module and configure our Azure subscription account to be managed by the Windows Azure Powershell toolkit. This is a one time step that should only need to be performed once on an individual machine that uses Boxstarter.

PREREQUISITES: There are two key prerequisites to running the software and commands in this tutorial:

  • Powershell V 3 or higher. This ships with Windows 8/2012 and higher. Windows 7 and server 2008 R2 can be upgraded with the latest Windows Management Framework.
  • A Windows Azure subscription. Trying to add a VM without an Azure subscription will only end in disappointment. Don’t be disappointed. Get an Azure subscription instead! If you have an MSDN subscription, you can get one for FREEEEEE!!!

Getting Boxstarter

Getting Boxstarter is easy especially since you can get Boxstarter with Boxstarter. If you do not already have Chocolatey installed (if you do, just CINST Boxstarter.Azure), this is a no brainer. Simply direct IE, or any browser that supports click once apps, to This invokes a Click Once app that will bootstrap Chocolatey and install all Boxstarter modules including the new Boxstarter.Azure module. As I already mentioned, this is all built on top of Nuget packaging which supports package dependencies. So along with the Boxstarter.Azure package, the Windows Azure .Net libraries and Windows Azure Powershell tools will be downloaded and installed. If you do not have the .Net 4.5 framework, you get that too.

launchendNote that the /nr/ in the URL you used to kick off the Boxstarter install tells Boxstarter not to reboot your machine. Without that, if Boxstarter detects a pending reboot at any time during the install, it will reboot, and automatically log you back in and restart the install. Any install packages already installed will be skipped. Most Boxstarter packages and prerequisites should not require a reboot to run however the .Net framework version 4.5 may be an exception. So if you do not have that, you may want to remove the /nr/ from the above URL or you can manually rerun the install if you receive an error during the install.

The Boxstarter Shell

While you can use any PowerShell console to load the Boxstarter modules and run its commands (see this page for details on running Boxstarter commands), launching the Boxstarter Shell shortcut ensures that all modules are loaded and prints some “getting started” text when the shell first loads.


Due to the improved module auto loading in PowerShell version 3, this is not as much of an issue as it is in PowerShell 2 environments (which are not supported for the Azure integration features in Boxstarter). That said, if you are not familiar with PowerShell and want to use Boxstarter’s core commands in a PowerShell 2 environment, you may find using the Boxstarter Shell to provide a better experience.

Importing your Azure subscription details

Before you can create VMs or interact at all with your subscription resources via the Azure PowerShell commands, you need to import your Azure subscription and authentication certificate so that the Powershell commands can properly associate you with your account. The easiest way to establish this association is by running:


This will launch your default browser and assuming that you have not recently logged into the Azure management portal, you will find yourself at a Microsoft Account login screen. Once you successfully authenticate with your account, your publisher settings file will begin downloading.

publishChoose to Save these settings. Then after the download completes, click the “Open Folder” link and note the location where the recently downloaded publish settings file was saved. Then run Import-AzurePublishSettingsFile and pass the path of the file. My import command looks like this:



The final step to get all of your subscription settings properly configured is to set the Storage  Account to be used for all operations invoked with the Azure Powershell tools which Boxstarter uses to access your VM. If you already have a Azure VM you plan to use for your TFS server, Boxstarter can set this on its own, but we are going to assume that is not the case and create a new VM. So we will need to set this value. To find all of your current Storage accounts, if any, run:

PS C:\> Get-AzureStorageAccount

StorageAccountDescription : Implicitly created storage service
AffinityGroup             :
Location                  : West US
GeoReplicationEnabled     : True
GeoPrimaryLocation        : West US
GeoSecondaryLocation      : East US
Label                     : portalvhdslwf0p2qrfyt34
StorageAccountStatus      : Created
StatusOfPrimary           :
StatusOfSecondary         :

This is a snippet of the first of my storage accounts which is the one I will use. So I now run:

Set-AzureSubscription -SubscriptionName Subscription-1 `
  -CurrentStorageAccountName portalvhdslwf0p2qrfyt34

Subscription-1 is the name of my subscription. Creative I know. If you do not have a storage account, you can create one either using the Azure Powershell commands or using the Azure management portal. Using PowerShell, one can create a new account using:

New-AzureStorageAccount -StorageAccountName newaccount -Location "West US"

One detail not to be missed here is that the StorageAccountName must be only lowercase letters or numbers. The Location must be a valid Azure data center location. You can find all of them using the Get-AzureLocation command.

Understand that everything we have done up until now has been a one time setup process that we should not need to repeat on the same machine if you plan to use Boxstarter again.

Creating the Azure VM

You can create an Azure VM in a single command. We will use the New-AzureQuickVM command. Since this command expects an Admin user name and password and we will need these same credentials when provisioning the VM with Boxstarter, we will store the credentials once in a variable:

$secpasswd = ConvertTo-SecureString "1276Tfs!" -AsPlainText -Force
$cred=New-Object System.Management.Automation.PSCredential ("TfsAdmin", $secpasswd) 

Now lets create the VM:

New-AzureQuickVM -ServiceName MyTfsVMService -Windows -Name tfs1 `
  -ImageName 3a50f22b388a4ff7ab41029918570fa6__Windows-Server-2012-Essentials-20131217-enus `
  -Password $cred.GetNetworkCredential().Password -AdminUsername $cred.UserName `
  -InstanceSize Medium -Location "West US" –WaitForBoot

This will create a new VM named tfs1 and since I do not have an Azure Cloud Service named MyTfsVMService, it will also create a new Cloud Service in which the VM will run. You can run multiple VMs in a single cloud service. Note that the cloud service name must be unique not only to your account but to all azure. This is because the service name forms the DNS name by which the VMs are reached. All VMs created inside of MyTfsVMService will be accessed via Multiple VMs are accessed through a different port. Of course now that I have created the service, you may not reuse the name unless I delete it, which I will likely do very soon. If you have an existing cloud service that you would like to reuse, you may specify that service. If you do, make sure to omit the -Location argument since the VM will use the location assigned to the service. Finally, if you are supplying a brand new service, use the same Location as the one used by the Storage account you chose above.

A couple other things to point out here. For our TFS server, specify an instance size of AT LEAST Medium. While I tend to use smaller VMs for my personal use, with TFS and SQL Server together, you are likely to have a much better trial experience with the Medium size with 3.5GB of ram as opposed to 1.75 in the Small sized instances. Of course you pay more for the larger VMs. This is one reason we are using a Windows Server 2012 R2 image as opposed to an image prebuilt with Sql Server. Since the Sql Server image costs include the additional SQL licensing costs, they are considerably more expensive. We will be installing the Sql Express SKU which will be quite sufficient for out purposes (and free). Furthermore, the Server 2012 R2 images, according to the current Azure pricing information as of this post is provided at the lower Linux rates.

Since we specified the –WaitForBoot argument, the command will not complete until our VM has completed its build cycle and is ready for connections….Oh look!…Its ready!


Provisioning with Boxstarter

Now that we have our VM, the next logical thing to do is install our software. So what does that look like with Chocolatey packages run through Boxstarter?

Package Composition

There are several ways to approach package creation. There is a page devoted to this topic in the Boxstarter documentation. Boxstarter provides some convenient commands to make package creation easy and sometimes altogether unnecessary.  We will use a Github Gist to compose the package script. So the next logical question is “What is a package and what can/should we include in the script?”

As already stated, Chocolatey packages are based on and completely comply with the Nuget packaging specification. In the common Chocolatey scenario, the package consists of two files:

  • A Nuspec file which is an XML formated manifest with metadata describing the package. This includes key things like the package name, its version, what other packages it depends on and what files are included. There is more but this covers the basics.
  • The ChocolateyInstall file. This is a PowerShell (.ps1) file that actually performs the installation. The beauty of this file is that it can contain absolutely any valid Powershell which gives us a lot of flexibility and power. When this script is executed inside of Chocolatey, it has access to the many commands that Chocolatey exposes to cover lots of common install scenarios like downloading, unzipping, and silently installing MSI files. There are commands for creating shortcuts, installing windows features, and more. When running with Boxstarter, there are even more commands covering scenarios around initial environment setup such as installing critical Windows updates.

You can supply more files. For example there may be config files specific to the applications you are installing that you might want to include in the package. All files in the package are zipped up into a single .nupkg file. This is the file that the underlying Nuget infrastructure unpacks.

Lets take a look at what our ChocolateyInstall script looks like:

cinst VisualStudioTeamFoundationServerExpress2013
cinst MsSqlServer2012Express

$tfsConfig="$env:ProgramFiles\Microsoft Team Foundation Server 12.0\Tools\TfsConfig.exe"
.$tfsConfig unattend /configure /type:standard
.$tfsConfig unattend /configure /type:build `
  /inputs:collectionurl=http://localhost:8080/tfs`;ServiceAccountName="LOCAL SERVICE"`;ServiceAccountPassword="pass"

This uses the Chocolatey Install command CINST to first install two packages: TFS 2013 Express and SqlServer 2012 Express. Both of these packages have their own dependencies. Sql depends on the .Net framework version 3.5 and TFS depends on version 4.5. Since we are installing on to Windows Server 2012 R2, we already have .Net 4.5 but R2 does not come preinstalled with v. 3.5 so that will be installed as well.

Once these are installed we will configure TFS with a standard server configuration. This will use the local default named sql instance for the TFS configuration and collection databases and create both of them. That creates a server capable of hosting source control and work item tracking. Next we configure Build services so that now we can add Build controllers, agents and Build definitions to be executed.

Our goal is that when these commands complete, we can navigate to from our local machine and see the web portal of our TFS collection.

Package Consumption

So how do we package up this script so that we can execute it and configure our VM? One answer is: we don’t need to. Boxstarter can take a file path or http URL and as long as they resolve to a raw text resource, Boxstarter will convert them to a temporary package and run them. This is very convenient for one off installs where you do not want to go through the trouble of composing a manifest and packaging process. Not that it is so onerous of a process. The down side to this approach is that if you plan to consume the same package again and again, a raw gist URL is very awkward to type and nearly impossible to memorize.

Lets say that we intend to use this package repeatedly and therefore want to invoke the package using a reasonably short and easy to remember label. Boxstarter provides a command that can create a minimal package from our gist.

New-PackageFromScript `
  -Source `
  -PackageName TfsServerWithBuild

This just created a TfsServerWithBuild.1.0.0.nupkg file in our “local package repository”. This is a special location on disk that Boxstarter looks for packages before attempting to fetch the package from a remote nuget feed. By default this is a folder in the same directory where the Boxstarter modules live, but you can configure Boxstarter to store them elsewhere. The local repo is great for personal use but cant likely be accessed let alone discovered by others. The best way to share your package with others is to publish the package to a feed.

Package Publishing

There are multiple options when it comes to publishing your package. If you think that the package provides value to a broad range of users and include those outside of your organization, the feed is likely the best place. If fact this is where the TFS and SqlServer packages reside that our package will install. If the package likely only has value for yourself or your own organization, then a feed provider like works great. You can create one or more of your own feeds on Myget. These can even be private and require authentication which is desirable especially when there is sensitive information contained inside of your package.

I’m going to publish this package to a Boxstarter Community feed on  By default, Boxstarter will include this feed in the feeds it scans to find packages. Here is how we publish:

PS C:\> ."$env:ChocolateyInstall\ChocolateyInstall\nuget" push C:\Users\Matt\App
Data\Roaming\Boxstarter\BuildPackages\TfsServerWithBuild.1.0.0.nupkg <My Own KEY> -Source
Pushing TfsServerWithBuild 1.0.0 to '
Your package was pushed.
PS C:\>

Note that as with any Nuget based package feed, you always push using an API key that identifies you as the publisher. You can sign up for a free personal account at Myget and do not have to pay for creating and publishing to feeds. Here we see our feed show up:image

Installing the Package

Finally we are ready to kick off our install. Here it goes:

PS C:\> Enable-BoxstarterVM -Provider azure -CloudServiceName MyTfsVMService -VM
Name tfs1 -Credential $cred -CheckpointName BareOS | Install-BoxstarterPackage -
PackageName TfsServerWithBuild
Boxstarter: Locating Azure VM tfs1...
Boxstarter: Installing WinRM Certificate
Boxstarter: Configuring local Powershell Remoting settings...

Powershell remoting is not enabled locally. Should Boxstarter enable powershell
Powershell remoting is not enabled locally. Should Boxstarter enable powershell
[Y] Yes  [N] No  [?] Help (default is "Y"):
Boxstarter: Enabling Powershell Remoting on local machine
Boxstarter: Testing remoting access on
Boxstarter: Creating Checkpoint BareOS for service MyTfsVMService VM tfs1 at


Here we see the beginning of the Boxstarter output. We are issuing two commands really – piping one to the other. The Enable-BoxstarterVM performs a VM specific implementation for finding the DNS name and WinRM port for connecting to the VM. It may also do some prep work to ensure that a connection can be made. In Azure’s case, this includes downloading the certificate from the VM and installing it into our root certificate store so that we can communicate with the VM using HTTPS, which is the protocol powershell remoting is using here.

VM Checkpoints

Also note that just before the install begins, a checkpoint is taken that we label “BareOS.” This is optional but convenient in the event something goes wrong with our package as a result of a mistake in our authoring. We can then Restore this checkpoint, fix the package and retry from the exact same state we had when we began without needing to wipe out and create a new VM. You will not find these Checkpoints in the Azure management portal. Boxstarter uses Azure Blob Snapshots to create an implementation of checkpoints similar to what you would find in Hyper-V or other VM technologies.

If the BareOS checkpoint already existed when we ran our command, instead of creating the checkpoint, Boxstarter would have restored it. So if we were to run the above command without any changes all over again, our VM would be restored to its original state first.

Boxstarter exposes some additional commands for listing, creating, restoring and deleting checkpoints. You can check out the Boxstarter Azure documentation for details.

ProTip #1: Substitute “HyperV“ for the “Azure” provider argument and remove the CloudServiceName argument and Boxstarter would look for a Hyper-V VM named tfs1 and provision it. With Hyper-V, Boxstarter may mount the VM’s VHD file to configure it for remote connectivity. That’s often not necessary.

You don’t think you could run this in a Hyper-V VM because you would need another Windows Server licence? Not true. You can get evaluation VHDs for free and they can legally be “reevaluated.” See my blog on the Boxstarter Hyper-V functionality that touches on this point and where you can find them.

Adding an Endpoint for port 8080

By default, TFS listens on port 8080 for requests to its web services. We need to provide an endpoint to our Azure Service that will forward all 8080 traffic to the same port on our VM. By default, when you create a new VM in Azure, it will automatically create endpoints for Remote Desktop and PowerShell remoting. Adding an endpoint is fairly straight forward. Here is the command we will use:

$vm = Get-AzureVM -ServiceName MyTfsVMService -Name tfs1
Add-AzureEndpoint -Name tfs -Protocol tcp -LocalPort 8080 -PublicPort 8080 -VM $vm | 

Lets check out or new TFS server

First lets take a look at the last bit of Boxstarter output:

Errors       : {}
ComputerName :
Completed    : True
FinishTime   : 1/26/2014 12:18:04 AM
StartTime    : 1/25/2014 11:47:21 PM

This is exactly what we want to see. Our installation completed with no errors. This means no exceptions were thrown and the final Exit Code was 0.

So lets see if we can create a new project in Visual Studio.

First we need to connect to our server:


You will be prompted for a user name and password. Provide the same credentials that you provided earlier when creating the VM admin account. Now lets create a new project:


Looks good so far. Now lets go to the web portal and create a work item.


Now THAT is a work item.

On Premise Install (aka physical machine install)

Boxstarter can install anywhere. We just saw Boxstarter work on an Azure VM and I mentioned how to accomplish the same with Hyper-V.  As long as Powershell Remoting or at least remote WMI is enabled on a machine, the Boxstarer user has admin rights and its available on the network, Boxstarter can be used to provision any physical or virtual machine using Install-BoxstarterPackage:

Install-BoxstarterPackage -ComputerName -Credential $creds -PackageName MyPackage


If you are actually on the local machine, just as we did at the beginning of this post to install the Boxstarter modules, you can use the click-once launcher from IE or any Click-Once enabled browser (extensions exist for both Chrome and Firefox). If your default browser can run click-once apps, you can even launch the installer from a command line:


Released Boxstarter 2.3: Windows Azure integration by Matt Wrock

Just a month after releasing integration with Hyper-V, I am pleased to announce support for auto connection configuration and first class Checkpointing in Windows Azure VMs. Boxstarter can now make cloud provisioning of your boxes a straight forward process.

What does this do?

For readers unfamiliar with Boxstarter or Chocolatey, this makes the provisioning of Azure VM’s with all of the tools, applications, settings, patches, etc that you require a simple and repeatable event. If you spend much time building out VMs, tearing them down and then rebuilding them again and again, please stop immediately. Create a Chocolatey package to define your server’s end state and let Boxstarter push it out.

What I have just described can be accomplished on a local physical box, a remote physical box or a VM. Below I will describe the features in this release that apply specifically to the newly released Azure Integration.

Auto Connection Configuration

Boxstarter uses Powershell remoting to initiate provisioning. By default, Azure enables Powershell remoting on all of its Windows server VMs. However, there are a few not so obvious steps involved to make a connection to the VM. Boxstarter will locate the DNS name of the VM and the WinRM port listening for remoting connections. Boxstarter will also download the SSL certificate on the machine and install it locally. This allows Boxstarter to make a secure connection with the VM and invoke the provisioning of your server.


If you are familiar with almost any popular VM technology, you are certainly familiar with checkpointing (also known as Snapshots). This is the ability to save the state of the VM at a particular point in time and later restore to that point. Unfortunately Azure VMs do not easily expose this same functionality. You can create SysPrepped images in the portal, attach and detach disks, but there is no clear and simple way to take and restore a checkpoint let alone several checkpoints.

Boxstarter makes this possible by leveraging Azure Blob Snapshots under the hood and exposing this via four commands:

  • Set-AzureVMCheckpoint
  • Get-AzureVMCheckpoint
  • Restore-AzureVMCheckpoint
  • Remove-AzureVMCheckpoint

Each of these take a VM and CheckpointName parameter. Just like the Hyper-V integration, Boxstarter can create and restore checkpoints as part of the provisioning process since you may want to take or restore a checkpoint just before provisioning begins. For more information regarding their usage, please view their commandline help or visit Boxstarter.orgs Azure documentation page.

Case Study: Provision a public Minecraft server

You and your friends have decided to play homage to your local NFL team’s upcoming Superbowl challenge by constructing a replica of the team’s arena within a Minecraft landscape. You need a server that everyone can connect to and contribute as time permits. It’s a common cloud deployment scenario. We’ve all been there.

Step One: Get the Boxstarter.Azure module

The Boxstarter.Azure module does not install along with the core Boxstarter bits. To download and install it along with all of its dependencies, run:

CINST Boxstarter.Azure

Note: You will need to be running at least Powershell version 3. You can run:


at any powershell console to determine your version and if you are running a previous version, you can install Powershell version 3 or 4 via Chocolatey.

Now, for best results, open the Boxstarter Shell to run the rest of this sample.

Step Two: Configure your Azure subscription

There is a one time configuration step required so that the Azure Powershell commands know which account to manage and also know that you are authorized to manage it. This step includes running three commands:


This command will launch your default browser and initiate a Publisher Settings download. First you will land on the Windows Azure sign in page and as soon as you successfully authenticate, the download will begin.

Now simply import the file that was just downloaded:

Import-AzurePublishSettingsFile -PublishSettingsFile C:\Users\Matt\Downloads\Subscription-1-1-19-2014-credentials.publishsettings


Finally, specify the name of the storage account you want to use. You can run Get-AzureStorageAccount for a list of all of your storage accounts.

Set-AzureSubscription -SubscriptionName MySubscription -CurrentStorageAccountName MyStorageAccount

Note: Boxstarter will attempt to set your Current Storage Account for you if it has not been specified. However, you will need to run the command yourself if you need to run other Windows Azure Powershell commands prior to using Boxstarter.

Thats it. You can now use the Windows Azure Powershell and Boxstarter commands to provision VMs in Azure.

Step Three: Create the Azure VM

$cred=Get-Credential AzureAdmin
New-AzureQuickVM –ServiceName BoxstarterTest1 -Windows -Name MyVM `
  -ImageName 3a50f22b388a4ff7ab41029918570fa6__Windows-Server-2012-Essentials-20131217-enus `
  -Password $cred.GetNetworkCredential().Password -AdminUsername $cred.UserName `
  -Location "West US" –WaitForBoot

This uses the Azure Powershell module to create a new Cloud Service and a new small VM with Server 2012 R2 in Azure’s West US data center which just so happens to reside relatively near your football team. How convenient. Ehh…maybe not. But it is a pretty neat coincidence. Note that if you are reusing an existing Cloud service in the command above, you want to omit the –Location argument since the location of the existing service will be used.

Step Four: Create your Chocolatey package

Getting a Minecraft server up and running is really pretty simple. Here is the script we will use:

CINST Bukkit
Install-WindowsUpdate –AcceptEula
New-NetFirewallRule -DisplayName "Minecraft" -Direction Inbound -LocalPort 25565 -Protocol TCP
-Action Allow
Invoke-WmiMethod Win32_Process Create -Args "$env:systemdrive\tools\bukkit\Bukkit.bat"

This installs Bukkit, a popular Minecraft server management software which will also install the Java runtime. It will install all critical Windows updates. Then we allow inbound traffic to port 25565, the default port used by Minecraft servers. We will save this in a Gist and use the RAW gist url as our package source. The Url for the gist is:

Step Five: Use Boxstarter to provision the server

$cred=Get-Credential AzureAdmin
Enable-BoxstarterVM -provider Azure -CloudServiceName BoxstarterTest1 `
  -VMName MyVM -Credential $cred –CheckpointName Fresh | 
    Install-BoxstarterPackage `

This creates a connection to the VM and runs the installation script on that server. This may take a little time and is likely to include at least one reboot.

Step Six: Create a new Azure endpoint for the Minecraft port

$vm = Get-AzureVM -ServiceName BoxstarterTest1 -Name MyVM
Add-AzureEndpoint -Name Minecraft -Protocol tcp -LocalPort 25565 -PublicPort 25565 -VM $vm | 

This is necessary so that traffic can be properly routed to our server.

Thats it, fire up Minecraft and connect to our server.


Other new features worth mentioning

Enable-MicrosoftUpdate and Disable-MicrosoftUpdate

The credit to these new functions goes to Gary Ewan Park (@gep13) who contributed both. Thanks Gary!! This adds the ability for windows to update many Microsoft products beyond just windows. It essentially toggles this:



The Boxstarter Shell

Especially if you are not comfortable with Windows PowerShell, you may prefer to use the Boxstarter Shell to run Boxstarter commands. The Boxstarter Shell will make sure that the user is running with administrative privileges, the execution policy is compatible and all Boxstarter PowerShell modules are loaded and accessible. This shell also prints some basic "Getting Started" text at startup to assist you in running your first commands.