Accessing chef node attributes from kitchen tests by Matt Wrock

This should not happen often and maybe never, but there have been a few occasions where I needed the attribute values stored in the "node under test" to verify my kitchen test verifications. For example, maybe your recipe uses the node's IP or machine names as part of writing a config file and your kitchen driver uses dhcp to obtain an IP and auto generates unique machine names. In this case, you cant know the IP or machine name at the time you are authoring your test to check that the correct text was used in the config file.

This post demonstrates a very small and simple technique to retrieve this data inside of the test in order to dynamically verify the correct values were used in your recipe that you are testing.

At CenturyLink Cloud we use this in a few serverspec tests. One place we use this is in testing our haproxy configuration. Some values we need to inject into the configuration include the subnet of the current node and its chef environment which we include in the server names. Depending on where the kitchen test is being run, a different environment may be used so we cant be sure what the names of the subnet or chef environment will be while writing the test.

We solve this by adding a recipe to the end of our kitchen run list called "export-node":

  - name: my-suite
      - recipe[cookbook-under-test]
      - recipe[export-node]

This is a very simple recipe that simply converts the current node's attributes to json and writes it to a known location in /tmp/kitchen on the node.

ruby_block "Save node attributes" do
  block do
    if Dir::exist?('/tmp/kitchen')
      IO.write("/tmp/kitchen/chef_node.json", node.to_json)

Then our test can read that file and parse its json to a hash to be used throughout the test.

require 'json'
require 'serverspec'

set :backend, :exec

describe file('/etc/haproxy/haproxy.cfg') do
  let(:node) { JSON.parse('/tmp/kitchen/chef_node.json')) }
  let(:subnet) {
    ip = node["automatic"]["ipaddress"]
  let(:env) { node["chef_environment"].upcase}

  it { should be_a_file }
  its(:content) {
    should match <<-EOS
      backend elasticsearch_backend
        mode http
        balance roundrobin
        server #{env}SRCH01 #{subnet}.121:92 weight 1 check port 92
        server #{env}SRCH02 #{subnet}.122:92 weight 1 check port 92
        server #{env}SRCH03 #{subnet}.123:92 weight 1 check port 92

      backend web_backend
        mode http
        balance roundrobin
        timeout server 5m
        server #{env}WEB01 #{subnet}.131:80 weight 1 check port 80
        server #{env}WEB02 #{subnet}.132:80 weight 1 check port 80

      backend rabbit_backend
        mode http
        balance roundrobin
        server #{env}RABBIT01 #{subnet}.141:1567 weight 1 check port 1567
        server #{env}RABBIT02 #{subnet}.142:1567 weight 1 check port 1567
        server #{env}RABBIT03 #{subnet}.143:1567 weight 1 check port 1567

Above is a serverspec test that parses the node json to a hash. That hash is then accessible to our it blocks.

I have extracted the handful of lines in the export-node recipe to its own cookbook so you can grab it here.

Windows containers - package your apps and bootstrap your chef nodes with! by Matt Wrock

I discovered a few months ago from some comment on twitter. They claimed to be bringing containers to windows. Not the ability to run linux containers on windows but actual windows containers. This perked my interest and I immediately started hacking. The technology was impressive. Creating and running these containers have many similarities to workflows I'm familiar with on docker. The file system and windows registry inside these containers are truly isolated and you can layer spoon images on top of each other as part of the container packaging process.

Despite these cool features, there were some gaps that limited their usability for my scenarios. The winsxs (god bless it) was not entirely isolated limiting MSI installations and many windows features available via server manager and DISM did not fully work in isolation. I totally respect the fact that this is not an easy problem to solve.

I've kept in touch with Kenji Obata, CEO and founder of Spoon who has been filling me in on the MSI support. Being able to isolate MSI installs is key and the team at spoon has been making lots of headway making this work. This week my daughter asked me to build a minecraft server and I thought this would be a great opportunity to play with spoon to see where it has come. Running a minecraft server is fairly straight forward. You need to install the java runtime, which installs as an MSI, and then invoke java.exe to run the minecraft server .jar which starts the process running the minecraft server. The minecraft server listens for TCP requests on a specified port to connect players with game activity.

I was not only able to create a container running a minecraft server, but was later able to create an empty container and bootstrap it as a chef node then let chef install java and the minecraft jar...and then a second container running a separate instance.

This post is going to walk you through

  1. How to package an application inside a spoon windows container.
  2. Create a base container connectable via SSH
  3. Bootstrap containers joining them to a hosted chef server

I'l be using the minecraft server as the example app because minecraft is cool. This is gonna be a whirl wind tour where we will use several popular "devops" style apps to interact with our windows containers including vagrant, chocolatey, and chef. If you would like to follow along, I am demonstrating all of this using a windows server 2012 R2 vagrant box as the container host. This box is publicly available from vagrant cloud:

vagrant init mwrock/Windows2012R2

Its the same minimal windows box described here a few weeks ago.

Getting and using spoon

Just like you need to install the docker engine to run docker containers, you need to install the spoon plugin on a host system where you intend to run spoon containers. You can download this from Lets install this on our vagrant vm now from a powershell console:

Invoke-WebRequest -Uri `
  -OutFile spoon-plugin.exe

You will need to open a new console for the changes to your path to take effect. Next you need to sign in to spoon. So make sure to sign up for a free account first. I'm now gonna sign in to the spoon CLI:

spoon login mwrockx

Now we are ready to interact with the spoon CLI, creating and running containers.

Containerizing a windows based Minecraft server

There are a few ways to containerize an app in spoon. All of these are well documented on the spoon documentation site. One way is to start with a "clean" image or a collection of base images that have already been created and have components required by your app. Then manually install and configure the rest of your application and finally commit the image. All containers are created from an image. Think of the "image" as a single snapshot that holds an app setup and configuration in a pristine state. One then "runs" containers from that image. Containers are the isolated environment actually running the application.

Since minecraft requires the java runtime and there is a oracle/jre spoon image, that's a great place to start the packaging. So we'll run the java spoon image:

PS C:\Users\Administrator> spoon run oracle/jre
Using VM 11.6.381 from local
Using jre from local
Using clean:4.0 from local
Running container 4ef7018d with visibility public (use `--private` for a private container)

Spoon just launched a new command window running inside of a new container that has java installed. See:

(4ef7018d) C:\Users\Administrator>echo %java_home%
C:\Program Files (x86)\java\jre7

(4ef7018d) C:\Users\Administrator>

Our java_home environment variable is set here. We won't see this variable in our host vm outside the container (assuming we have not installed java). Also note the hash visible at every prompt inside a container. This helps us to identify which container we are inside of. We can list all containers on the host with:

PS C:\Users\Administrator> spoon containers
ID        Images                              Command  Created                 Status   Visibility
--        ------                              -------  -------                 ------   ----------
4ef7018d  spoonbrew/clean:4.0,oracle/jre               11/30/2014 9:38:42 PM   Running  Public
c431ad42  spoonbrew/clean:4.0,freeSSHd:1.2.6           11/30/2014 12:05:16 PM  Running  Public
605f8124  spoonbrew/clean:4.0,freeSSHd:1.2.6           11/30/2014 3:11:20 AM   Running  Public

You'll see I am also running a couple of containers from a freeSSHd image. We'll get to those later. Now lets get the minecraft server .jar file. There is a different .jar for each minecraft version. I run 1.7.2 because that is the version compatible with all the minecraft mods my daughters run.

mkdir %programdata%\spoon
powershell -command wget -outfile $env:programdata\spoon\minecraft_server.jar

Great. We have all the bits we need. So now we just need to kick off the server:

(4ef7018d) C:\Users\Administrator>"%java_home%\bin\java" -Xmx1024M -Xms1024M -ja
r %programdata%\spoon\minecraft_server.jar nogui
[22:01:33] [Server thread/INFO]: Starting minecraft server version 1.7.2
[22:01:33] [Server thread/INFO]: Loading properties
[22:01:33] [Server thread/WARN]: does not exist
[22:01:33] [Server thread/INFO]: Generating new properties file
[22:01:33] [Server thread/INFO]: Default game type: SURVIVAL
[22:01:33] [Server thread/INFO]: Generating keypair
[22:01:33] [Server thread/INFO]: Starting Minecraft server on *:25565
[22:01:34] [Server thread/WARN]: Failed to load operators list:
oundException: .\ops.txt (The system cannot find the file specified)
[22:01:34] [Server thread/WARN]: Failed to load white-list:
Exception: .\white-list.txt (The system cannot find the file specified)
[22:01:34] [Server thread/INFO]: Preparing level "world"
[22:01:34] [Server thread/INFO]: Preparing start region for level 0
[22:01:35] [Server thread/INFO]: Preparing spawn area: 10%
[22:01:36] [Server thread/INFO]: Preparing spawn area: 19%
[22:01:37] [Server thread/INFO]: Preparing spawn area: 29%
[22:01:38] [Server thread/INFO]: Preparing spawn area: 39%
[22:01:39] [Server thread/INFO]: Preparing spawn area: 49%
[22:01:40] [Server thread/INFO]: Preparing spawn area: 59%
[22:01:41] [Server thread/INFO]: Preparing spawn area: 67%
[22:01:42] [Server thread/INFO]: Preparing spawn area: 80%
[22:01:43] [Server thread/INFO]: Preparing spawn area: 92%
[22:01:44] [Server thread/INFO]: Done (10.273s)! For help, type "help" or "?"

Our minecraft sever is now up and running. Lets try to find the java process running from the host vm:

PS C:\Users\Administrator> ps -Name java | select path

C:\Program Files (x86)\java\jre7\bin\java.exe

You might be wondering why java is running from that path. Afterall, this java is supposed to be isolated to our container and not hanging out in our Program Files. So we sanity check with:

PS C:\Users\Administrator> Test-Path (ps -Name java | select path)

Hmm. Its not really there. Where is it then? Pop open the windows Task Manager and look at the details tab. Notice the "Image Path Name" of java.exe. Here we find the true path:


We can also use the spoon CLI to inspect the processes running in our containers:

PS C:\Users\Administrator> spoon ps
PID   Name      Container  User
---   ----      ---------  ----
2968  java.exe  4ef7018d   vagrant
4196  cmd.exe   4ef7018d   vagrant

We should also see this java process listening on port 25565:

PS C:\Users\Administrator> netstat -a -o | select-string "Listening" | select-string "2968"

  TCP          WIN-OHS0CTBB1TK:0      LISTENING       2968
  TCP    [::]:25565             WIN-OHS0CTBB1TK:0      LISTENING       2968

OK everything looks good and we should be able to run a minecraft client against our new server connected on that port. However, we are gonna want to stop the container before saving it as a new minecraft server spoon image. That means that any time I want to run a container off of the new image in the future, I'm going to have to make that same call to java.exe to launch the server. Wouldn't it be nice if that call was simply wrapped into the container startup? We can do that by building our image with a file.

Automating spoon image builds with a file

If you are used to docker, a is the moral equivalent of a docker file. Spoon provides a small collection of scripting primitives that can be used to automate the creation of an image. This exposes some nice functionality like the "startup file" command that allows us to run a command whenever the container starts up. This also allows us to easily repeat the creation of an image and version them in source control.

Lets blow away our container where we created our minecraft server:

PS C:\Users\Administrator> spoon rm 4ef7018d
Container 4ef7018d has been removed

Now lets create a file called with the following content:

from oracle/jre

mkdir %programdata%\spoon

cmd powershell -command wget -outfile $env:programdata\spoon\minecraft_server.jar

startup file ("c:\Program Files (x86)\java\jre7\bin\java", "-Xmx1024M", "-Xms1024M", "-jar", "c:\programdata\spoon\minecraft_server.jar", "nogui")

This script will pull down the java image, download the minecraft server jar and finally ensure that any time the image is run, the minecraft jar will be launched. Lets build our image:

spoon build -n=minecraft-server:1.7.2

We now have an image that we can run with:

spoon run minecraft-server

Note how fast these containers are created from our built image. Sure is faster than downloading and installing java.

Lets take our container for a spin and run a minecraft client. First we will need to open up port 25565 on the container host:

New-NetFirewallRule -DisplayName "Minecraft" `
                    -Direction Inbount –LocalPort 25565 `
                    -Protocol TCP -Action Allow

I'm gonna assume you know how to run a minecraft client and point it at a server. All you need to know is the host IP address of your container host.

Bootstrapping and building spoon containers with chef

So I was super curious to see if I could integrate a spoon container into a chef workflow. Chef is a popular configuration management platform for building and maintaining infrastructure. In my mind there are two huge use cases for chef/container interaction:

  1. Using chef instead of (or to supplement) a file. The chef ecosystem provides a rich set of resources just as capable of building a container image as a vm.
  2. Using containers to test chef recipes that will eventually be used in a vm or bare metal. As you have probably noticed, it takes much (MUCH) less time to start a container than a fresh vm.

Working with a chef server

Sure we could just use chef-zero/chef-solo to run a recipe locally, but lets try to actually perform a remote bootstrap of a container node against a serer to get the full effect here. We could use any server. Here, I'm using a hosted chef instance that I sometimes use for personal testing. Anyone can sign up for a hosted chef account giving you access to your own publicly available chef server endpoint.

See this post for a brief run through on getting started with hosted chef. I'm not going to get into the details of setting up a chef server here but you will need:

  • you user .pem
  • your organization validator.pem
  • a knife.rb config file

Knife windows

We are going to run knife bootstrap using the knife windows gem. If you have installed chef using the ChefDK, you should have this. Otherwise you will need to run:

gem install knife-windows

You could run this directly from the container host vm. I'm just going to run it from my local os that has network access to my vm.

A minecraft cookbook with windows support

I could not find a chef cookbook for a minecraft server that runs on windows. So I wrote one and it can be found on github where you can view or clone the full cookbook. Lets take a look at the default recipe (the only recipe):

include_recipe 'chocolatey'
chocolatey 'javaruntime'

directory "#{ENV['programdata']}/minecraft"

remote_file "#{ENV['programdata']}/minecraft/minecraft_server.jar" do
  source "{node['minecraft']['version']}/minecraft_server.#{node['minecraft']['version']}.jar"

cookbook_file "minecraft.bat" do
  path "#{ENV['programdata']}/minecraft/minecraft.bat"

batch "start minecraft" do
  code <<-EOH
    powershell -command "start #{ENV['programdata']}/minecraft/minecraft.bat"
  action :run

A couple things worth noting:

  1. It uses chocolatey to install java. The java community cookbook that most use requires the java MSI to be internally hosted for windows platforms. I didn't want to bother with that and I know there is a chocolatey package that will do this job just fine. For those unfamiliar with chocolatey, it is a package management solution for windows similar to brew, yum or apt-get on mac/linux.
  2. I store the call to java.exe in a cookbook file, minecraft.bat, and use the chef batch resource to invoke that file via powershell Start-Process so that it does not hang the chef client run because that batch file will not return.

Upload the cookbooks to the chef server

You will need to upload the above mentioned minecraft cookbook as well as its dependencies which include:

  • chef-handler
  • windows
  • chocolatey

All of these dependencies are on the chef supermarket. You can use berkshelf to easily download all dependencies. Again if you use the ChefDK, you have this already, otherwise install the berkshelf gem. Using the berkshelf vendor command you can grab all dependencies and save them all to a "vendor path" you specify:

berks vendor [vendor path]

Make sure your cookbook_path in your knife.rb includes the vendor path otherwise knife upload will not find them.

Establishing SSH connectivity with your containers

Currently spoon containers do not play nice with winrm and cant isolate winrm to specific containers. Spoon is aware of this and its on their radar. However, ssh works just fine and you can remotely manage your containers as long as they have an ssh server running on the container.

I have created a spoon image that runs freeSSHd. This will be the base container we will use to bootstrap with chef. You can go ahead and start up an ssh enabled container with:

spoon run -d mwrockx/freeSSHd

The -d tells spoon to run the container in the background. Note that if you do not already have this image locally (you likely don't) then spoon will download it from the public spoon hub. Now here is another cool thing about containers. Run:

spoon images

to list the images locally stored on your container host. Take a look at the size of this freeSSHd image. Its 3.7MB. That is much better than 10+GB that most windows VMs consume.

If you will be accessing the containers outside of the container host VM, you will need to allow port 22 traffic on the windows firewall:

New-NetFirewallRule -DisplayName "SSH" `
                    -Direction Inbount –LocalPort 22 `
                    -Protocol TCP -Action Allow

I built this image giving both the Administrator and the vagrant users SSH access. Once the container is running, you can edit this in the freeSSHd settings by right clicking the desktop tray icon.

I'm assuming that you have access to an ssh client. I use the one that installs with msysgit located at C:\Program Files (x86)\git\bin\ssh.exe by default. Lets ssh to our container just to verify that everything works:

Yup. Looks good and the hash in the prompt correlates with my container id. 

One oddity that I noticed when using ssh is that upon succesfully connecting, the ssh prompt appears at the top of the console in the midst of whatever text was already there. I'm not sure why this happens but find that calling cls to clear the screen before beginning my ssh session helps.

Bootstrap the container

ok. Enough yaking. Lets install a minecraft server with chef on a windows container:

knife bootstrap windows ssh -P vagrant -x vagrant -r 'recipe[minecraft]' -N mc1

This will install the chef client on our container, add the container as a chef node on the server, download the minecraft cookbook and dependencies from the chef server and run the recipes to install java, the minecraft server jar file and launch the server. I use the default vagrant credentials which uses 'vagrant' for both user name and password. This will likely take a few minutes. The longest part is the download of the chef client.

I also see the node in my hosted chef organization:

Bootstrapping a second node

So just to be fancy, lets bootstrap a second node.

Networking considerations

We will have two containers running ssh and minecraft on what THEY THINK are ports 22 and 25565, we will need to setup proper NAT rules that redirect different ports coming into the container host to these ports on the containers. Spoon provides mechanisms to accomplish this. We will have our second container route ports 2222 and 35565 to 22 and 25565 respectfully. Spoon provides lots of options for mapping ports and DNS as well as linking containers to one another. See the spoon documentation for those details.

So first step we need to take is allow ports 2222 and 35565 on the container host:

Get-NetFirewallPortFilter | ? { 
  $_.LocalPort -eq 22 
} | Set-NetFirewallPortFilter -LocalPort @(22,2222)
Get-NetFirewallPortFilter | ? { 
  $_.LocalPort -eq 25565 
} | Set-NetFirewallPortFilter -LocalPort @(25565,35565)

Now lets run our second container informing spoon of the desired routes:

PS C:\Users\Administrator> spoon run -d --route-add=2222:22 --route-add=35565:25565 freeSSHd
Using VM 11.6.381 from local
Using freeSSHd:1.2.6 from local
Using clean:4.0 from local
Running container 9a3b62af with visibility public (use `--private` for a private container)

Just to confirm that we can ssh to the new container on port 2222:

Microsoft Windows [Version 6.3.9600]master]> ssh vagrant@ -p 2222
(c) 2013 Microsoft Corporation. All rights reserved.

(9a3b62af) C:\Users\Administrator>

Bootstrapping the second container

We are now ready to join the new container to the chef server and install the minecraft server recipe:

knife bootstrap windows ssh -P vagrant -x vagrant -r 'recipe[minecraft]' -N mc2 -p 2222

Now I can see 2 nodes on the hosted server instance:

Once a node is joined to the server, we can rerun the recipes locally using chef-client. No need to bootstrap again. Lets change one of the nodes to run a different version of minecraft. We'll change the mc2 node to run 1.7.9:

Now we'll ssh to mc2 and reconverge it with the new version:

Running a minecraft client pointed to both containers, we can see the version mismatch on the second node.

Diffing containers

Here is a feature I find really clever - the ability to see how a container differs from the host. Spoon can list all files and registry keys that have changed:

PS C:\Users\Administrator> spoon run -d freeSSHd
Using VM 11.6.381 from local
Using freeSSHd:1.2.6 from local
Using clean:4.0 from local
Running container de8d95f9 with visibility public (use `--private` for a private container)
PS C:\Users\Administrator> spoon diff de8d95f9
Using clean:4.0 from local
Using freeSSHd:1.2.6 from local
File system changes:
C C:\Program Files (x86)
C C:\Program Files (x86)\freeSSHd
C C:\Program Files (x86)\freeSSHd\FreeSSHdService.ini
C C:\ProgramData
C C:\ProgramData\Microsoft
C C:\ProgramData\Microsoft\Windows
C C:\ProgramData\Microsoft\Windows\Caches
C C:\ProgramData\Microsoft\Windows\Caches\{6AF0698E-D558-4F6E-9B3C-3716689AF493}.2.ver0x0000000000000006.db
C C:\ProgramData\Microsoft\Windows\Caches\{8290F723-67BF-4FDA-BEEA-B0AF3ADC7010}.2.ver0x0000000000000004.db
C C:\ProgramData\Microsoft\Windows\Caches\{DDF571F2-BE98-426D-8288-1A9A39C3FDA2}.2.ver0x0000000000000004.db
C C:\ProgramData\Microsoft\Windows\Caches\cversions.2.db
C C:\Users\Administrator\AppData\Local
C C:\Users\Administrator\AppData\Local\Microsoft
C C:\Users\Administrator\AppData\Local\Microsoft\Windows
A C:\Users\Administrator\AppData\Local\Microsoft\Windows\INetCache
A C:\Users\Administrator\AppData\Local\Microsoft\Windows\INetCache\counters.dat
C C:\Users\Administrator\AppData\Local\temp
A C:\Users\Administrator\AppData\Local\temp\wodCmdTerm.exe

Registry changes:
C @HKCU@\Software
C @HKCU@\Software\Microsoft
C @HKCU@\Software\Microsoft\Windows
C @HKCU@\Software\Microsoft\Windows\CurrentVersion
C @HKCU@\Software\Microsoft\Windows\CurrentVersion\Explorer
A @HKCU@\Software\Microsoft\Windows\CurrentVersion\Explorer\SessionInfo
A @HKCU@\Software\Microsoft\Windows\CurrentVersion\Explorer\SessionInfo\2
C @HKCU@\Software\Microsoft\Windows\CurrentVersion\Internet Settings
A @HKCU@\Software\Microsoft\Windows\CurrentVersion\Internet Settings\5.0
A @HKCU@\Software\Microsoft\Windows\CurrentVersion\Internet Settings\5.0\Cache
A @HKCU@\Software\Microsoft\Windows\CurrentVersion\Internet Settings\5.0\Cache\Content
A @HKCU@\Software\Microsoft\Windows\CurrentVersion\Internet Settings\5.0\Cache\Content\CachePrefix
A @HKCU@\Software\Microsoft\Windows\CurrentVersion\Internet Settings\5.0\Cache\Cookies
A @HKCU@\Software\Microsoft\Windows\CurrentVersion\Internet Settings\5.0\Cache\Cookies\CachePrefix
A @HKCU@\Software\Microsoft\Windows\CurrentVersion\Internet Settings\5.0\Cache\History
A @HKCU@\Software\Microsoft\Windows\CurrentVersion\Internet Settings\5.0\Cache\History\CachePrefix
A @HKCU@\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Connections
A @HKCU@\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Connections\SavedLegacySettings
A @HKCU@\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ProxyEnable
C @HKCU@\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap
C @HKCU@\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap\AutoDetect
C @HKCU@\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap\IntranetName
C @HKCU@\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap\ProxyBypass
C @HKCU@\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap\UNCAsIntranet
A @HKCU@\Software\Microsoft\Windows\CurrentVersion\OnDemandInterfaceCache
C @HKLM@\SOFTWARE\Wow6432Node\Microsoft
C @HKLM@\SOFTWARE\Wow6432Node\Microsoft\Windows
C @HKLM@\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion
A @HKLM@\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Internet Settings
A @HKLM@\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap
C @HKLM@\System
C @HKLM@\System\CurrentControlSet
A @HKLM@\System\CurrentControlSet\Services
A @HKLM@\System\CurrentControlSet\Services\Winsock
A @HKLM@\System\CurrentControlSet\Services\Winsock\Parameters
A @HKLM@\System\CurrentControlSet\Services\WinSock2
A @HKLM@\System\CurrentControlSet\Services\WinSock2\Parameters
A @HKLM@\System\CurrentControlSet\Services\WinSock2\Parameters\AppId_Catalog
A @HKLM@\System\CurrentControlSet\Services\WinSock2\Parameters\NameSpace_Catalog5
A @HKLM@\System\CurrentControlSet\Services\WinSock2\Parameters\Protocol_Catalog9

This can come in handy for all sorts of debugging scenarios where you need to track down changes impacted by an individual install.

Are we living with rainbows and unicorns?

I'm not quite ready to confirm that but I do feel like I just got extra sprinkles added to my ice cream. There is still much to be done to fill in some isolation gaps and provide integration and orchestration tooling.

There is incredible potential here that can be taken advantage of today. Things are going to be very interesting once Microsoft reveals more details around its own container strategy and partnership with docker. However, so far that story has been very foggy and its unclear exactly what they will deliver in the next server edition. Likely many will simply wait and see what emerges there, but I think spoon has made some great strides forward here and deserves a close look for its potential as a serious player in the windows container space.

Managing team dev and build environments with vagrant and chef by Matt Wrock

This week the CenturyLink Cloud blog published a post I wrote that covered the use of vagrant and chef on the Infrastructure Automation team where I work. While we use these tools to build out infrastructure for our public cloud data centers, the post focuses on how we use them to build out personal development environments as well as building out the build agents that run our tests.

If this interests you, hop on over to the CenturyLink Cloud blog and give it a read.

I especially like how we use the same Chef cookbooks that provision our personal vagrant environments to provision our build agents. Anything one can do to make a dev box as close to the environments where tests are run only stands to increase test reliability.

Deannoyafying a default windows server install by Matt Wrock

This is a somewhat opinionated follow up to last week’s post on how to create a windows vagrant box. What I left out were a few settings you might want to include to prevent you from going absolutely bonkers. While many may argue that it is too late for myself, one goal of this post is to keep others from my own fate.

I’m just going to cover three configuration settings here and one of them applies to windows client installations, not just server SKUs. There are other improvements one can make for sure but these are easily up at the top.

IE Enhanced Security Configuration – turn it off

Sure, the Amarican Society of therapists and social workers does not want me to tell you this. But trust me on this one because both you and those around you will benefit. Unless you belong to the small subculture of technology workers that enjoy pointing and clicking several times prior to previewing any single web page, add this to your windows base images:

Write-Host "Disabling IE Enhanced Security Configuration (ESC)."
$key = "HKLM:\SOFTWARE\Microsoft\Active Setup\Installed Components"
$AdminKey = "$key\{A509B1A7-37EF-4b3f-8CFC-4F3A74704073}"
$UserKey = "$key\{A509B1A8-37EF-4b3f-8CFC-4F3A74704073}"
if(Test-Path $AdminKey){    
  Set-ItemProperty -Path $AdminKey -Name "IsInstalled" -Value 0    
if(Test-Path $UserKey) {    
  Set-ItemProperty -Path $UserKey -Name "IsInstalled" -Value 0    

This is a script I use generically on many windows machines and it will simply do nothing on client SKUs since it checks for the registry locations.

Do not open server manager at logon

Don’t worry everyone. I assure you that you can still manage your server even without server manager. This is the dashboardy looking app that comes up on windows server immediately after logon. Its one redeeming feature is that it exposes a way to turn off IE Enhanced Security Configuration. Here is an excerpt from our CenturyLink Cloud default windows Chef recipe that turns this off:

# Disable the ServerManager.
registry_key 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ServerManager' do
  values [{    
    :name => 'DoNotOpenServerManagerAtLogon',    
    :type => :dword,    
    :data =>  0x1    

Of course you can do the same without Chef with this bit of powershell:

Write-Host "Disabling Server Manager opening at logon."
$Key = "HKLM:\SOFTWARE\Microsoft\ServerManager"
if(Test-Path $Key){  
  Set-ItemProperty -Path $Key -Name "DoNotOpenServerManagerAtLogon" -Value 1

Sensible windows explorer settings

This tends to drive me nuts. I can’t tell .bat files from .cmd files. I cant find my ProgramData folder. I’m initially excited that my 100TB page file has disappeared then crushed to discover its there and I just don’t see it. The world is not well. This script makes it right:

$key = 'HKCU:\Software\Microsoft\Windows\CurrentVersion\Explorer\Advanced'
if(Test-Path -Path $Key) {  
  Set-ItemProperty $Key Hidden 1  
  Set-ItemProperty $Key HideFileExt 0  
  Set-ItemProperty $Key ShowSuperHidden 1
  Stop-Process -Path explorer

Who would NOT want to show Super hidden files?

Adding a couple light weight apps for extra sanity

There are a couple other things that I find unacceptably annoying but there is no simple configuration fix for these. Both mentioned here can be solved with a small install:

A tears free command line console

Windows 10 (in technical preview as of this post) is finally fixing some of this but in the meantime I use console that supports key mappings for human consumables copy/paste shortcuts and tabbed consoles since it is not uncommon for me to have half a dozen command lines open. It works with both powershell (site staff recomended) as well as the traditional command line. Many others really like the ConEmu console emulator.

Text editor that understands how lines end

When it comes to lite text editing, notepad is not so entirely bad in a 100% windows universe. However these days I work in a mixed environment and transfer bootstrap scripts from a linux box. If I need to inspect those files after they have made their way to a windows server, I'll often be faced with a one line file. While reducing the prior logic to a single line is impressive, the end result may mean having to use Wordpad to view the files. Is that wrong? Yes...yes it is. Text editors can be a very personal decision. I prefer sublime text 3 on which I am happy to spend the 50 bucks or whatever it was I spent, but there are lots of good free options available.

A chocolatey package to automate away the annoying

I have wrapped all of the three config changes above into a chocolatey package win-no-annoy. You can check out the complete powershell script that it will run here. If you are working with a newly provisioned machine or any machine that does not have chocolatey installed, use boxstarter to run the script which will also install chocolatey. Assuming IE is still the default browser, run the following command:


I hope your windows server experience will be much less annoying with these improvements made.

In search of a light weight windows vagrant box by Matt Wrock

Update 2: See this new post that includes everything here and also automates everything via Packer and Boxstarter.

Update: More on packaging windows 10 Hyper-V and comparison of LZMA vs. GZIP compression can be found in this more recent post. My conclusion is that gzip is bigger but better.

Windows... No getting around it - its a beast compared to the size of its linux step sibling. When I hear colleagues complaining about pulling down a couple hundred MB to get their linux vagrant box up and running, I feel like a third world scavenger of disk space while they whine about their first world 100mb problems. While this post does not solve this problem, it does provide a way to deliver a reasonably sized windows box weighing in under 3GB.

This post will explain how to:

  • Obtain a free evaluation server 2012 R2 ISO
  • Prepare a minimally sized VM image
  • Package the VM for use via vagrant both in VirtualBox and Hyper-V
  • Allow the box to be accessed by others over the web. The world wide web.

I may follow up with a more automated approach but here I'll be walking through the process fairly manually. However my instructions will largely be command line samples so one should be able to cobble together a script that performs all of this. I will be doing just that.

Downloading an evaluation ISO

The Technet evaluation center provides free evaluation copies of the latest operating systems. As of this post, the available operating systems are:

  • Server 2012R2
  • Server 2012
  • Server 2008 R2
  • Windows 8.1
  • Windows 8
  • Windows 7

These are fully functional copies with a limited lifetime. I believe the client SKUs are 90 days and the server SKUs are 180 days. Server 2012 R2 is definitely 180 days. Once the evaluation period expires, you can download a new evaluation and the trial period begins anew. Some may say like a new spring. Others will choose to describe it differently but either way it is perfectly legit.

The evaluation center provides several formats for download including ISO, VHD, and Azure VM. I prefer to download the ISO since it is much smaller than the VHD and it can be used to create both a Hyper-V and VirtualBox VM.

Create the VM

I am assuming here that you are familiar with how to do this on your preferred hypervisor platform. However I recommend that you choose the following file formats:

VHD (gen 1) on Hyper-V

Unless you know that this vagrant box will only be used on windows 8.1 or server 2012 R2 hosts, a generation 2 .vhdx format will not work on older OS versions. Chances are that most vagrant use cases will not be taking advantage of the generation 2 features anyways.

.vmdk on Virtual Box

The reason for this will become more apparent when we get to packaging the VM. The short answer here is that this is going to put you in a better position for a more optimally compressed (likely much more) box file.

Note: You cannot run Hyper-V and VirtualBox on the same machine. You can create second boot record, but I have had too many bad experiences uninstalling VirtualBox on windows and ending up with a corrupted network stack. So I am using Hyper-V on my windows box and VirtualBox on my Ubuntu machine.

When the ISO installation begins, you will be given a choice of Standard or DataCenter editions as well as whether to install with the GUI or not. I am installing Server 2012 R2 Standard with GUI.

Preparing the image

Here we want to ensure our VM can interact with vagrant with as little friction as possible and be as small as possible.

Configuring the vagrant user

By default, vagrant handles all authentication with a user named 'vagrant' and password 'vagrant'. So we will rename the built-in administrator to 'vagrant' and change the password to 'vagrant'.

First thing that needs to happen here is to turn off the password complexity policy that is enabled by default on server SKUs. To do this:

  1. Run gpedit from a command prompt
  2. Navigate to Computer Configuration -> Windows Settings -> Security Settings -> Account Policies -> Password Policy
  3. Select 'Password must meet complexity requirements' and disable

Good riddens complexity!

Now we jump to powershell and rename the administrator account and change its password to vagrant:

$admin.UserFlags.value = $admin.UserFlags.value -bor 0x10000

This may be obvious but that second to the last line instructs the password to never expire. Its a good idea to reboot at this point since you are currently logged in as this user and things can and will get weird eventually if you don't.

Configure WinRM

While server 2012 R2 will have powershell remoting automatically enabled, we need to slightly tweak the winrm configuration to play nice with vagrant's ruby flavored winrm implementation:

winrm set winrm/config/client/auth '@{Basic="true"}'
winrm set winrm/config/service/auth '@{Basic="true"}'
winrm set winrm/config/service '@{AllowUnencrypted="true"}'

Allow Remote Desktop connections

Vagrant users may use the vagrant rdp command to access a vagrant vm so we want to make sure this is turned on:

$obj = Get-WmiObject -Class "Win32_TerminalServiceSetting" -Namespace root\cimv2\terminalservices

Enable CredSSP (2nd hop) authentication

Users may want to use powershell remoting to access this server and need to authenticate to other network resources. Particularly on a vagrant box, this may be necessary to access synced folders as a network share. In order to do this in a powershell remoting session, CredSSP authentication must be enabled on both sides. So we will enable it on the server:

Enable-WSManCredSSP -Force -Role Server

Enable firewall rules for RDP and winrm outside of subnet

Server SKUs will not allow remote desktop connections by default and non domain users making winrm connections will be refused if they are outside of the local subnet. So lets turn that on:

Set-NetFirewallRule -Name WINRM-HTTP-In-TCP-PUBLIC -RemoteAddress Any
Set-NetFirewallRule -Name RemoteDesktop-UserMode-In-TCP -Enabled True

Shrink PageFile

The page file can sometimes be the largest file on disk with the least amount of usage. It all depends on your configuration but lets trim ours down to half a gb.

$System = GWMI Win32_ComputerSystem -EnableAllPrivileges
$System.AutomaticManagedPagefile = $False

$CurrentPageFile = gwmi -query "select * from Win32_PageFileSetting where name='c:\\pagefile.sys'"
$CurrentPageFile.InitialSize = 512
$CurrentPageFile.MaximumSize = 512

You will need to reboot to see the page file change size but you can continue on without issue and reboot later.

Change the powershell execution policy

We want to make sure users of the vagrant box can run powershell scripts without any friction. We'll assume that all scripts are safe within the blissful confines of our vagrant environment:

Set-ExecutionPolicy -ExecutionPolicy Unrestricted

Unrestricted. To some its an execution policy. Others - a lifestyle.

Install Important Windows Updates

Not gonna bore you with the details here. Simply install all important windows updates (dont bother with optional updates), reboot and repeat until you get the final green check mark indicating you have gotten them all.

Cleanup WinSXS update debris

Now that we have installed those updates there are gigabytes (not many but enough) of backup and rollback files lying on disk that we dont care about. We are not concerned with uninstalling any of the updates. New in windows 8.1/2012 R2 (and back ported to win 7/2008R2) there is a command that will get rid of all of this unneeded data:

Dism.exe /online /Cleanup-Image /StartComponentCleanup /ResetBase

This may take several minutes, but if you can make it through the updates, you can make it through this.

Additional disk cleanup

On windows client SKUs you have probably used the disk cleanup tool to remove temp files, error reports, web cache files, etc. This tool is not available by default on server SKUs but you can turn it on. We will go ahead and do that:

Add-WindowsFeature -Name Desktop-Experience

This will require a reboot to take effect, but once that completes, the "Disk Cleanup" button should be available on the property page from the root of our C: drive or we can launch from a command line:

C:\Windows\System32\cleanmgr.exe /d c:

Go ahead and check all the boxes. Chances are this wont get rid of much on this young strapping box but might as well jettison all of this stuff.

Features on Demand

One new feature in server 2012 R2 is the ability to remove features not being used. Every windows installation regardless of the features enabled (IIS, Active Directory, Telnet, etc) has almost all of these features available on disk. However the majority will never be used and there is alot of space to be saved here by removing them.

Its really up to your own discretion as to what features you will need. In this post we are going to remove practically everything other than the bare essentials for running a GUI based server OS. If you plan to use the box for web work, you may want to go ahead and enable IIS and any related web features you think you will need now. Once a feature is removed, some features are easier to restore than others. For instance, the telnet-client feature will just be downloaded from windows update server if you choose to add it later. However, if you need to re-add the desktop-experience feature, you will need installation media handy and must point to a source .wim file. I really don't know what the logic is that determines what can be downloaded from the public windows update service and what cannot but its good to be aware of this.

First we will remove (not uninstall yet) the features that are currently enabled that we do not need:

  'Powershell-ISE') | Remove-WindowsFeature

Now we can iterate over every feature that is not installed but 'Available' and physically uninstall them from disk:

Get-WindowsFeature | 
? { $_.InstallState -eq 'Available' } | 
Uninstall-WindowsFeature -Remove

At this point its a good idea to reboot the vm and then rerun the above command just to make sure you got everything. For some reason I find that the InkAndHandwritingServices feature is still lying around. A fantastic feature I have no doubt but trust me, you don't need it.

Defragment the drive

Now we will defrag and possibly retrim the drive using the powershell command:

Optimize-Volume -DriveLetter C

Purge any hidden data

Its possible there is unallocated space on disk with data remaining that can hinder the ability to compact the disk and also limit the compression algorithm's ability to fully optimize the size of the final box file. There is a sysinternals tool, sdelete, that will "0 out" this unused space:

wget -OutFile
[System.IO.Compression.ZipFile]::ExtractToDirectory("", ".") 
./sdelete.exe -z c:



Packaging the box

At this point you should hopefully have a total disk size of roughly 7.5GB. For a windows server, thats pretty good. We are now ready to package up our vm into a vagrant .box file that can be consumed by others. The process for doing so is going to be different on Hyper-V and Virtualbox so I will cover both separately.


First thing, compact the vhd:

Optimize-VHD -Path C:\path\to\my.vhd -Mode Full

On my laptop, the VHD just shrank from 13 to 8 GB.

Next we will export the VM:

Export-VM -Name win2012R2 -Path C:\dev\vagrant\Win2k12R2\test

Note that my VM name is win2012R2 and after this command completes, I should have a directory named c:\dev\vagrant\Win2k12R2\test\win2012R2 with three subdirectories:

  • Snapshots
  • Virtual Hard Disks
  • Virtual Machines

We can delete the Snapshots directory:

Remove-Item C:\dev\vagrant\Win2k12R2\test\win2012R2\Snapshots -Recurse -Force

Now there are two text files that we want to create in the root c:\dev\vagrant\Win2k12R2\test\win2012R2 directory. The first is a small bit of JSON:

  "provider": "hyperv"

Save this as metadata.json. Lastly, and this is optional, we will create an initial Vagrantfile:

# -*- mode: ruby -*-
# vi: set ft=ruby :


Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.guest = :windows
  config.vm.communicator = "winrm"

By adding this to the Box, it ensures that a user can run vagrant up without any initial vagrant file, and have a succesful windows experience. However, it is customary for most vagrant projects to include these settings in their project Vagrantfile in which case having this file in the box file is redundant.

We are now ready to create the final box file. This is expected to be a .tar archive with a .box extension. The name (before the extension) doesn't matter but keep it short with no spaces. I found that not just any tar archiver will work on windows and this can be a big stumbling point on a windows system where tarring is not "normal" and you may not even have tar.exe or bsdtar.exe installed.

A note of caution: You obviously have vagrant installed and vagrant comes with a version of bsdtar.exe compiled for the windows platform. However, as of version 1.6.5, this executable will not work on tar files with a final size greater than 2GB. Sadly, ours will be greater than that. Also, if you have GIT installed, it too has a tar.exe installed with similar limitations. Many may assume they can create a .tar using the popular 7zip. Technically this is true but I have not been succesful in having the final .tar play nice with vagrant. I have had success with two available solutions:

  • The version of tar.exe that is available with cygwin. I'm not a big cygwin fan and this tar requires several command switches be properly set to produce an archive that will work with vagrant: tar --numeric-owner --group=0 --owner=0 -czvmf * is what worked for me. The options here ensure the ACLs are cleared out and the m option clears the timestamps.
  • (recommended) Nikhil Benesch has been kind enough to make available a version of bsdtar.exe that works great and can be downloaded here.

So assuming you have Nikhil's bsdtar on your path, run this inside the same directory that holds your exported Hyper-V directories and the two text files (metadata.json and Vagrantfile) you created:

bsdtar --lzma -cvf *

Very important: the --lzma switch instructs bsdtar to use the lzma compression algorithm. This is the same algorithm 7zip uses as its default. This can dramatically reduce the size of the final box file (like 40%) over gzip. It also dramatically increases the time (and CPU) bsdtar will take to create the file. So some patience is needed but in my opinion it is well worth it. This is a one time cost and it will NOT impact decompression time which is what really matters.


We will be following roughly the same process we used for Hyper-V with a couple deviations. Note that the instructions that follow differ from the guidance given in the vagrant documentation as well as most blogs. These instructions are optimized for a small file size. You can use the vagrant package command to produce virtualbox packages but that will produce a much larger file (3.9GB vs 2.75GB) than the steps that follow.

The first main difference is that we will not compact the .vmdk. Virtualbox can only compact its native .vdi files. This is ok since the final compression step will take care of this for us.

First step: export the virtualbox VM:

vboxmanage export win2012r2 -o box.ovf

Here win2012r2 is the name of the vm in the virtualbox GUI. It is important that the name of the outputted .ovf file is box.ovf. Do not use a different name.

Now for something not so intuitive: delete the exported .vmdk file that was created. It is dead to us. Regardless of the initial file type you use to create a virtualbox vm, the export command will always convert the drive to a compressed .vmdk. This compression is not ideal. What you want to do after deleting this file is copy the original .vmdk file of the vm to the same directory where your box.ovf file exists and rename it to the same name of the .vmdk you just deleted. This should be box-disk1.vmdk.

This is why we used a .vmdk (the native vmware format) in the first place. Later when this file is imported back to virtualbox by vagrant, it is going to expect a vmdk file. However, if we used the one compressed by the export command, we are stuck with the compression it yields. By using the original uncompressed .vmdk, we allow the archiver to use lzma and can get a MUCH smaller file.

Just as was shown with Hyper-V above, we want to add the same two text files to this export directory but their contents will be slightly different. The first one, metadata.json should contain:

  "provider": "virtualbox"

Next our Vagrantfile should look like:

# -*- mode: ruby -*-
# vi: set ft=ruby :

# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.guest = :windows
  config.vm.communicator = "winrm"
  config.vm.base_mac = "080027527704"
  config.vm.provider "virtualbox" do |v|
    v.gui = true

Your file should have one key difference. Make sure that the MAC address matches the one in the settings of your VM. I'm pretty sure its gonna be different. So you should now have four files in your export directory:

  • box.ovf
  • box-disk1.vmdk
  • metadata.json
  • Vagrantfile

Navigate to this directory. I'm on an ubuntu desktop so I'm just gonna use the native tar. If you are on a windows host. Follow the same instructions for the compression above for Hyper-V. On my linux box I will package using:

tar --lzma -cvf *

Again, the name of the box file other than the extension is not important. Also, this may take a long time. Maybe an hour. Its worth it. Its a one time cost and while much longer than producing a gzip archive, you should end up with a 2.75 GB file that will take just as long to import as the same 3.9GB file gzip would produce.

Its a win-win!

Do I have to create the box twice?

No. You can create the box in Virtual box and then clone to a .vhd based box. Then you can create a new Hyper-V VM and instead of creating a new drive, use the existing .vhd from your virtualbox vm. Just make sure your Hyper-V vm is a generation 1 vm. Thats probably the easiest way to do it. You could also create the vm in Hyper-V and then create a .vhd based virtualbox vm using your Hyper-V vm instead of creating a new one. Then copy the vhd to a vmdk based vm. Granted this is still not an ideal or seamless migration but it is one way to get around having to run through the entire box preparation twice and ensuring the two boxes are identical.

Testing your box

Before going to the trouble of uploading your box to the web and making it public, you will want to test vagrant up on the package you just created locally. This is just three commands:

vagrant box add test-box /path/to/
vagrant init test-box
vagrant up

This first places the box into your global vagrant cache. By default this is a folder named .vagrant.d in your home directory. You can move it by creating an environment variable VAGRANT_HOME that points to the directory where you want to keep it. I keep mine on a different drive.

Next, vagrant init creates a Vagrantfile in your current directory so that now any time you want to interact with this box, issuing vagrant commands from this directory will affect this box.

Vagrant up to Hyper-V

Finally vagrant up will create the box in your provider and start it. For Hyper-V, or any provider other than virtualbox, you need to either include a --provider option:

vagrant up --provider hyperv

or you can set an environment variable VAGRANT_DEFAULT_PROVIDER to hyperv.

Uploading your box to the world wide web

In order for your packages to be accessible to others, you are gonna want to do two things:

  1. Upload the .box files somewhere where they can be accessed via HTTP.
  2. Optionally, publish those URLs on VagrantCloud.

You have lots of options for where to place the box files. Amazon S3, Azure storage and just recently I have been using CenturyLink Cloud object storage. I believe dropbox works but I have not tried that. Here is a script I have used in the past to upload to my Azure storage account:

$storageAccountKey = Get-AzureStorageKey 'mystorageaccount' | %{ $_.Primary }
$context = New-AzureStorageContext -StorageAccountName 'mystorageaccount' -StorageAccountKey $storageAccountKey
Set-AzureStorageBlobContent -Blob '' -Container vhds -File 'c:\path\to\' -Context $context -Force

Publishing your box to

You can certainly share your vagrant box once accessible from a URL. You simply need to share a Vagrantfile with a url setting included:

config.vm.box_url = ""

This tells vagrant to fetch the image from the provided URL if it is not found locally. However, vagrant hosts a repository where you can store images for multiple providers and where you can add documentation and versioning. This repository is located at and allows you to give a box a canonical identifier where users can get the latest version (or specify a specific version) of the box compatible with their hypervisor.

Simply create an account at vagrantcloud. Creating an account is free as long as you are fine with your box files being public and you host the actual boxes elsewhere like amazon, azure or CenturyLink Cloud. Once you have an account, you can create a box which initially is composed of just a name and description.

Next you can add box URLs for individual providers that point to an actual .box file. With a paid account you can host the box files on vagrant cloud.

Now your username and box name form the cannonical name of your box. Vagrant will automatically try to find boxes on vagrantcloud if the box cannot be found locally. Every box on vagrantcloud includes a command line example of how to invoke the box. So my box, Windows2012R2 can be installed by any vagrant user by running:

vagrant init mwrock/Windows2012R2

If I am on my windows box with Hyper-V, it will install the Hyper-V box (make sure to add --provider hyperv to your vagrant up command or set the VAGRANT_DEFAULT_PROVIDER environment variable). On my Ubuntu desktop with VirtualBox it will download the VirtualBox box. After it downloads the box, it caches it locally and does not need to download again.

So there you have it a free windows box under 3GB. Ok...ok, thats still crazy huge but I didn't name this blog Hurry up and wait! for no reason. A little hurry...a little wait...ok maybe alot of wait...whatever it takes to get the job done.