Architecture & DesignVirtualize Your Windows Development Environments with Vagrant, Packer, and Chocolatey, Part 1

Virtualize Your Windows Development Environments with Vagrant, Packer, and Chocolatey, Part 1

Developer.com content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

Editor’s Note: This is Part 1 of a two-part article. The second part has already been published.

If you are a developer, did you mess up your laptop or desktop with different projects’ environment software setups? Have you ever had the feeling that your desktop is fully loaded with unnecessary software and services running even after projects were completed and slowing the performance of your desktop?

If you are traveling and would love working at different offices, are you tired of re-creating the same project development environment? Have a different desktop environment for software development that runs on servers on production?

Want to avoid the long waiting time for getting virtual machines from your organization’s IT team and looking for simple self-service tools to manage your virtual machines? Using existing tools to manage virtual machines, do you see pain in maintaining development virtual machines either with Hyper-V or VirtualBox or cannot meet complex configuration requirements of development virtual machines?

Are you looking for simple self-service tools that can help you manage different development environments right on your desktop or laptop?

In this article, we will see how all these developer pain points can be addressed with the lightweight open source tool named Vagrant. Vagrant was traditionally successful in creating and distributing Linux virtual machines; now, with the latest version 1.6, Windows guest virtual machines are added as first class citizens. With Windows support, the next set of challenge is to build or find and discover Windows OS base boxes because these are not free. So, we will look at additional open source tools such as Packer and Chocolatey that compliment Vagrant in building Windows boxes and provision with the required software.

Developer Pain Points in Maintaining Development Environments

I have been playing the role of developer for the last nine years. For different engagements and projects, I have been involved in developing software that leverages .NET, ASP.NET stack of technologies, SharePoint, SQL Server, SSRS, SSIS, SSAS, BizTalk, Windows Azure, and open source technologies like NodeJS, PostgreSql, MongoDB, Docker. With this multitude of technologies, I have experienced many pain points that I would like to highlight and assume similar pain points would also be faced by all other developers.

Tools Conflict with Different Software Running on the Desktop

For one of the projects, I had the requirement of leveraging a Windows Server AppFabric caching service. For another project that I was part of, I had to leverage Windows Azure AppFabric and was not able to install tools for this because a few DLLs of Windows Azure AppFabric were conflicting with DLLs of the Windows Server AppFabric cache service. After spending a lot of time investigating, I found a solution by uninstalling the Windows Server AppFabric to make Azure AppFabric work.

Degraded Desktop Performance over a Period of Time

As I was working on different projects, I reached a point where my desktop performance was quite degraded. Investigating with a diagnostic tool revealed that my memory usage was above 90% and looking into different processes running revealed that multiple services that I was no longer using were running. These services are nothing but different databases like PostgreSQL and SQL Server 2008 that I installed a long time back for projects that were completed. I had to manually clean up all software and services to get back my desktop performance to normal.

Re-creation of Development Environments

When I was getting onboarded to a project that involved BizTalk usage to connect to different enterprise products, I had to follow instructions to create a development environment by installing and configuring the required software. This involved the installation of Visual Studio, SQL Server, BizTalk Server, and configuring them as well. I spent at latest two days doing this. If a new member joins the team, this process has to be replicated manually. And, it creates more problems to team if team members don’t use the recommended tools and have their own preferences of development tools.

Sharing Pet Project Progress to Solicit Quick Feedback

For one open source pet web project, I had to quickly share the progress of a web site to a different UX team, which is geographically located at a different place, to get quick feedback and act accordingly. Because the web site that I was developing has dependency on web technology frameworks, I had to rely on some hosting/cloud providers like Windows Azure, Amazon Web Service, Heroku, and so on to make it public and accessible. I had to go through the process of setting up an account and use the respective tools to host it publicly and then share. All these I had to do just for sharing my in progress with the web site one time.

Development Environments Made Easy with Vagrant

Although all the pain points mentioned previously can be solved if there is a dedicated IT team to manage the different VMs, what if you do not want to have dependency with other infrastructure teams and would like to do a self-service job with simple tools? Vagrant fits perfectly here because we can do self-service and manage our own environments with ease. But, why should we use Vagrant when we can use virtual machine management tools like Hyper-V or VirtualBox? Vagrant simplifies the management of VMs and can be scripted and versioned for different machine requirements. Also, with Vagrant workflow, which abstracts different providers, the VM setup will work the same irrespective of which provider you want to use. And what is more useful is that the same workflow can be used to create Amazon Web Service EC2 VMs as well.

Okay, so what is Vagrant and how does it work?

Vagrant is an open source tool that helps you create and configure lightweight, reproducible, and portable development environments. Under the hood, Vagrant leverages virtualization providers like VirtualBox, VMWare, AWS, Hyper-V, and so forth. It picks up a base box and runs in a virtualized environment with the help of a virtualization providers.  Vagrant base box is defined as package format for vagrant environments. A box can be used by anyone on any platform that Vagrant supports to bring up an identical working environment. The easiest way to use a box is to add a box from Vagrant Cloud at https://vagrantcloud.com. Vagrant cloud, a cloud service from Vagrant, is meant to simplify the discovery and distribution of complete development environments.

Up until version 1.5, Vagrant simplifies the creation and distribution of Linux virtual machines. With 1.6, Vagrant supports Windows guests as first class citizens officially. Vagrant can communicate with Windows over either SSH or WinRM

Spinning up Ubuntu VM is as simple as typing two vagrant commands:

Initialize VM:

vagrant init hashicorp/precise32

Boot:

vagrant up

The Vagrant init command will produce a Vagrantfile that describes the type of machine required for the project. In the previous command, a Vagrantfile is produced with a base box hashicorp/precise32 that’s downloaded from VagrantCloud. You can modify to further configure the machine before starting it with the “up” command. You can commit the Vagrantfile to your version control so that other developers in your team can check out the configuration file and just run the “vagrant up” command and be on their way.

By default, Vagrant uses VirtualBox as a provider to run the virtual machine. So, the previous commands set up a virtual machine with VirtualBox. It picks up aa base box named hashicorp/precise32 from VagrantCloud. We also can create own base box and upload it to VagrantCloud to share privately or publicly with others. Once we set up up the required VM with Vagrant using VirtualBox, and if we are ready to move the setup to AWS cloud, just we can achieve it with one command:

vagrant up --provider=aws

For more details on learning, take a look at https://docs.vagrantup.com/v2/.

Create Windows Boxes with Packer

Linux distributions are free and lightweight; therefore, it is very easy to find Linux base boxes from VagrantCloud or other sources. But, this is not true with Windows operating systems. Even if you are able to find Windows boxes, we would not be sure which key was used to configure these operating system machine images. So, it is best if we can build our own Windows base boxes. But, building our own Windows operating system is not a cake walk; it takes lot of time to get it done. Thanks go to Packer, an open source tool that helps us to create machine images from a single source configuration. All we need to do is configure the instructions and give it to Packer, which automates as per our configuration and gives the machine image. In addition, we can do software provisioning with configuration management tools like Chef, Puppet, shell scripts, and so on. With Packer post-processors, VirtualBox images and a Vagrant base box can be created.

Install Packer from https://www.packer.io/downloads.html and set the Packer executable path to the “Path” environmental variable so that it is available to use from all locations.

We will see more about this tool usage in the developer use cases section.

Simplify Software Provisioning with Shell Scripts and Chocolatey

Even though we can script installations and configurations of software with batch and PowerShell scripts, for development tools like Visual Studio or pgAdmin for PostgreSQL, it is a little tedious to script manually and use it. Thanks to another open source tool called Chocolatey. It lets you install Windows applications quickly from the command line via a central catalog of installation scripts. If you are familiar with Linux application package management tools such as yum or apt-get, this tool is similar but targets the Windows operating system. You can install tools like Git, Notepad++, and even Microsoft Office, not to mention Visual Studio, given a key. It uses PowerShell scripts and NuGet packaging format to install applications for you.

Install Chocolatey with the following commands and get going:

Command prompt installation:

C:>@powershell -NoProfile -ExecutionPolicy unrestricted -Command"iex
   ((new-object net.webclient).DownloadString
   ('https://chocolatey.org/install.ps1'))"
   && SET PATH=%PATH%;%ALLUSERSPROFILE%chocolateybin

PowerShell installation:

PS:> iex ((new-object net.webclient)
   .DownloadString('https://chocolatey.org/install.ps1'))

To install the MongoDB client tool called Robomongo, type in the following command either with a command prompt or PowerShell.

choco install RoboMongo

Virtualize01
Figure 1: Chocolatey command output

The ecosystem of Chocolatey is getting better and better day by day. Configuration management tools like Chef, Puppet, and Boxstarter are using Chocolatey. Microsoft has also decided to use the Chocolatey framework with its upcoming OneGet client. The CTP release of the OneGet client has a Chocolatey-compatible package manager that can install existing Chocolatey packages.

Developer Use Cases for Virtual Machine Requirements

Now that I have introduced various open source tools and their purposes, let us dive into different use cases a typical developer would land into and look for a better solution. Please note that I have used these tools on Windows 8 and built target images and virtual machines required. These tools should work on any operating system, including Linux distributions and Mac OS. Also I have used VirtualBox as a provider for Vagrant, but all the steps mentioned in each use case should work with any provider that Vagrant supports including Hyper-V, VMWare, AWS, and so on.

Use Case 1: Create a Windows 2012 R2 Sever box, Host It, and Remote into It to do Development Work

Okay let’s start with building a machine image for a Windows 2012 R2 server and hosting it on the desktop and be able to remote into it to work. The steps in this use case are base steps for all other use cases in this article; therefore, I will be more descriptive while detailing each step.

With Packer, we can automate the build process of any operating system images but that needs a little bit of a learning curve to configure the JSON configuration file that Packer requires. But, as we see, the open source community is vibrant; when tools/codebases are maintained at Github, we can get lots of starter templates and thanks to Packer Windows Github at https://github.com/joefitzgerald/packer-windows that has starter templates (JSON configuration files) and related scripts that help in building a Vagrant base box with VirtualBox builder.

The Packer build command takes a JSON configuration file. A JSON configuration file has three sections:

  • Builders: A builder is an array of all builders that Packer should use to generate different types of machine images. Each builder is responsible for creating the machine and generating an image for specific platforms. For example, there are separate builders for AWS EC2, VMware, VirtualBox, DigitalOcean, Google compute engine, Docker, and so on. Packer comes with many builders by default, and can also be extended to add new builders.
  • Provisioners: To install additional software beyond the base operating system software.
  • Post-processor: To produce special-format files, such as a Vagrant base box.

Let’s try to understand file windows_2012_r2.json as we will be building Windows Server 2012 R2. Please comment out or remove the vmware-iso section from the json file because we will not be using the VMWare builder. Let’s review the virtualbox-iso builder section.

 1. {
 2.    "type": "virtualbox-iso",
 3.    "iso_url": "http://download.microsoft.com/download/6/2/A/62A76ABB-9990-4EFC
       -A4FE-C7D698DAEB96/9600.16384.WINBLUE_RTM.130821-1623_X64FRE_SERVER_EVAL_EN
       -US-IRM_SSS_X64FREE_EN-US_DV5.ISO",
 4.    "iso_checksum_type": "md5",
 5.    "iso_checksum": "458ff91f8abc21b75cb544744bf92e6a",
 6.    "headless": true,
 7.    "boot_wait": "2m",
 8.    "ssh_username": "vagrant",
 9.    "ssh_password": "vagrant",
10.    "ssh_wait_timeout": "2h",
11.    "shutdown_command": "shutdown /s /t 10 /f /d p:4:1
       /c "Packer Shutdown"",
12.    "guest_os_type": "Windows2012_64",
13.    "disk_size": 61440,
14.    "floppy_files": [
15.       "./answer_files/2012_r2/Autounattend.xml",
16.       "./scripts/icrosoft-updates.bat",
17.       "./scripts/win-updates.ps1",
18.       "./scripts/openssh.ps1",
19.       "./scripts/oracle-cert.cer"
20.    ],
21.    "vboxmanage": [
22.       [
23.          "modifyvm",
24.          "{{.Name}}",
25.          "-memory",
26.          "2048"
27.       ],
28.       [
29.          "modifyvm",
30.          "{{.Name}}",
31.          "-cpus",
32.          "2"
33.       ]
34.    ]
35. }

Listing 1: Windows_2012_r2.json file contents

The following points describe the code mentioned in Listing 1:

  • Line 2 defines type. The value of virtualbox-iso indicates we will use the VirtualBox builder.
  • Line 3 defines the source for the OS ISO image file. This can be a URL or local file.
  • Line 6 defines whether we want to visually see VM being built. False indicates no visual indication.
  • Lines 8 and 9: Packer communicates with the virtual machine that is being built using SSH. These lines define what username and password Vagrant should use to communicate to the virtual machine that is being built.
  • Lines 14 to 19 define floppy files that will be mounted and executed while installing the operating system. Autounattend.xml is an important file that contains answers to OS installation questions.
  • Lines 21 to 23: vboxmanage. Resource configuration is to be allocated when installing the operating system. It can help speed up the provision process by providing more resources like more CPU and RAM memory.

The next important file to review is Autounattend.xml. This file should contain all answers to questions asked when the operating system is being installed.

...
...
...
  8. <InputLocale>en-US</InputLocale>
  9. <SystemLocale>en-US</SystemLocale>
 10. <UILanguage>en-US</UILanguage>
 11. <UILanguageFallback>en-US</UILanguageFallback>
 12. <UserLocale>en-US</UserLocale>
...
...
 63. <UserData>
 64. <!-- Product Key from http://technet.microsoft.com/
     en-us/library/jj612867.aspx -->
 65. <ProductKey>
 66. <!-- Do not uncomment the Key element if you are using
     trial ISOs -->
 67. <!-- You must uncomment the Key element (and optionally insert
     your own key) if you are using retail or volume license ISOs -->
 68. <!--<Key>D2N9P-3P6X9-2R39C-7RTCD-MDVJX</Key>-->
 69. <WillShowUI>OnError</WillShowUI>
 70. </ProductKey>
 71. <AcceptEula>true</AcceptEula>
 72. <FullName>Vagrant</FullName>
 73. <Organization>Vagrant</Organization>
 74. </UserData>
...
...
...
236.    <!-- WITHOUT WINDOWS UPDATES -->
237.       <!--
238.       <SynchronousCommand wcm_action="add">
239.          <CommandLine>cmd.exe /c C:WindowsSystem32
                 WindowsPowerShellv1.0powershell.exe -File a:
                 openssh.ps1 -AutoStart</CommandLine>
240.          <Description>Install OpenSSH</Description>
241.          <Order>99</Order>
242.          <RequiresUserInput>true</RequiresUserInput>
243.       </SynchronousCommand>
244.       -->
245.       <!-- END WITHOUT WINDOWS UPDATES -->
246.       <!-- WITH WINDOWS UPDATES -->
247.       <SynchronousCommand wcm_action="add">
248.          <CommandLine>cmd.exe /c
                 a:microsoft-updates.bat</CommandLine>
249.          <Order>98</Order>
250.          <Description>Enable Microsoft Updates</Description>
251.       </SynchronousCommand>
...
...
...
269.    <UserAccounts>
270.       <AdministratorPassword>
271.          <Value>vagrant</Value>
272.          <PlainText>true</PlainText>
273.       </AdministratorPassword>
274.       <LocalAccounts>
275.           <LocalAccount wcm_action="add">
276.             <Password>
277.                <Value>vagrant</Value>
278.                <PlainText>true</PlainText>
279.             </Password>
280.             <Group>administrators</Group>
281.             <DisplayName>Vagrant</DisplayName>
282.             <Name>vagrant</Name>
283.             <Description>Vagrant User</Description>
284.          </LocalAccount>
285.       </LocalAccounts>
286.    </UserAccounts>
...
...
...

Listing 2: Autounattend.xml file contents

The following points describe the code mentioned in Listing 2:

  • Lines 8 to 12: Define language-related settings
  • Line 68: Defines the product key with which the operating system to be installed. Leave blank for the trial installation.
  • Lines 236 to 245 Comment section to disable Windows update. Lines 246 to 251 instructs you to proceed with the Windows update while building a base image.
  • Lined 269 to 286: Define local accounts to be created, association to groups, and the like.

The file vagrantfile-windows_2012_r2.template, present in the scripts folder, defines Vagrant template script file that define the standards settings that would be inherited when a user instantiates the box.

...
...
...
 7. config.vm.define "vagrant-windows-2012-r2"
 8.    config.vm.box = "windows_2012_r2"
 9.    config.vm.communicator = "winrm"
10.
11.    # Admin user name and password
12.    config.winrm.username = "vagrant"
13.    config.winrm.password = "vagrant"
14.
15.    config.vm.guest = :windows
16.    config.windows.halt_timeout = 15
17.
18.    config.vm.network :forwarded_port,
       guest: 3389, host: 3389,
       id: "rdp", auto_correct: true
19.    config.vm.network :forwarded_port,
       guest: 22, host: 2222,
       id: "ssh", auto_correct: true
...
...
...

Listing 3: vagrantfile-windows_2012_r2.template file contents

The following points describe the code mentioned in Listing 3:

  • Line 9: Defines config.vm.communicator to instruct vagrant to use winrm for communication with a guest operating system.
  • Lines 12 and 13: Define credentials to be used when communicating with a guest operating system.
  • Line 18: Defines the forward port that maps a guest operating system RDP port number 3389 to host system port number 3389.

Virtualize02

Figure 2: Build flow with Packer

Figure 2 summarizes the Packer build flow. Builder takes the ISO location and the Autounnattent.xml file to build a base operating system machine image, provisioner picks up this base operating system machine image and installs additional software as per scripts and post-processor packs up the final machine image as the Vagrant box image and generates a Vagrant file (based on file vagrantfile-windows_2012_r2.template).

Now that we understood the process and different configuration files, let us build our first virtual machine and run it.

Steps

    1. Validate JSON configuration.
      Git clone the repo https://github.com/joefitzgerald/packer-windows and navigate to folder packer-windows.
packer validate .windows_2012_r2.json
    If there are any errors, correct accordingly and validate again.
  • Build a VirtualBox Vagrant base box.
    packer build .windows_2012_r2.json

    This command builds the base box by hosting a virtual machine in VirtualBox, copies ISO to guest CD drive, and installs the operating system.

    Figure 3 shows command output, and Figure 4 shows visual feedback of a guest virtual machine operating system installation. This feedback is visible because the key “headless” in windows_2012_r2.json was set to false.

    Virtualize03
    Figure 3: Packer build command output

    Virtualize04
    Figure 4: Visual feedback of the Packer build command

    The Packer build command outputs windows_2012_r2_virtualbox.box to the current directory.

  • Add the generated box file to Vagrant box list.

    Vagrant box list is a list of registered boxes per machine and per user. The box names present in this list can be used in multiple Vagrantfiles.

    vagrant box add WinServer2012R2 . windows_2012_r2_virtualbox.box
  • Verify the box list.
    vagrant box list

    You should see the entry for WinServer2012R2.

  • Initialize the Vagrant box.
    vagrant init WinServer2012R2

    This produces a Vagrantfile that has inherited its configuration from vagrantfile-windows_2012_r2.template.

  • Start the virtual machine.
    vagrant up

    To confirm that the machine is running, open the Oracle VM VirtualBox Manager and you should see an entry as shown in Figure 5. The name of this entry is in the format <boxname>_default_<numeric> etc. This entry is created with the Vagrant up command.

    Virtualize05
    Figure 5: VirtualBox Manager showing entry for WinServer2012R2 that is running

    Or, you can use the “Vagrant status” command that should output status as “running”.

  • Remote desktop to the running virtual machine.
    vagrant rdp

    Please note that, for any reason if you are not able to log into the box with the previous command, we can always login manually with MSTSC and the computer option as 127.0.0.1:3389. Remember that 3389 is a forwarded port that maps the host machine port 3389 to guest machine port 3389.

Get the Free Newsletter!

Subscribe to Developer Insider for top news, trends & analysis

Latest Posts

Related Stories