Dev Environments: The Next Generation

02:30 PM - 03:25 PM on August 16, 2015, Room 705

Travis Thieman

Audience level:

novice

Watch:

http://youtu.be/pYLOuuR7HI0

Description

Development environments are a necessary part of every developer's workflow. They can also be a great source of friction. What may begin as simply running python my_app.py eventually bloats as you add more apps, more databases, more testing frameworks, and more developers. We'll talk about the evolution of a typical development environment, how it lets us down, and how we try to make it better. We'll end with an introduction to Dusty, a new tool which uses Docker containers to take our development environments to the next level.

Abstract

Each of us has one or more development environments which we rely on to help us manage the details of getting code to execute correctly on a machine. Our environments affect everything that we do as developers, so it's crucial that we are able to create and manage effective development environments which do their job well. The alternative is a constant battle against bad tooling which is slowing you down at every turn.

To understand exactly what makes a "good" environment versus a "bad" one, we'll start with simply running python my_app.py and see where we run into problems. This is the easiest way to get something running on our own systems, as we can run ad-hoc commands to install dependencies or fix system issues until our code works. However, this approach makes it impossible to reproduce our results on another machine or share our code with others. Without the proper environment, the code cannot run.

When the need to share or reproduce the application comes up, the necessary libraries involved in making the code work can be shared through a requirements file and installed in a virtualenv. The isolation of the virtualenv affords some level of reproducibility, but is still subject to the rest of the state presented by the system. Which system libraries are installed? Where are they located? What environment variables are set? These questions all present portability challenges and make it harder for us to share and reproduce our code.

To solve this problem, we turn to virtual machines, the next evolution in the development environment. We gain more ability to control the environment in which our code executes, and further isolation over the system is achieved. With virtual machines, though, we incur significant costs. VMs are slow and memory-intensive, and managing the VMs themselves is often difficult enough that configuration management software (itself presenting a significant cognitive burden) must be layered on top to keep everything in line. Our development environment has made progress, but we can do better.

Containers may give us the opportunity to further evolve our development environments. Out of the box, they appear to solve many of the issues with virtual machines. With containers, isolation is simple without incurring the performance impact of running many small VMs. The ecosystem being built around Docker is making this goal more achievable every day. We'll talk about what we might want out of a container-based development environments solution, and introduce Dusty, our attempt to leverage containers to make our development environments better.

Dusty is MIT licensed and available at http://dusty.gc.com