Anatomy of a Software Development Role: DevOps
It's been nearly a dozen years since I first wrote "Cracking the Code: Breaking Down the Software Development Roles" and the associated specific role articles. The world has changed substantially in the last dozen years, but strangely, relatively little has changed in the roles for software development—except in the transformation of the deployment role into what is now being called "DevOps"—a contraction of Development-Operations. In short, we've changed how we operationalize the deployment of our code into our environments and into customer systems. It's time to address the changes that have come to the world of software deployment.
Continuous Release Cycles
Iterative and agile development is the starting point. Getting code cycles to happen faster and get code into the hands of the users quicker is a good thing—but it was just the first thing. The next challenge was how to automate the building of the software. Instead of a developer needing to do a build themselves, build servers and continuous integration allowed us to have up-to-date code a few minutes after a developer had checked in their code.
Once you can build code in a repeatable way, how do you integrate it into the infrastructure process of a configuration management system? This is at the heart of the changes in the role and the transition from development getting reliable builds of the code being written to code being deployed in the infrastructure. Automating this process and making it consistent made it possible to deploy updated versions of software daily if necessary.
Gone are the days when development would turn over a build every year or two and infrastructure would deploy it. Instead, writing software, testing, and implementations are happening much more frequently, and often in overlapping processes where there doesn't seem to be a beginning or end.
Optimizing Cycle Costs
To make agile and iterative processes really work, it's necessary to reduce the burden associated with completing each cycle. For example, say five percent of the overall effort is being devoted to deployment, because each 80 hours of work requires four hours of deployment work. This can be a huge drain on efficiency.
If you can push the effort for deployment to less than an hour per cycle, you can get the overhead of completing a cycle—due to deployment—down to around 1% of the total productive time. That means 4% more for getting "real work" done.
To get these optimizations, DevOps professionals leverage the same sorts of tools that the deployment role did years ago: batch files, scripting languages, virtualization, and so forth. These skills are pulled together with newer tools that are designed to make the process of repeatedly deploying code easier.
Virtualization and Containers
Over the past dozen years, we've moved from virtualization of servers being a new concept to it being the accepted practice and beyond, with Amazon, Microsoft, and others offering virtualized servers available for rent. Infrastructure as a Service (IaaS) became a thing, and no longer was it expected that a server meant a physical piece of hardware.
Scripting the virtualization process allowed for the teardown and rebuild of test environments to ensure that code was being tested in an unaltered environment.
Containerization, particularly Docker, reduced the overall footprint of the virtualized server into just a container that had many of the same advantages of virtualization but was substantially lighter in terms of memory footprint. This lowered the bar for developers to automate their developments, paving the way for easier integrations between development and operations.
Constraining the Skills Needed
The DevOps role—like the deployment role—is a hybrid role that requires significant development skills. It takes more than just a cursory understanding of what's going on in the solution to know why it's breaking in a new environment or a new container. Similarly, it takes a firm understanding of infrastructure concepts like virtualization, prerequisites management, and networking to know how to create the scripts that build and deploy containers from one environment to the next, as well as draining the existing installation's connections and slowly rehydrating the connections to the new container or virtual server.
The DevOps role is one of a diverse set of skills—as much as any role in the process, except for the solution architect. That's why, in the DevOps world, it's important to identify the key things that the role must do and separately establish ways that the role can enhance the project and reduce the overhead costs.
Where's the Position Heading, Anyway?
DevOps is, right now, a very poorly-defined and -understood role, as the DevOps moniker is just now taking hold. Like any new role or term, confusion will exist for several years as the industry begins to realize the essential elements of the role and the non-essential elements that can be ignored.
Certainly, the tooling is getting better rapidly, making it easier for folks to accomplish their work quicker; however, these tools are unlikely to reduce the knowledge burden on the role anytime soon. There will remain a high skills bar for the role for the foreseeable future.
The Good, the Bad, and the Ugly
Though there are many great things about being in DevOps today, there are at least a few things that aren't great.
- Good: Tooling is much better than it has been.
- Bad: Few organizations can invest in DevOps—just like few could invest in continuous integration.
- Bad: The amount of knowledge needed to be successful is large and always growing. It means that you'll need to maintain a relatively high learning rate to stay on top of things.
- Ugly: Small changes in environments means constant refinement of deployment processes—often very late at night or early in the morning.
Although in the past the deployment role wasn't seen as a particularly challenging or interesting part of the software development process, skilled developers with a passion for infrastructure can find that the DevOps role is a wild ride that can be a great deal of fun—if you're willing to do the learning necessary to do it well.