Integrating DevOps tools into a Service Delivery Platform (VIDEO)

The ecosystem of open source DevOps-friendly tools has experienced explosive growth in the past few years. There are so many great tools out there that finding the right one for a particular use case has become quite easy.

As the old problem of a lack of tooling fades into the distance, the new problem of tool integration is becoming more apprent. Deployment tools, configuration management tools, build tools, repository tools, monitoring tools -- By design, most of the popular modern tools in our space are point solutions.

But DevOps problems are, by definition, fundamentally lifecycle problems. Getting from business idea to running features in a customer facing environment requires coordinating actions, artifacts, and knowledge across a variety of those point solution tools. If you are going to break down the problamatic silos and get through that lifecycle as rapidly and reliably as possible, you will need a way to integrate those point solutions tools. 

The classic solution approach was for a single vendor to sell you a pre-integrated suite of tools. Today, these monolithic solutions have been largely rejected by the DevOps community in favor of a collection open source tools that can be swapped out as requirements change. Unfortunately, this also means that the burden of integration has fallen to the individual users. Even with the scriptable and API-driven nature of these modern open source tools, this isn't a trivial task. Try as the industry might to standardize, every organization has varying requirements and makes varying technology decisions, thus making a once-size-fits-all implementation a practical impossibilty (which is also why the classic monolithic tool approach achieved, on averaged, mixed results at best). 

DTO Solutions has made a name for itself through helping it's clients sort out requirements and build toolchains that integrate open source (and closed source) tools to automate the full Development to Operations lifecycle. Through that work, a series of design patterns and best practices have proven themselves to be useful and repeatable across a variety of sizes and types of companies and environments. These design patterns and best practices have over time become formalized into what DTO calls a "Service Delivery Platform".

I recently sat down with my colleague at DTO Solutions, Anthony Shortland, to have him walk me through the Service Delivery Platform concept.

In this video, Anthony covers:

• The "quadrant" approach to thinking about the problem
• The elements of the service delivery platform
• The roles of various tools in the service delivery platform (with examples)
• The importance of integrating both infrastructure provisioning and application deployment (especially in Cloud environments)
• The standardized lifecycle for both infrastructure and applications 

Below the video is a larger version of the generic diagram Anthony is explaining. Below that is an exmaple of a recent implementation of the design (along with the tool and process choices for that specific project).

Using Rundeck and Chef to Build DevOps Toolchains at #ChefConf 2012 (VIDEO)

I presented at #ChefConf 2012 in Burlingame last Thursday on using Rundeck and Chef to Build DevOps toolchains.

The heart of the presentation was a demonstration of continuous build and deployment showing Adam Jacob's chef-rundeck plugin working as a Rundeck resource model source (node provider) and jobs using knife and the Chef server API to manage databag-based application configuration.

At the process level, the presentation connects the dots between service delivery platform design and the loosely-coupled toolchains.

Despite the hotel-wide power outage in the middle of the presentation, the video crew recovered nicely! Below you will find the video and the slides.

DTO #ChefConf2012

View more presentations from AnthonyShortland

High Velocity Release Management with Alex Honor and Betsy Hearnsberger (VIDEO)

This one is for the managers out there who straddle the Dev and Ops divide.

Alex Honor and Betsy Hearnsberger have seen the importance of release management dramatically change over the past decade. Through their collective experiences working inside organizations like E*TRADE, Ask.com, NASA Ames, and Zynga (as well as Alex's subsequent consulting work at DTO Solutions) they've each amassed a wealth of experience and insight into dealing with high velocity release engineering in large scale and complex organizations.

Since their professional paths have crossed multiple times, I figured I'd get Alex and Betsy together in front of a whiteboard for a chat. In these videos they talk about the common challenges they see, advice for managers addressing these issues, solution approaches that work, and criteria for tool selection.

Please note that these videos were originally shot on July 29, 2011. Due to a technical problem is was thought that these videos were lost. Lucky for us, they have been fully recovered. I'll have to get them both on camera again soon to discuss how their thinking has evolved since then. 

Part 1: Common problems

Part 2: Management's approach to the problems

Part 3: Solution patterns and tool selection

Kanban and DevOps Roundtable (Video)

Ok so it's more of a semi-circle than a roundtable... I was at the first ever Kanban for DevOps class this past week in Sunnyvale, CA and after looking around the room I couldn't let these folks go without getting them on video:
- Luke Kanies (Puppet Labs)
- John Willis (Enstratus)
- Gene Kim (Author)
- Dominica DeGrandis (David J. Anderson & Associates)

Lucky for our readers, they didn't disappoint. We talk about why we think Kanban is an excellent tool for solving DevOps flow problems and our Kanban experiences thus far. 

Here is the video:

Update: If you are in the Atlanta area, John Willis has started the Atlanta Limited WIP Society!

Crowbar is quietly getting more interesting (video)

Crowbar is an interesting project that I've covered before. Born out of Dell's cloud group, much of the initial buzz described it as an installer for the cloud era... "kickstart on steroids", if you will.

Crowbar's close association with the OpenStack project has further cemented its reputation as an installer to watch. But's it's Crowbar's quiet potential as a stack management tool that is the most interesting. Through the use of barclamps (Crowbar's modules) you can tell Crowbar to build a full stack from the BIOS config all the way up to your middleware and applications. John Willis on an episode of DevOps Cafe called it "Data Center as Code".

Crowbar barclamps are also an interesting way for independent projects or vendors to ensure that their projects/products can be easily integrated into a custom platform (today this type of focus is usually in the context of making things work on OpenStack). Want to add a new component to your platform? Grab the barclamp and Crowbar will know how to do the rest. Or at least that is the promise. The project is still young and the community is still forming.

Leading open source software projects is new territory for Dell, as a company, but the Crowbar team does seem committed and community focused. I've heard some grumbles from developers that barclamp development and testing cycles can be a bit tedious due to the nature of what you are building. But no reason to believe that those types of issues won't get sorted out over time. 

A couple of Crowbar related videos are below:

The first video was made by my DTO Solutions colleague, Keith Hudgins, after he wrote a barclamp for Zenoss. It's a short demo and tour that can give you a feel for Crowbar and Barclamps.

The next video is Barton George (Dell) interviewing Rob Hirshfeld (Dell). They start off talking about the Hadoop barclamp but quickly getting into a broader discussion about Crowbar.