Integrating DevOps tools into a Service Delivery Platform (VIDEO)

The ecosystem of open source DevOps-friendly tools has experienced explosive growth in the past few years. There are so many great tools out there that finding the right one for a particular use case has become quite easy.

As the old problem of a lack of tooling fades into the distance, the new problem of tool integration is becoming more apprent. Deployment tools, configuration management tools, build tools, repository tools, monitoring tools -- By design, most of the popular modern tools in our space are point solutions.

But DevOps problems are, by definition, fundamentally lifecycle problems. Getting from business idea to running features in a customer facing environment requires coordinating actions, artifacts, and knowledge across a variety of those point solution tools. If you are going to break down the problamatic silos and get through that lifecycle as rapidly and reliably as possible, you will need a way to integrate those point solutions tools. 

The classic solution approach was for a single vendor to sell you a pre-integrated suite of tools. Today, these monolithic solutions have been largely rejected by the DevOps community in favor of a collection open source tools that can be swapped out as requirements change. Unfortunately, this also means that the burden of integration has fallen to the individual users. Even with the scriptable and API-driven nature of these modern open source tools, this isn't a trivial task. Try as the industry might to standardize, every organization has varying requirements and makes varying technology decisions, thus making a once-size-fits-all implementation a practical impossibilty (which is also why the classic monolithic tool approach achieved, on averaged, mixed results at best). 

DTO Solutions has made a name for itself through helping it's clients sort out requirements and build toolchains that integrate open source (and closed source) tools to automate the full Development to Operations lifecycle. Through that work, a series of design patterns and best practices have proven themselves to be useful and repeatable across a variety of sizes and types of companies and environments. These design patterns and best practices have over time become formalized into what DTO calls a "Service Delivery Platform".

I recently sat down with my colleague at DTO Solutions, Anthony Shortland, to have him walk me through the Service Delivery Platform concept.

In this video, Anthony covers:

• The "quadrant" approach to thinking about the problem
• The elements of the service delivery platform
• The roles of various tools in the service delivery platform (with examples)
• The importance of integrating both infrastructure provisioning and application deployment (especially in Cloud environments)
• The standardized lifecycle for both infrastructure and applications 

Below the video is a larger version of the generic diagram Anthony is explaining. Below that is an exmaple of a recent implementation of the design (along with the tool and process choices for that specific project).

Video Q&A: Aaron Peterson and Kevin Gray on self-healing infrastructure

At LISA 2010, I caught up with Aaron Peterson (Opscode) and Kevin Gray (Dyn) after they gave a very interesting presentation/demo called "DevOps Gameday".

From the title, I think a number of attendees were expecting to see the standard Dev to Ops promotion/deployment of code that is so common to the DevOps discussion. Instead the presenters (Opscode, Zenoss, Dyn Inc.) focused on what happens when you have a failure after the code has been deployed. This demo was about self-healing infrastructure... breaking a multi-node system and having it heal itself.

Of course, this kind of canned demo isn't all that new in the vendor world. However, what is very interesting about their efforts is they want to capture the best practices required to do it and share the code with the world through their combined project (hosted on GitHub). 

If they fulfill the mission of their open project, it's exactly the kind of "here is how you can do what the big players do" sharing that is good for our industry. 

Anthony Shortland on strategies for solving DevOps problems (Video)

Last night, Adrian Cole (of jclouds fame) organized a Java-oriented DevOps Meetup here in San Francisco. 

There were a number of freeform lightening talks used as an excellent device to get the conversation started. Below is an interesting talk from DTO's Anthony Shortand. Anthony gave an excellent conceptual introduction to two strategies for solving DevOps problems, "Full Lifecycle Traceability" and "Establishing a Formal Separation of Concerns".

DevOps (live) at OSCON

Early reports from OSCON are that DevOps is a topic of much discussion. My fellow dev2ops.org contributor Alex Honor and I are headed to Portland this morning to give DevOps related talks at OSCON. If you are there Wednesday or Thursday, please come by and say hello!

Wednesday (7/21) 1:40pm in room Portland 251 is Alex's presentation...
Open Source Tool Chains for Cloud Computing 

Thursday (7/22) 10:40am in room D135 is Damon's presentation... 
The IT Philharmonic: How Out of Tune Are Your Operations? 

Both talks feature lots of new content (even though the titles and outdated descriptions on the OSCON site are similar to our Velocity talks) 

Panel discussion from OpsCamp San Francisco (video)

My fellow dev2ops.org contributor, Lee Thompson, and I were asked to be a part of an impromptu panel discussion that kicked off OpsCamp San Francisco.

The results were an interesting and organic conversation that ranged from "DevOps", to "repository and dependency management", to "security is not compliance", to "managing multiple data centers", and to a variety of other topics in the span of 24 minutes.

Luckily, we got it all on video.  (Special thanks to Erica Brescia from Bitnami for doing the camera work!)

OpsCamp San Francisco 2010 Panel Discussion from dev2ops.org on Vimeo.