Cloud Foundry, Crowbar, and You

This is part 2 in our ongoing series on Crowbar and the new Cloud Foundry barclamp. If you haven't read part 1, go here. We'll wait for you. I've got some nice sweet tea when you get back.

If you've followed the "cloud world," you'll know that in CloudLand, there's three kinds of aaS. Crowbar was built as an *ahem* lever to get your IaaS in place. With the CloudFoundry barclamp, you can now begin to build a PaaS installation (Okay, this one's brand new and has some limitations, we'll talk about that in just a bit...) that can host web applications built on a wide variety of frameworks. (Seriously... with the latest announcement, just about any Linux/open-source based web application can be adapted to fit.)

Cloud Foundry in a Nutshell

Before we dive into running the tutorial, let's get acquainted with Cloud Foundry and it's architecture.

Cloud Foundry is a hosting platform for web applications. It provides runtime environments for Java, Ruby, Python, and PHP, along with a handful of dependent services like MySQL, RabbitMQ, and Redis. So you can build, say, a lightweight Sinatra app that is only an API broker to other services, or a full-blown Spring-based website with a Redis cache and MySQL back end and host them in the same hands-off (from the developer's perspective, anyway) platform.

How does that work? Magic, my friend, magic... well, really it's an asynchronous Rails app handling the API, talking via a NATS messaging queue to spin up the various parts for you as needed. There's a gem you can install that'll do all the API work for you, called vmc. You can always use the code inside the gem for some further automation if you need it, or just wrap it into your continuous integration loop to automatically deploy the latest version of your app on a successful build. (My team at DTO did it, roughly, a few weeks ago as a proof of concept)

I've got some links for you with much, much more in-depth information. Please wade through them at your leisure:

• Our own Cloud Foundry notes page, compiled by Alex Honor.
• In-depth discussion of the components, by Ezra Zygmuntowicz (one of the core Cloud Foundry team)
• More under-the-hood stuff on application deployment from the Cloud Foundry blog

Enough already, on with the worky bits!

I've put together a tutorial on how to get our own Cloud Foundry instance running on your Crowbar environment.

Now, this barclamp is in its early stages: it's a port of the chef-solo cookbooks VMWare put together in their development environment install script... so it only supports a self-contained single box install at the moment. Work will be proceeding on shoring up the installation so that we can break out the components across multiple servers so you can create a truly dynamic environment to host just about everything. Have questions? (Yup, their cert's bad. They know.) Want to help? Get involved!

Introduction to Crowbar (and how you can do it, too!)

Dell's Crowbar is a provisioning tool that combines the best of Chef, OpenStack, and Convention over Configuration to enable you to easily provision and configure complex infrastructure components that would take days, if not weeks to get up and running without such a tool.

What is Crowbar?

Crowbar was oribinally written as a simplified installer for OpenStack's compute cloud control system Nova. Because you can't run Nova without having at least three separate components (one of which requiring its own physical server), creating an automated install system requires more than just running a script on a server or popping in a disk.

Crowbar consists of three major parts:

1. Sledgehammer: A lightweight CentOS-based boot image that does hardware discovery and phones home to Crowbar for later assignment. It can also munge the BIOS settings on Dell hardware.
2. OpenStack Manager: This is the Ruby-on-Rails based web ui to Crowbar. (Fun fact: it uses Chef as its datastore... there's some pretty sweet Rails model hackery inside!)
3. Barclamps: The meat and potatoes of Crowbar - these are the install modules that set up and configure your infrastructure (across multiple physical servers, even)

Dell and other contributors have been working on extending Crowbar outside of just OpenStack as a generalized infrastructure provisioning tool. This article is the first in a 3-part series describing the development and build-up to a Barclamp (A crowbar install snippet) that gets CloudFoundry running for you.

So, that's neat and all, but what if you want to hack on this stuff and don't have a half-rack worth of servers lying around? This is DevOps, man, you virtualize it! (Automation will come later. I promise!)

Virtualizing Crowbar

The basics of this are shown on Rob Hirschfield's blog and a little more on project wiki. Dell, being Dell, is pretty much a Windows shop. We don't hold that against them, but the cool kids chew their DevOps chops on Mactops and Linux boxen. Since VMWare Fusion isn't that far from VMWare Workstation, it only took a half day of reading docs and forum posts to come up with a reliable way to do it on a Mac

Those Instructions Are Here.

Bonus homework: Get that running and read down to the bottom of the DTO Knowledge Base article I just linked, and you'll get a sneak preview of where we're going with this article series.

Now, go on to part 2!

DevOps HackDay at VMware featuring CloudFoundry

Start: 2011-09-08 09:00

End: 2011-09-08 18:00

Timezone: US/Pacific

Register now to spend a day hacking in the clouds with CloudFoundry.  The focus of this hack is to get a few working applications up in the cloud the #devops way: a fully programmable infrastructure that extends from the OS through the application lifecycle.  

We will be building a pipeline line that starts out in source control, flows through a build process, get packaged as Infrastructure as Code and ends up as a working application deployed to the CloudFoundry Open Source PaaS. The project will also include underlying instrumentation to collect data along the pipeline and also include an infrastructure for data storage and analysis.

The initial plan is implement this pipeline for a sample Java application. We'll be using Chef to automate the installation and configuration of the OS and middleware stack, then use CloudFoundry to automate application provisioning. Have a CloudFoundry ready app that you want to try out? Bring it along. Hopefully at the end of the day we will have a few Java, Ruby on Rails, and NodeJS applications up and running. 

Some of the tools we'll be using:

• Source Control - GIT
• Build - Jenkins
• Artifacts - Nexus
• DNS - DynDNS
• Release - Rundeck
• Configuration - Chef
• Deployment - IaaS Cloud
• Data Collection: Zenoss

Whether you are coming to learn from the experts, have ideas you want to share, or just like the camaraderie of a good Hack Day -- Bring your laptop and come ready to participate.  

DTO Solutions will lead this full day (9am-6pm) hands-on hackathon. We will have coffee/pastry in the morning, lunch, and a beer bash from 4-6pm with CloudFoundry/Layer 2 folks to discuss take-aways from the day.

Sign up now for your spot.

Huddle.com - Being Agile about Agile with Andy McLoughlin

Last week I attended a Pacific Crest Mosiac Summit in Vail Colorado and met with a number of institutional technology investors.  During the summit there were a number of interesting BOF’s and I was fortunate enough to moderated a #devops BOF.  Andy McLoughlin the founder of Huddle.com, was one of the individuals in the Devops BOF and he blew me away with some of his internal #devop practices.  I am happy to share some of Andy’s insights here with you in the video.

Full video of Israel Gat interview (Agile in enterprise, DevOps, Technical Debt)

After posting the exceprt on Technical Debt, I've gotten a number of requests to post the full video of my beers-in-the-backyard discussion with Israel Gat (Director of Cutter Consortium's Agile Product & Project Management practice).

We covered a number of interesting topics, including bringing Agile to enterprises, how Israel found himself part of the DevOps movement, and the measurement of Technical Debt.