Services designed to fit your needs, mitigate your risk and deliver results.

Developer Exchange Blog

  • Home
    Home This is where you can find all the blog posts throughout the site.
  • Articles
    Articles Displays all articles posted on this site.
  • Bloggers
    Bloggers Search for your favorite blogger from this site.

DevOps - Get everyone on the same page


Are your processes separating or bringing together your development and operations teams?  If you want to get better, look at how the big guys do what they do so well, DevOps.

Over the last decade, we've seen the introduction of a lot of structured process into various organizations.  More often than not the process was introduced where formerly there was none.  As is often the case, the pendulum appears to have swung too far, and in many instances, the process itself is now an impediment to the results they seek to achieve.

Over the last decade many have pursued IT governance processes in an effort to wrangle out-of-control, seat-of-the-pants activities that were causing development and operations to deliver sub-standard results.  Ten years ago or more it was not uncommon for us to find corporate environments where there was no defined process for moving software from development to production.  It was a time when web based solutions were emerging fast and furious with much evolution occurring in the delivery platforms.  The result was constant chaos.  My opinion is that this was due to the newness of the environments at the time.  Operations folks were simply not up-to-speed on the variety of emerging tools that were being used to bring solutions about; so, they had little choice but to allow the developers to perform deployments themselves.  The result was operating environments that the operations team, themselves, didn't understand, dependencies that were not understood, and at the end of the day, less than stellar operational results manifested in down-time and unintended consequences when portions of the infrastructure were updated or changed.

Today, generalizing, quite the opposite seems to be the case.  Over the past decade governance processes have stepped in, introducing rigorous processes that have created a wall between development and operations.  This has been, to some degree, a response to statutory and regulatory reforms like Sarbanes-Oxley, Gramm-leach Bliley, HIPAA, PCI and so forth.  However, I would urge you to at least consider whether these processes have improved your business results or not.

So, what are the questions to ask?  I'd say they are results oriented.  Here are a few candidates:

  • What is the average time it takes for a release candidate to be tested, certified and released into the production environment?
  • What is the ratio of production releases that have to be rolled back due to critical errors that are found post-release?
  • What is the effort related to each software release? (i.e., what are the costs of going from release candidate to production release?)
  • How many seconds of down-time are incurred on each production release?

Certainly, those are not exhaustive, but what I'm suggesting is that you need to focus on two things: getting quality software released and getting it released efficiently and quickly.

So, how do the "big guys" do this?  The buzzword is DevOps.  By bringing development and operations closer together - not throwing up walls between the two, rather by having them work hand-in-hand to develop efficient, automated procedures for getting things done.

DevOps is, to some degree, tearing down the walls between development and operations.  Getting everyone on the same page.  Making sure that the build-test-release process is efficient, as automated as possible, and well understood.

Some say that DevOps brings agile methodology to the system administration space.  It exists at the intersection of Development, Quality Assurance, and Operations.  It has been said that in an efficient DevOps environment, software can, essentially, be continuously released.  I've heard stories of DevOps teams being able to perform up to 10 releases per day. Yes, per day. And you're probably thinking - right, some small app with tiny changes.  No, I'm talking about large complex public solutions that have quite a number of moving parts.

The reason they can do this is that they have automated the process to a very large degree. They are all on the same page with regard to how changes are integrated into the software, and they know how to isolate changes so that if they do fail, they fail in a way that isn't overly risky.  They also know how to rollback efficiently.

Doing this requires effort.  You're not going to get there if individuals have to manually perform a large part of the process.  You not going to get there if your deployments are hideously complicated.  Your probably not going to get there if you don't automate all deployments EXACTLY THE SAME WAY (e.g., deployment to test environments should be IDENTICAL to deployments to production environments).  You're not going to get there if you don't have post deployment tests that automatically run against the newly deployed system.  And, you're not going to get there if you can't easily recover from a deployment where an error is identified late.

I'm not trying to make the case that we need to return to anarchy.  I'm just saying that somewhere in the swing of the pendulum we went past optimal into a zone where process is the focus at the expense of results.  We simply need a better answer that provides the control and results the business seeks.

There is quite a bit of writing on this subject available on the Internet. You can easily start with a google search on the topic.  So, ask yourself whether your current process is producing the best results possible for your organization.  If not, maybe a few hours of research on DevOps would provide some better ideas. 

Quick chronological bio: Panther, Gopher, Blazer, Husband, Father, Doozer.