Sunday, September 30, 2007

Agile Business Intelligence

I was reading through the blogosphere his morning and came across some interesting observations on Agile BI. Basically, the author threw out the question, "what would our children turn out like if we raised them like we create BI systems?" The authors contention is that most people treat BI like a protected entity until it is absolutely ready for consumption, and even then rolls it out in a very limited fashion.

This got me to thinking about what Agile really means in the context of BI systems. The way I see it, Agile and BI are so naturally complimentary that you almost have to embrace agile in order to build and deploy a successful BI system. BI systems are all about condensing data into usable information then helping the consumer to develop that information into workable knowledge. Everyone who's ever built a BI system knows that the best measure of success in BI is how well that knowledge is used to close the loop and affect change on the processes that the BI system measures. If this doesn't scream "agile, iterative development" I really don't know what does. Consider the following tenets of agile programming:

  • Individual Interactions over Processes and Tools
  • Working Software over comprehensive documentation
  • Customer Collaboration over Contract Negotiation
  • Responding to change over following a plan

Now consider how they apply to a BI project:

  • It doesn't matter what you use to build the product, it's all about understanding the requirements and working towards a system that fulfills those requirements.
  • The system is going to change many times throughout the development lifecycle. Building a system that actually works and provides the output that the consumer needs is much, much more important that providing a manual and weeks of training.
  • Understanding what the customers need is more important that telling them what you're going to build.
  • See the second bullet. The system is going to change. The information needed to form knowledge will change. The desired output will change. The loop that you're trying to close will change. Everything changes.


I always tell my team that it is OK to be uncomfortable during the development cycle. As a matter of fact, I try and make sure that they ARE uncomfortable because that tells me that they're pushing the envelope. Waterfall methodologies just don't work in a BI world, and while agile can lead to discomfort, I really believe that the end result is well worth it.


Adron said...

Having been in Agile"ish" shops and waterfall shops I have become an advocate of Agile. The waterfall doesn't really work for anything but the most stodgy, slow moving, antiquated corporate beasts. Even those would do better to adapt fast moving, quick, elite Agile teams instead of waterfall departments.

I recently must say, I'm also exited recently as I've changed positions and am moving in the BI direction again. After almost 2-1/2 years of not touching that end of the development arena I have returned. Now that I have several more years of development experience, and specifically experience learning about and even using Agile approaches, what you say really hits me.


BI is an ABSOLUTE fit for Agile, you couldn't be more correct in this decree. With this one entry you grabbed another reader, I look forward to your ongoing writings.

Martin Proulx said...

This is really amazing. As I was searching for sites and blogs to help us refine our approach to Agile BI I searched on Google to see where I could find good content and much to my surprise this blog post comes up in the top results with very little competition. Seeing this post was written over 1.5 years ago I am wondering why so few see the tremendous benefits of going Agile. Hopefully more organizations will want to capitalize on getting value for their dollars and will finally understand that the traditional approach can no longer work.