Tuesday, June 23, 2009

Cross-Domain Business Intelligence

One problem that seems to plague organizations these days is a lack of understanding of how Business Intelligence techniques apply to more than just the typical “what’s our sales performance” question. (See my previous post for more on applying BI techniques to IT data).

I think the root of this problem is much deeper than simply trying to understand BI technologies. I believe the problem stems from a fundamental lack of understanding how to apply a technology solution to a business problem. This may sound a bit like Dilbert-esque crazy talk, but I think I can make a pretty good case for this argument.

There’s a lot of talk and “noise” in the industry right now about “Cross Domain BI” (some call this pervasive BI, but I don’t agree with applying that term to this problem) where BI techniques are being used to tie together data from multiple dissimilar sources within an organization. This provides a unified view of just how well each aspect of the organization is performing. I think that this movement is destined for a very bumpy road unless organizations fundamentally change how they approach problem solving in general, and “BI” in particular.

Distilling the Problem

Anyone who’s been in an engineering role (not necessarily limited to software engineering by the way) for awhile has been faced with the problem of imprecise requirements or specifications. As engineers, we tend to understand how to deal with that problem (it depends a lot on the engineer, sometimes the lack of a good spec makes for a great excuse not to get the job done, or worse, leads to a product simply “built to spec” and sometimes it forces the engineer to become more involved in understanding the problem they are trying to solve) and move on. Unfortunately the trait doesn’t always hold true with those outside of engineering who typically drive Business Intelligence projects.

Agile Business Intelligence

I’ve made the base before that BI projects *must* be driven by Agile methodologies if they are going to succeed. The main point of my argument there is that a successful BI project must be able to adapt to changing requirements along the way, and must be extremely flexible in terms of the data provided to the end-user. I believe it’s also true that for “Cross-Domain” BI to succeed, there must be an Agile component to the business as a whole. If an organization is rigidly structured, with well-defined “silos” of information, any attempt to develop cross-domain BI will likely end up in several BI silos that ultimately become useless when combined. For a cross-domain BI project to succeed, each of the silos of information must understand how data from other silos can be used to improve their own performance. In order to accomplish this, there needs to be an over-arching description of the business goals for the BI project, as well as a description of the goals for each silo. Generally speaking, this is done by following the “Business Scenario”-focused process such as the Microsoft Solutions Framework (MSF).

This brings me back to my original point. In order to properly apply BI techniques to the “Cross Domain” problem, organizations must first understand the problem that they are trying to solve. If they do this by creating an over-arching “cross domain” Business Scenario that contains the following steps:

  1. What questions are you trying to answer?
  2. What data do you need to answer the questions?
  3. Where does the data exist?

They are more likely to succeed at delivering a useful solution. If they don’t follow this simple approach, they are likely to be left wondering what happened.


Anonymous said...

Excellent post.

In my experience the three questions at the end are often subject to change as a result of the new perspective the data offers. Meaning that not only must the development cycle be iterative but the interaction with the Business Analyst (BA) and development team with the business unit must also be iterative. The answers to the main three questions will certainly change as they are exposed to the new data.

I also prefer your definition of "cross-domain BI" versus pervasive BI. I use the term pervasive BI with singular business units, not the enterprise. By pervasive, I mean that it goes deep in an a area versus broad across an area.

Again, a great post that many organizations and BI teams out there should be keen to understand and adopt.

Ted Malone said...

Thanks for the comments. I wholeheartedly agree on the fact that the 3 questions themselves are agile in nature. When dealing with BI projects, I believe the Businss Analyst is an absolutely must have position...

Anonymous said...

I would add that in order to be successful with Cross-Domain BI, using an Agile iterative and incremental approach will allow to build across domain as the need for information arises. Pervasive BI may not be good but a unique and centralized DW isn't the right way to go either. Incrementally building bridges between the cross-domain BI systems will deliver better value.

Steve J. Laye said...

I completely agree that the answers to the main three questions will certainly change as they are exposed to the new data.
Symtex Business Intelligence is delivered through a trusted and familiar environment that solves the business needs of organizations of all sizes. Powered by the flexibility and horsepower of Microsoft SQL Server and delivered through the trusted and familiar 2007 Microsoft Office system, our business intelligence solutions work together to provide value to all users across your organization.

tccflipmode said...

While I agree that cross-domain bi is best approached using an iterative approach, that iterative approach doesn't necessarily have to be Agile.

Since "Data Warehouse" was first coined by Bill Inmon in the early 90's, the power of an interative approach in building a centralized, single version of the truth for enterprise data has be extoled by many.