Times are changing in the world of business intelligence it seams. Over the past decade we have seen BI tools evolve from the crude integrated development environment (IDE) into stunning tools that can be used by business analysts and power users. In many cases these tools have become so easy to use, that even a novice business user could analyze data. Many years ago BusinessObjects started this trend with Desktop Intelligence (As it’s known today) and Web Intelligence. While these tools were often labeled “ad hoc” reporting tools, a new breed of “self-service” BI tools have recently emerged.  With self-service BI gaining ground every day, one should ask why this trend has momentum. I believe there are a few reasons self-service is becoming a necessity in the mind of some organizations. At the same time I see huge potential for these tools to repeat the mistakes of the past. Within this article I will share a few thoughts on why we need something like self-service and why it bares a resemblance to the mistakes of the past.

Given that BusinessObjects started this trend many years ago, let’s take a moment to look at BusinessObjects Web Intelligence. If we take a moment to think back on its legacy, we might remember that Web Intelligence was marketed as an ad hoc reporting tool. Ad hoc reporting had many of the characteristics of today’s self-service BI tools. However, it lacked a sound means for users to manage data outside the guise of the BusinessObjects Universe. Later we will talk about why this matters and how self-service tools allow for data management. It’s true that the Web Intelligence Rich desktop tool provided this capability to a degree. However, this was not exactly a solution for users to easily manage local data sources. In the mind of the business user, this relegated Web Intelligence to being mostly a powerful reporting and interactive analysis tool. It was a tool that required an IT resource to provide access to the data source. For the novice user, Web Intelligence was also a bit overwhelming. It was packed full of powerful features. However, without training, it was difficult for a business user to comprehend its queries, data providers, report level aggregation, calculation contexts and a host of other features. Within most organizations, ad hoc report development was then pushed back on IT resources or savvy technical business analysts. With the report development back in the hands of IT, BI was often never fully adopted by the business users. Then there was the other big problem. User would often use these expensive tools to export data into Excel. For these users, it was easier for them to conceptualize data analysis in a spreadsheet. For most organizations, exporting data to Excel defused the ROI proposition for Web Intelligence. It also resulted in users that are never fully satisfied. I’m not of the opinion that these challenges were cause to chastise the BusinessObjects products. Most of these challenges were the result of a poor implementation by the adopting organization. There is often a lack of training, talent, management and leadership within an organization when it came to implementing Web Intelligence. Despite many of these adoption issues, Web Intelligence is still widely used within organizations today. In some organizations, it is used by all levels of the organization to interactively analyze data.  For others, they simply view static Webi reports built by IT.

Although, I view Web Intelligence as the perfect ad hoc reporting tool, the challenges inherit to implementing Web Intelligence have to be addressed. I’m not advocating the death of Web Intelligence or even the need to make changes. Those that understand how it operates can analyze data and make smart decision with little effort. However, I have come to the conclusion that everyone has a limit to their capabilities and we need a variety of tools to address different levels of user competency. Without a variety of choices, organizations might find their users revolting and seeking different tools.

With that said, there are four reasons that we need to include self-service tools in our BI product portfolio. Each reason on its own might not be compelling. However, I encourage you to think through the reasons in combination. Hopefully you will gain a better understanding of self-service tools. It might also help you understand why user are often annoyed with their technical BI resources.

Four Reasons Self-Service tools are needed

• Users understand data better when they can visualize it.

Is it true that most humans’ analyze data better in visual form? I’ve never conducted a scientific study to confirm, but my firsthand experience says “yes”. Remember the saying “a picture is worth a thousand words”? A trending column chart, with peaks and valleys, often stands out over a table full of numbers. A pie chart with a big slice gets my attention every time. It also makes me hungry for some pie. What about a bubble chart? It can express correlations between measures better than a paragraph in a power point slide. The list can go on and on, but I truly believe we understand data quicker and better when it is expressed visually.

If we accept this as true, we then have to ask the following question. Will users understand data even more if they create the visualizations themselves? This poses an interesting thought. Will they learn more from creating the visualization or will they learn the same by simply consuming it? In my mind, simply creating visualization is not the point. The point is allowing the user to create any number of visualizations while they work to understand the data. To me, easy to use self-service is a more important need. When we look at self-service BI tools such as SAP Lumira and SAP BusinessObjects Explorer, I find a user interface that allows the user to easily visualize their data. They can create any number of charts, graphs and analytics to help them see the data better. While I can do the same in Web Intelligence, these tools take the complexity down a notch or two. The end-result is a happier, smarter and more productive user.

• Users need a way to integrate data into their models.

It started with the data warehouse and then led to the semantic layer. IT departments everywhere found the perfect way to integrate all of an organization’s data into a single source and then present it to users in business-centric terms. The solution offered a perfect way to not only bring the data together but to also clean and conform it into actionable structures. So what could possibly be wrong with the methodology? On paper, nothing is wrong with this methodology. However, if you ask most business users and business managers this same question, you’re likely going to see their eye’s role around in their head. Why is this often the case?

There are often both technical and non-technical obstacles impeding the success of an EIM (Enterprise Information Management) and IDD (Information Discovery and Design) project. The technical obstacles are vast but often solvable with enough time and money. The non-technical reasons are often related to the organization. Again, a lack of management, talent, leadership and business ownership cause this process to appear to be impossible ( like moving a mountain). As with all mountain moving, the moving process is often very slow. Because its slow, it does not answer questions fast enough for the users. While IT resources are off devising solutions, users are kept in the dark. I’m not advocating the end of the traditional EIM process here. However, BI leaders might want to make room for intermediate alternatives to this process to pacify the needs of the business. If we truly believe that business users own their data, we can’t take their data on an extended vacation while we navigate the technical gauntlet of the EIM process. We also have to remember that what’s good for a fortune 500 company might not always be practical for a SMB (Small and Medium Businesses).

Self-service BI tools like SAP Lumira and SAP BusinessObjects Explorer allow business users to not only visualize data; they also allow business users to integrate existing data without an IT project. SAP Lumira achieves this with a simple user interface that walks the user through the processes in terms they can understand. Explorer allows you to upload a supported data file into its interface. I don’t think this will or should replace the traditional EIM processes, but it will afford business users with an alternative while IT does it the correct way. In addition, I believe there is real value in handing the initial management of the data over to the users. I can recall hundreds of times where IT worked effortlessly to integrate data into a model only to discover that the business only used it once or twice. What was the ROI on that project? I can also see this methodology playing a role in helping IT resources understand what the user it truly trying to analyze. Think of their models as a form of communication. Identify how they integrated the data and what they found useful. Then make this a part of a model that everyone in the organization can use and trust. Also, don’t forget that user often believe your idea better when they think it was their idea to begin with.

• Tools need to guide the user to the answer, not frustrate them with buttons.

Imagine you walk into an airplane cockpit during your flight and a frantic pilot asks you to take over the landing processes because he is sick. You take a quick survey of the cockpit. There are over 500 buttons, switches and gauges scattered around. One look at the dashboard in a commercial aircraft and you would likely have your life flash before your eyes. Now imagine you walked into the cockpit and there was only an LCD screen with a few buttons on a touch screen. One button said automated takeoff the other said automated landing. There was also an automated attendant button ready to walk you through the process. Assuming you trusted the on-board computer, you might think taking over from the pilot will be easy. Now imagine you are a business user asked to create a bar chart for an important executive meeting starting in 20 minutes. You run to the BI Launch Pad and crack open the Web Intelligence design mode to create the chart. You then remember that you skipped the training because you were busy with another task. For most users, this will be a similar experience as the one described above. In a perfect world we would all have pilot training. In the real world, we often have little time to truly master the tools available to us.

To this point, this might be an area where the current self-service tools need to improve. The current group of SAP self-service tools has really simplified the interface by eliminating many of buttons. However, there is not an overabundance of options to walk the user through the solution. I have seen a glimpse of the self-service future, so I know something like this is on the horizon. Even if it’s not, I believe it really should be. Imagine a tool that will analyze your dataset in the back ground and then present you with interesting facts. Imagine a tool that starts you off with a series of simple questions and then builds out most of your model. We have seen this for year in video games. They often walk you through the process by starting you off with small simple tasks until the user becomes an expert. BI tools could also use something like this to make users more confident when using the tool. Start them off with just a few buttons and simple tasks. As their usage score increases, give them new tasks and expose more buttons. Tools that make the user smarter, not those that require a smarter user.

• IT needs to give ownership of the data back to the users.

In my mind, this is the most important component of all BI projects. How do you get buy in to your BI initiative or project? The answer is often far too easy. Make sure the users are actively involved in the project. That seems simple enough right? Even before the advent of self-service BI tools, we find that a proper alignment between business owners and IT resource results in a successful BI project.

So how do self-service BI tools help give ownership of the data back to the users? Easy, if the business users start the analysis process using these tools, they feel like they own it when they come to IT and ask them to make their data available to everyone. Because they think they own the data (BTW they should), they are likely to be involved in the process to help you build a better model for everyone. The biggest receipt to disaster (in a BI project) is one where the waterfall project management style is applied. Business users ask IT to create something but they only provide IT with a list of lofty requirements. They don’t really understand the data or the tools, but they expect you to read their mind.  What are the chances you get it perfect and they love the solution “IT” designed? In my experience, the chance of success is minimal. Now imagine we start the project with the business user actively involved in the process. Let’s also assume that they already understand the data and data challenges because they have been working with it for several months using self-service tools. All of the sudden, your ETL process looks like data automation magic. You might just earn their respect and appreciation for the “long winded” EIM process.

Four Reasons Self-Service BI might fail

So all of this sounds like a great idea. However, this concept is nothing new. Users have been doing something like this for years using tools like Excel and Access. While these tools are not always great at BI, they do offer users with self-service comforts. When I look back on this old school self-service model, I am reminded that there were several issues associated with it. As I see it, there are four areas where we need to be cautious of a self-service model. I could also say, we need to make sure that self-service does not become the entire BI process within an organization. Some self-service BI vendors would like for you to believe they can replace your enterprise BI solution.

• Truth is objective or at least it should be.

When an organization implements a BI solution, it is paramount that the results of their efforts are consistent and true. If we believe this to be the case, we must expect that our EIM and BI processes are sound. One of the risks associated with a self-service BI solution is that the governance of the process and data can be nonexistent. Most self-service BI tools offer little in the form of data quality or data validation. There are also the technicalities of working with a hodgepodge of data sources. Self-service user need to make sure they understand the difference between an INNER JOIN and LEFT OUTER join when connecting to a RDMS source, for example. They also need to sometimes understand the differences between data types, numeric precision and database middleware. If they fail to understand even the most minor technicality, they can produce results that are understated or incorrect.

This is, in part, why IT resource often implements elaborate and sophisticated EIM solutions. In most cases these solutions are a mix of technology and strong data governance checks and balances. Because they often understand the technicalities of working with data, they are far more capable of producing repeatable data extraction processes that can massage data into a correct and optimal format. At the same time, they can also implement logical checks and balances to identify issues with the data.

The caution with self-service BI is that end users might not produce the correct results because they are not experts in dealing with data or setting up check and balances to validate their results. They might also be too busy to deal with validation and testing. If their processes are not sound, the results they produce can quickly be deemed unreliable and disregarded.

• One version of the truth please.

The last thing an organization needs is a disparity of truth. Imagine that the marketing team and operations team each manage their on self-service BI initiative. They both use some of the same data, but their processes of obtaining and enriching the data are very different. In the end, it is likely that their results will not match. This too can result in solutions that are unreliable and disregarded by the organization as a whole. If they don’t trust their BI tools, user might just resort to making decision on gut feelings or assumptions.

• Simple tools often produce simple results.

One real issue with self-service BI tools is that they often offer simple analysis capabilities. This is something that I recently experienced while implementing a SAP HANA and BusinessObjects BI solution for a large organization. The SAP HANA and BusinessObjects solution was chosen specifically to replace a 3^rd party self-service tool that was “easy to use” However, it lacked the capabilities to produce robust calculations. The solution we designed was based on Web Intelligence and the users were enamored by its ability produce complex calculations. My experience might not be applicable to every situation, but it is important that we make sure the users have the ability to produce complex calculations when the need arises. In some ways, the same can be said of SAP Lumira and SAP Explorer. Both tools are easy to use, but they often lack advanced features related to formatting and building complex calculations. This is likely by design (a need for simple tools), but it does prove that we need more than just self-service tools within an organization.

• Enterprise EIM and reporting isn’t dead.

BI is not always about empowering user to analyze data with whiz-bang, easy tools.  Sometimes we just need to distribute static content to the masses. There is also a need to secure data based on the consumer’s role in the organization. Take for example all of the features of the SAP BusinessObjects Enterprise platform. Many of the current self-service tools “don’t hold a candle” to the feature of a mature BI platform. To this end, self-service BI tools might have a lot of appeal to business users, but they often lack basic capabilities needed by all BI consumers.

Lets not forget about the data either. Conforming data for analysis can often be too complicated for the average user. This is especially true when the process of obtaining data and confirming it has to be repeated on a recurring basis. The traditional EIM tools and processes are well equipped to repeat this process. They also offer a very cost effective means to repeat the process after the initial development is completed.  In many cases, self-service BI tools lack the ability to automate data acquisition and data updates. This can result in a very labor intensive process and a proliferation of content.

In my opinion, implementing self-service BI is not about replacing the legacy EIM and BI processes within an organization. It is about finding a way to allow both methodologies to live in the same space and at the same time. In the sections above, where I described why self-service BI is often needed, I hinted to some of the ways organization can have their whole cake and eat some of it too. There are more ways to integrate the two methodologies but I will save those thoughts for another posting.

In my mind, self-service BI is a methodology that complements traditional BI not replaces it. We still need to make sure that our tools produce correct results. We also need to make sure that there is only one version of the truth throughout the organization. We also need to have tools that can manage both simple calculations and complex calculations. At the same time we have to remember that users can’t always wait on IT to produce a solution for every business question that arises.