Semantic Unification of Business Information Systems

Business Information Systems continue to grow increasingly powerful as we become better at being able to represent the various types of information in the enterprise. The walls between performance management, content management, knowledge management, and business process management, will continue to crumble in the next couple of years as enterprises realize the strategic value of developing information models that leverage all of these technologies to form rich information tapestries within which the fabric of good decisions can be woven. Whether one considers a relational or multidimensional database, all the documents in a content management system, all of the data being passed through web services, or the trillions of web pages available, it seems obvious that there is value in developing semantic models that can unify the understanding of this data across these various domains.

While there is a lot of research and implementation of techniques of semantic integration of metadata stored in relational databases used by performance management vendors (e.g. CWM, or the Common Warehouse Metamodel), there is definitely a renewed interest in standards-based metadata representations of web content, leading to the "semantic web". Naveen Balani presents a very thorough overview of the semantic web that is well worth reading in his article on IBM's Developer Works entitled, "The future of the Web is Semantic".

The Semantic Web provides a common framework that allows data to be shared and reused across application, enterprise, and community boundaries. You can think of the Semantic Web as an efficient way to represent data on the World Wide Web, or as a database that is globally linked, in a manner understandable by machines, to the content of documents on the Web. Semantic technologies represent meaning using ontologies and provide reasoning through the relationships, rules, logic, and conditions represented in those ontologies.

The difference between a "Scorecard" and a "Dashboard"

With the release of Hyperion System 9, we've provided a single workspace environment for any user to engage in Performance Management activities, including working with both dashboards and scorecards. Unfortunately, I frequently hear our customers and prospects get confused on the difference between them, so I thought I would take a moment to clarify the difference.

When running any business, it is important to understand the historical and current drivers of success. A dashboard provides a window into the historical and current drivers of success by providing a navigable, interactive environment in which KPIs can be explored to uncover the detailed root causes of the existing business situation. Furthermore, dashboards provide a link to other operational information for further insight.

A Dashboard in Hyperion System 9

In Hyperion System 9, our dashboards provide:

• Associations. Allows authorized end-users associate objectives, metrics, targets, and initiatives with each other.
• Multiple Targets. Allows users apply two or more targets and associated thresholds to each metric, including forecasts, budgets, prior actuals, and external benchmarks, among others.
• Groupings. Allows authorized end-users categorize objectives, metrics, and initiatives by different perspectives.
• Guided Navigation. Uses steps to guide less experienced users through the data or analysis by limiting the drill down/across paths and providing context-sensitive recommendations for next steps (i.e. reports to see or actions to take).
• Dynamic Views. Allows users to define and subscribe to new views of “right-time” data coming from one or more operational systems.
• Advanced Analysis. Allows users perform “what if” analysis to model scenarios and perform regressions to improve the accuracy of forecasts, among other things.

Of course, any well-run business will not be focused on the past, but also care deeply about the future. What should the business goals be? What strategies should we use to achieve those goals? What metrics should we use to help us indicate whether we are on the right track? A scorecard is a framework for aligning corporate goals, the strategy used to achieve those goals, and the measurement of those goals. In other words, it is a forward-looking vehicle that can be used to drive the performance of the business.

A Scorecard in Hyperion System 9

In Hyperion System 9, our scorecards provide:

• Support for Kaplan and Norton's Balanced Scorecard Methodology, Stern Stewart's Integrated EVA Scorecard, and the Malcolm Baldridge frameworks
• Strategy, Cause-and-Effect, and Accountability Mapping
• Alerting when KPIs exceed established thresholds
• Initiative Tracking
• Performance Reporting

A scorecard or dashboard implementation can be very valuable to a business, but as you can surmise from the above, they are particularly powerful when used together. Having a window into the past, present, and future performance of your business is a clear strategic advantage, which is why Performance Management technologies are at the top of every executives mind these days.

Simple Semi-Structured Data

This is an excellent article by David Loshin on the value of what he terms "semi-structured data". I've seen this term being used to describe a wide variety of data, including raw HTML, XML, etc., but I think that Loshin captures a more precise and hence useful definition.

"There is an intermediate classification of content called “semi-structured data.” This refers to sets of data in which there is some implicit structure that is generally followed, but not enough of a regular structure to “qualify” for the kinds of management and automation usually applied to structured data. We are bombarded by semi-structured data on a daily basis, both in technical and non-technical environments. For example, web pages follow certain typical forms, and content embedded within HTML often have some degree of metadata within the tags. This automatically implies certain details about the data being presented. A non-technical example would be traffic signs posted along highways. While different areas use their own local protocols, you will probably figure out which exit is yours after reviewing a few highway signs."

"This is what makes semi-structured data interesting—while there is no strict formatting rule, there is enough regularity that some interesting information can be extracted. Often, the interesting knowledge involves entity identification and entity relationships. " This doesn't sound a lot different than classic ERD modeling or relational data warehouse modeling. With existing pattern recognition techniques, I wonder how difficult it would build a generic parser across semi-structured and structured data that could come up with composite entity models across multiple information domains?

How to handle deleted records from a source system in a data warehouse?

Question from comp.databases.olap USENET group:

I would like to see if someone can share their experience in handling deleted records from legacy source system in data warehouse. I don't see much coverage on this topic in Kimball literatures on how to handle this in ETL and model design. Did I miss something?

My response:

Hi Doug,

This is an excellent question. In my experience, the event of deleting records can be a very valuable source of information about a business process, and thus, it is very useful to capture the event in the data warehouse. I have typically handled this by adding a DELETED column to the fact or dimension table that stores a value of Y or N (or 0 or 1) for deleted versus valid records.

Then, from the end-user tool, during query execution, you can modify your query criteria to check for records that are marked deleted versus valid. To ensure adequate performance, make sure that the DELETED column is indexed using the appropriate technique for your database. In Oracle DBs, low cardinality columns like this are usually retrieved most efficiently by using bitmap indexes.

Note that for auditing purposes, this method works very well, because the underlying integrity of the systems are not challenged by using flags to mark deleted records. It is entirely possible to see the deleted records with the appropriate query.

Good Luck,

Nenshad

Under the Radar: Get Yourself Together! All that floating data needs some structure.

Another great post by Jeff Nolan (surprise, surprise). There is no question that there is a great interest in mining unstructured data, but its a vast space in terms of breadth and depth. For example, just because one can do text search across e-mails, PDFs, PPTs does not mean that one can easily facilitate searching audio, video, and other richer media types. More importantly, there are massive privacy implications for this type of unstructured data mining. Do you want your manager knowing what web pages you visited, what PDFs you read, what audio and video you downloaded? How do I separate that which I want indexed versus that which I do not?

More importantly, I think that unstructured data will have to be searched with structured data using some kind of unified metaphor. Yes, it's true that I have a hard time finding the right e-mail, PDF, PPT, etc., but what's even more frustrating is that I can't link these to existing corporate information assets, like my CRM system, or ERP system, or BPM system. I'm still waiting for the company that understands that what is necessary is an information model that can span structured data like RDBMSs, Multidimensional Databases, and XML data stores and can model the relationships between these entities and unstructured data. When I want a 360 degree view of the customer, I want one information system that understands the relationship between the customers service requests, open opportunities, e-mail correspondence, sales presentations, etc., and can give me a real-time barometer of the health of my relationship with them and more importantly what I can do to improve my relationship with them down the line.

Companies would be willing to pay A LOT for this kind of technology. Maybe I can get SAP Ventures to fund my next startup idea. :)

Hyperion Debuts Industry's First and Only Comprehensive Performance Management Solution: Hyperion System 9

So the cat is finally out of the bag and I can talk about it publicly. Hyperion System 9 is finally here! System 9 is the industry's first Business Performance Management System. What does that mean?

Well, business performance management, or corporate performance management as it is defined by Gartner, "includes the processes used to manage corporate performance (such as strategy formulation, budgeting and forecasting); the methodologies that drive some of the processes (such as the balanced scorecard or value-based management); and the metrics used to measure performance against strategic and operational performance goals." (Gartner Research, "Corporate Performance Management Applications Explained, October 4, 2005).

How is this different from Business Intelligence? Business Intelligence, as envisioned by Howard Dresner, who coined the phrase, probably would not be much different from Business Performance Management if it evolved along the route that he expected it would. However, what Business Intelligence has wound up becoming is the manifestation of reporting taken to its logical extreme. State of the art pure-play Business Intelligence tools allow end-users to format and layout reports in any way imaginable. They also allow for data exploration using the established techniques pioneered by OLAP vendors.

However, a report in and of itself is hardly valuable without a notion of the process with which it is attached to. And a process in and of itself is hardly valuable without an understanding of the goals of the organization. And the goals of an organization remain ephemeral if they are not operationalized in the form of strategic, operational, and tactical plans. So Business Performance Management software attempts to provide a closed-loop, management decision-making infrastructure that builds upon Business Intelligence technologies but applies it as part of a structured management methodology, such as that formalized by the discipline of Management Accounting.

How does System 9 meet the Business Performance Management mandate? We'll dive into the details in an upcoming set of posts. In the meanwhile, there is a ton of new information on http://www.hyperion.com/.

Data Visualization, just a flash in the pan?

Dan Linstedt has very interesting and progressive ideas on Business Intelligence visualization, an area I have spent a lot of time researching and working on. In this excellent article, Dan talks about the tension between applying really cutting edge technologies versus end-users reluctance to use them.

The most interesting quote: "It's funny, we say business needs to drive technology, true. But sometimes business doesn't know what's possible until technology says: Hey, look at me, this is a new way of thinking - do you have a use?" I want to explore this theme in a future post, but it's time to head to the airport!

Good Morning Silicon Valley: Reader arrives weakly

Good Morning Silicon Valley is a really funny, offbeat look at what is going on in the Wild West. This article examines the new Google Reader, an RSS aggregator.

I set up a number of feeds by hand to test it out, and for a beta, it works pretty well, but of course it's far from perfect. As others have noted, it is a bit slow to load up, and I have gotten some strange javascript errors. That being said, I think that the approach of releasing software in beta and continually refining is definitely the way to go. Repeated customer feedback will shape the tool far more effectively than any developers or product marketers sitting in their cubicles discussing what they think "the market" wants.

This is one huge advantage of web-based software. You have the flexibility to do this. It is much more difficult when the customer owns the hardware, pays an IT staff for maintenance, and of course, when the application is mission critical, like a financial consolidation system. Still, I think there are principles from this type of development methodology that are certainly worth adopting for the larger companies that sell packaged software.

MDX Solutions : with Microsoft SQL Server Analysis Services 2005 and Hyperion Essbase

MDX Solutions : with Microsoft SQL Server Analysis Services 2005 and Hyperion Essbase

I just wanted to give a quick plug for George Spofford et al's MDX Solutions 2nd Edition. I work with George at Hyperion Solutions, and besides being a wonderful guy, he has phenomenal depth in understanding both Analysis Services and Hyperion's Essbase. What's great about the book is that he provides a number of interesting comparisons between the two technologies. For example, the book shows comparisons in how the same use case is addressed slightly differently in the MDX that is specific to each product. I hope the book does really well.