Introducing SAP Analytics - Process-Based, Composite Analytical Applications

Dear Readers,

I must apologize for not blogging in a whole month. A lot has changed in that time! I am pleased to share with all of my readers that I have recently joined SAP Labs, LLC as Director, Solution Marketing for SAP Analytics and am very excited to share some of what we are doing with you.

What is SAP Analytics? SAP Analytics is a suite of composite applications built on the SAP NetWeaver platform to enable seamless integration of analysis, transactions, and collaborative steps across the entire value network. For as many years as I was a consultant, I wondered why reporting, OLAP, analysis, etc. were disconnected from the CRM and ERP applications that business users worked on all day. If I were a sales rep working on my pipeline, couldn't I do a much better job if I had access to aggregated, analytical information with important sales KPIs in the same place that I kept my opportunity list? Wouldn't I be more likely to take the appropriate action, like moving my high value deal with a high close probability to the top of my priority list, if I had this rich insight right in front of me?

This is the premise of SAP Analytics. While no one questions the value of a stand-alone report or ad hoc analysis conducted using a BI tool, we believe that it can be even more valuable to the business users to gain analytical insight at the exact moment when they need it the most: at the point where they make their decisions. So, in keeping with NetWeaver and Enterprise Services Architecture, we're breaking down the traditional functional application barriers between ERP and CRM and BI and CPM, and giving business users the opportunity to use the most relevant, actionable analytical information we can provide them in the form of composite applications. Take a look at one of the screenshots below of a category promotion analysis application:

The graphical richness and interactivity of SAP Analytics is provided by our partnership with Macromedia (now Adobe), where we are leveraging the rich internet applications technology of their Flex product line. Even cooler, these applications can be constructed very easily by business users using the model-driven approach of our Visual Composer technology.

I will be writing about these technologies and many other things in much more detail in the coming months. I look forward to sharing more with you. In the meanwhile, I encourage you to take a look at the following link on the SAP.com website to learn more:

http://www.sap.com/analytics

Best Regards,

Nenshad Bardoliwalla

Automated Methods of Semantic Reconciliation of Structured Enterprise Data Sources

One of the biggest difficulties in building enterprise-wide data warehouses is assembling logical models of entities sourced from the data within ERP/SCM/CRM/BPM schemas, where the same entity (e.g. customer, product, employee) exists in multiple locations. In any given corporation, there are extraordinarily few, if any, individuals who know the details of more than one of the underlying schemas. For example, I may know where the customer data in a Siebel CRM system is held, but I would not have any idea where this is held in Oracle Financials or SAP R/3. To enable BI application construction where data is sourced from multiple transactional systems, vendors will have to lower the barrier to building an enterprise-wide information model, otherwise I don’t believe it will be adopted. The paper below discusses the challenges in accomplishing this, which are substantial, but there is tremendous room for innovation here that is necessary to make enterprise-wide performance management feasible.

An excellent article from ACM Queue discusses some recent research in this area that shows some promise for making this crucial task much easier:

Making heterogeneous schemas play nicely together has challenged computer
scientists for years, but we're on the path to better behavior.

Reconciling the vocabularies of different data sources is also the subject of the thesis by Dr. AnHai Doan which won the 2003 ACM's Prestigious Doctoral Dissertation Award. It's a fascinating read.

Semantic Unification of Business Information Systems

Business Information Systems continue to grow increasingly powerful as we become better at being able to represent the various types of information in the enterprise. The walls between performance management, content management, knowledge management, and business process management, will continue to crumble in the next couple of years as enterprises realize the strategic value of developing information models that leverage all of these technologies to form rich information tapestries within which the fabric of good decisions can be woven. Whether one considers a relational or multidimensional database, all the documents in a content management system, all of the data being passed through web services, or the trillions of web pages available, it seems obvious that there is value in developing semantic models that can unify the understanding of this data across these various domains.

While there is a lot of research and implementation of techniques of semantic integration of metadata stored in relational databases used by performance management vendors (e.g. CWM, or the Common Warehouse Metamodel), there is definitely a renewed interest in standards-based metadata representations of web content, leading to the "semantic web". Naveen Balani presents a very thorough overview of the semantic web that is well worth reading in his article on IBM's Developer Works entitled, "The future of the Web is Semantic".

The Semantic Web provides a common framework that allows data to be shared and reused across application, enterprise, and community boundaries. You can think of the Semantic Web as an efficient way to represent data on the World Wide Web, or as a database that is globally linked, in a manner understandable by machines, to the content of documents on the Web. Semantic technologies represent meaning using ontologies and provide reasoning through the relationships, rules, logic, and conditions represented in those ontologies.

The difference between a "Scorecard" and a "Dashboard"

With the release of Hyperion System 9, we've provided a single workspace environment for any user to engage in Performance Management activities, including working with both dashboards and scorecards. Unfortunately, I frequently hear our customers and prospects get confused on the difference between them, so I thought I would take a moment to clarify the difference.

When running any business, it is important to understand the historical and current drivers of success. A dashboard provides a window into the historical and current drivers of success by providing a navigable, interactive environment in which KPIs can be explored to uncover the detailed root causes of the existing business situation. Furthermore, dashboards provide a link to other operational information for further insight.

A Dashboard in Hyperion System 9

In Hyperion System 9, our dashboards provide:

• Associations. Allows authorized end-users associate objectives, metrics, targets, and initiatives with each other.
• Multiple Targets. Allows users apply two or more targets and associated thresholds to each metric, including forecasts, budgets, prior actuals, and external benchmarks, among others.
• Groupings. Allows authorized end-users categorize objectives, metrics, and initiatives by different perspectives.
• Guided Navigation. Uses steps to guide less experienced users through the data or analysis by limiting the drill down/across paths and providing context-sensitive recommendations for next steps (i.e. reports to see or actions to take).
• Dynamic Views. Allows users to define and subscribe to new views of “right-time” data coming from one or more operational systems.
• Advanced Analysis. Allows users perform “what if” analysis to model scenarios and perform regressions to improve the accuracy of forecasts, among other things.

Of course, any well-run business will not be focused on the past, but also care deeply about the future. What should the business goals be? What strategies should we use to achieve those goals? What metrics should we use to help us indicate whether we are on the right track? A scorecard is a framework for aligning corporate goals, the strategy used to achieve those goals, and the measurement of those goals. In other words, it is a forward-looking vehicle that can be used to drive the performance of the business.

A Scorecard in Hyperion System 9

In Hyperion System 9, our scorecards provide:

• Support for Kaplan and Norton's Balanced Scorecard Methodology, Stern Stewart's Integrated EVA Scorecard, and the Malcolm Baldridge frameworks
• Strategy, Cause-and-Effect, and Accountability Mapping
• Alerting when KPIs exceed established thresholds
• Initiative Tracking
• Performance Reporting

A scorecard or dashboard implementation can be very valuable to a business, but as you can surmise from the above, they are particularly powerful when used together. Having a window into the past, present, and future performance of your business is a clear strategic advantage, which is why Performance Management technologies are at the top of every executives mind these days.

Simple Semi-Structured Data

This is an excellent article by David Loshin on the value of what he terms "semi-structured data". I've seen this term being used to describe a wide variety of data, including raw HTML, XML, etc., but I think that Loshin captures a more precise and hence useful definition.

"There is an intermediate classification of content called “semi-structured data.” This refers to sets of data in which there is some implicit structure that is generally followed, but not enough of a regular structure to “qualify” for the kinds of management and automation usually applied to structured data. We are bombarded by semi-structured data on a daily basis, both in technical and non-technical environments. For example, web pages follow certain typical forms, and content embedded within HTML often have some degree of metadata within the tags. This automatically implies certain details about the data being presented. A non-technical example would be traffic signs posted along highways. While different areas use their own local protocols, you will probably figure out which exit is yours after reviewing a few highway signs."

"This is what makes semi-structured data interesting—while there is no strict formatting rule, there is enough regularity that some interesting information can be extracted. Often, the interesting knowledge involves entity identification and entity relationships. " This doesn't sound a lot different than classic ERD modeling or relational data warehouse modeling. With existing pattern recognition techniques, I wonder how difficult it would build a generic parser across semi-structured and structured data that could come up with composite entity models across multiple information domains?

How to handle deleted records from a source system in a data warehouse?

Question from comp.databases.olap USENET group:

I would like to see if someone can share their experience in handling deleted records from legacy source system in data warehouse. I don't see much coverage on this topic in Kimball literatures on how to handle this in ETL and model design. Did I miss something?

My response:

Hi Doug,

This is an excellent question. In my experience, the event of deleting records can be a very valuable source of information about a business process, and thus, it is very useful to capture the event in the data warehouse. I have typically handled this by adding a DELETED column to the fact or dimension table that stores a value of Y or N (or 0 or 1) for deleted versus valid records.

Then, from the end-user tool, during query execution, you can modify your query criteria to check for records that are marked deleted versus valid. To ensure adequate performance, make sure that the DELETED column is indexed using the appropriate technique for your database. In Oracle DBs, low cardinality columns like this are usually retrieved most efficiently by using bitmap indexes.

Note that for auditing purposes, this method works very well, because the underlying integrity of the systems are not challenged by using flags to mark deleted records. It is entirely possible to see the deleted records with the appropriate query.

Good Luck,

Nenshad

Under the Radar: Get Yourself Together! All that floating data needs some structure.

Another great post by Jeff Nolan (surprise, surprise). There is no question that there is a great interest in mining unstructured data, but its a vast space in terms of breadth and depth. For example, just because one can do text search across e-mails, PDFs, PPTs does not mean that one can easily facilitate searching audio, video, and other richer media types. More importantly, there are massive privacy implications for this type of unstructured data mining. Do you want your manager knowing what web pages you visited, what PDFs you read, what audio and video you downloaded? How do I separate that which I want indexed versus that which I do not?

More importantly, I think that unstructured data will have to be searched with structured data using some kind of unified metaphor. Yes, it's true that I have a hard time finding the right e-mail, PDF, PPT, etc., but what's even more frustrating is that I can't link these to existing corporate information assets, like my CRM system, or ERP system, or BPM system. I'm still waiting for the company that understands that what is necessary is an information model that can span structured data like RDBMSs, Multidimensional Databases, and XML data stores and can model the relationships between these entities and unstructured data. When I want a 360 degree view of the customer, I want one information system that understands the relationship between the customers service requests, open opportunities, e-mail correspondence, sales presentations, etc., and can give me a real-time barometer of the health of my relationship with them and more importantly what I can do to improve my relationship with them down the line.

Companies would be willing to pay A LOT for this kind of technology. Maybe I can get SAP Ventures to fund my next startup idea. :)

Hyperion Debuts Industry's First and Only Comprehensive Performance Management Solution: Hyperion System 9

So the cat is finally out of the bag and I can talk about it publicly. Hyperion System 9 is finally here! System 9 is the industry's first Business Performance Management System. What does that mean?

Well, business performance management, or corporate performance management as it is defined by Gartner, "includes the processes used to manage corporate performance (such as strategy formulation, budgeting and forecasting); the methodologies that drive some of the processes (such as the balanced scorecard or value-based management); and the metrics used to measure performance against strategic and operational performance goals." (Gartner Research, "Corporate Performance Management Applications Explained, October 4, 2005).

How is this different from Business Intelligence? Business Intelligence, as envisioned by Howard Dresner, who coined the phrase, probably would not be much different from Business Performance Management if it evolved along the route that he expected it would. However, what Business Intelligence has wound up becoming is the manifestation of reporting taken to its logical extreme. State of the art pure-play Business Intelligence tools allow end-users to format and layout reports in any way imaginable. They also allow for data exploration using the established techniques pioneered by OLAP vendors.

However, a report in and of itself is hardly valuable without a notion of the process with which it is attached to. And a process in and of itself is hardly valuable without an understanding of the goals of the organization. And the goals of an organization remain ephemeral if they are not operationalized in the form of strategic, operational, and tactical plans. So Business Performance Management software attempts to provide a closed-loop, management decision-making infrastructure that builds upon Business Intelligence technologies but applies it as part of a structured management methodology, such as that formalized by the discipline of Management Accounting.

How does System 9 meet the Business Performance Management mandate? We'll dive into the details in an upcoming set of posts. In the meanwhile, there is a ton of new information on http://www.hyperion.com/.

Data Visualization, just a flash in the pan?

Dan Linstedt has very interesting and progressive ideas on Business Intelligence visualization, an area I have spent a lot of time researching and working on. In this excellent article, Dan talks about the tension between applying really cutting edge technologies versus end-users reluctance to use them.

The most interesting quote: "It's funny, we say business needs to drive technology, true. But sometimes business doesn't know what's possible until technology says: Hey, look at me, this is a new way of thinking - do you have a use?" I want to explore this theme in a future post, but it's time to head to the airport!

Good Morning Silicon Valley: Reader arrives weakly

Good Morning Silicon Valley is a really funny, offbeat look at what is going on in the Wild West. This article examines the new Google Reader, an RSS aggregator.

I set up a number of feeds by hand to test it out, and for a beta, it works pretty well, but of course it's far from perfect. As others have noted, it is a bit slow to load up, and I have gotten some strange javascript errors. That being said, I think that the approach of releasing software in beta and continually refining is definitely the way to go. Repeated customer feedback will shape the tool far more effectively than any developers or product marketers sitting in their cubicles discussing what they think "the market" wants.

This is one huge advantage of web-based software. You have the flexibility to do this. It is much more difficult when the customer owns the hardware, pays an IT staff for maintenance, and of course, when the application is mission critical, like a financial consolidation system. Still, I think there are principles from this type of development methodology that are certainly worth adopting for the larger companies that sell packaged software.

MDX Solutions : with Microsoft SQL Server Analysis Services 2005 and Hyperion Essbase

MDX Solutions : with Microsoft SQL Server Analysis Services 2005 and Hyperion Essbase

I just wanted to give a quick plug for George Spofford et al's MDX Solutions 2nd Edition. I work with George at Hyperion Solutions, and besides being a wonderful guy, he has phenomenal depth in understanding both Analysis Services and Hyperion's Essbase. What's great about the book is that he provides a number of interesting comparisons between the two technologies. For example, the book shows comparisons in how the same use case is addressed slightly differently in the MDX that is specific to each product. I hope the book does really well.

MySQL 5.0

I just read that MySQL 5.0 is now available from Jeff Nolan's excellent SAP Ventures Blog. It is pretty impressive. Stored procedures/functions, triggers, views, archive storage, and a data dictionary start to make this look like a real enterprise-class player. Open Source software continues to make impressive in-roads into the enterprise software market. I wonder whether it is ready to be the engine of an enterprise-size data warehouse.

Combine MySQL with the emerging products proliferating from a number of open-source BI vendors (e.g., Pentaho, BIRT, OpenI), and you start to question why customers would want pay large amounts of money for most of the same functionality they can download off of SourceForge. I don't think we're quite there yet, but two years from now, the major players are going to have to come up with products of demonstrably higher value than what they offer now if they hope to compete with where Open Source BI software is going.

Web 2.0

Web 2.0 is currently one of the hottest topics amongst the technology cognoscenti. Tim O'Reilly has been promoting this idea for some time, and I love reading his thoughts and those of his colleagues at the O'Reilly Radar blog (http://radar.oreilly.com/). This slide is a really nice summary of many of the ideas, and I think it's influence will be increasingly felt across all software vendors, including those in the enterprise space.

A Compendium of Improvisational Guitarists Deserving Wider Recognition

This was originally posted to rec.music.makers.guitar.jazz about a year ago:

Dear friends,

Through the amazing resource that is this newsgroup and my own exploration, I've discovered many wonderful improvisational guitar players that I had never heard of before over the last year. I navigated to a lot of websites, listened to the clips, and wind up buying the CDs of many artists who I felt had a very distinguishable, personal voice. My own proclivities tend towards very modern guitar music, so artists that would fall under the "fusion" or "avant-garde" labels have a very strong representation in this list. However, there is no shortage of players on this list who can burn on "Joy Spring" or "Countdown" with the best of 'em.
I have no affiliation with any of these artists, other than to respect them for devoting their lives to an art form and instrument I cherish. I hope this list will offer many of you the chance to experience the thrill of discovering someone you never heard of previously.
If there are other artists that you believe are deserving of wider recognition, please add to this thread along with a link to their websites, and we can generate some interest in the work of our brethren.

Best Regards,
Nenshad Bardoliwalla

Adam Rogers http://www.crisscrossjazz.com/artist/RogersAdam.html
Andrew Cheshire http://www.users.voicenet.com/~joule/cheshire.html
Prasanna http://www.guitarprasanna.com/
Alex Skolnick http://www.alexskolnick.com/
Andre Bush http://www.andrebush.com/
Brad Shepik http://www.bradshepik.com/
Bruce Arnold http://www.arnoldjazz.com/
Bruce Eisenbeil http://www.eisenbeil.com/
Dan Faehnle http://www.danfaehnle.com/
John Catler http://microtones.com/
Dave Creamer http://www.davecreamer.com/
Dave Fiuczynski http://www.torsos.com/
Gene Bertoncini http://www.genebertoncini.com/
David Gilmore http://www.dgilmore.com/
David Torn http://www.splattercell.com/
Derek Bailey http://www.shef.ac.uk/misc/rec/ps/efi/mbailey.html
Henry Kaiser http://www.henrykaiser.net/
Brad Buethe (Fog) http://www.enigmaterial.com/jazz/jh2003/jh03_buethe1.html
Fred Fried http://www.fredfried.com/
Fred Frith http://www.fredfrith.com/
Nels Cline http://nelscline.com/
Liberty Ellman http://www.libertyellman.com/
Jackie King http://www.jackieking.com/
James Blood Ulmer http://www.geocities.com/BourbonStreet/Quarter/7055/Ulmer/Disko-ulmer.htm
James Emery http://www.james-emery.com/
Jay Azzolina http://www.jayazzolina.com/
Joe Morris http://www.joe-morris.com/
John Stowell http://www.johnstowell.com/
Shawn Lane http://www.shawnlane.com/
Jonathan Kreisberg http://www.jonathankreisberg.com/ Khabu Doug Young http://www.dkmjf.net/ducret.htm
Lorne Lofsky http://www.lornelofsky.com/
Chuck Underwood (Mantra) http://www.phillipsmusic.net/chuck/
Marc Ducret http://www.screwgunrecords.com/page_a.php?pageid=collaborators&sub=ducret
Michael Coppola http://www.9string.com/
Mike Moreno http://www.mikemoreno.com/
Mordy Ferber http://www.mordyferber.com/
Tisziji Munoz http://www.tisziji.com/tisziji/Default.htm
Neil Haverstick http://www.microstick.net/
Ron Affif http://www.fantasyjazz.com/html/affifrbio.html
Sheryl Bailey http://www.sherylbailey.com/
Sid Jacobs http://www.sidjacobs.com/
Steve Cardenas http://www.stevecardenasmusic.com/
Sylvain Luc http://alexandre.lacombe.free.fr/
Pete McCann (Mahavishnu Project) http://www.mahavishnuproject.com/
Tim Cummiskey http://www.tc7string.com/
Vic Juris http://www.steeplechase.dk/steeplechase/query.cgi?query=Juris&category=artist
Wayne Krantz http://www.waynekrantz.com/
Willie Oteri http://willieoteri.com/

Why software sucks - scottberkun.com

Scott Berkun has written a terrific book, the Art of Project Management, published by O'Reilly. His frank take on "Why software sucks (and what to do about it)" should be read by anyone in the industry.

My daughter sucking her thumb...in utero!

Technology is just unbelievable. Cari Jaquet, a really great gal whom I work with at Hyperion, was sweet enough to give my wife and I a gift certificate to 3D Baby Vu so we could check out what our daughter was going to look like. The procedure is very similar to any other ultrasound, except that they have this very advanced visualization technology that allows you to see the baby in all his/her glory. So this is one of the first pictures of our little girl, three months before she makes her debut. Well, hopefully three months! I'm not sure we're quite ready for her yet.

An Introduction to Nenshad

Hi, my name is Nenshad, and I'm finally joining the blog world. I have no idea how much I will get to write on a weekly basis, but I figured I would give it a shot. I have a huge diversity of interests: Aesthetics, Artificial Intelligence, Audio Engineering, Comparative Religion, Composition, Cognitive Psychology, Cognitive Science, Computer Science, Consciousness, Databases, Evolutionary Psychology, Expert Systems, Fuzzy Logic, Data Mining and Data Visualization, Developmental Biology, Economics, Entrepreneurship, Guitar, Information Architecture, Jazz, Mathematics, Neural Networks, Operating Systems, Philosophy of Science, Physics, Social Psychology,Statistics,Support Vector Machines, Technology Product Strategy, Zoroastrianism.

I'm a Senior Product Marketing Manager at Hyperion Solutions, Inc., in Santa Clara, California. My focus is Business Intelligence technologies, although I have worked on a lot of intersecting enterprise software technology areas as well, especially CRM, thanks to my erstwhile employer, Siebel Systems, Inc. I'm also soon to be a dad in December 2005. I am really excited and pretty frightened at the same time, but I think that is natural. Finally, I am an avid musician, although you wouldn't know it from my practice habits these past few years. I've played guitar and percussion for over twenty years and even foolheartedly recorded a jazz CD where I played all of the instruments myself.

I will try to touch on many of them at various times in my blog's evolution. Comments, especially those of the inciteful variety, are welcomed. Let's get this show on the road.