For Business Intelligence which type of solution is best will ultimately depend on individual business circumstances. The solution type assessment that has been discussed in this series of articles will aid that decision.

We are in the age of careful spending. So companies need to know that each dollar spent is going to provide business value. It is important to develop a consolidated global view irrespective of operational software used. Consolidating data into a common platform is not always easy. The Data Warehouse must always be the single source of the truth for corporate information. This must be based on an open and flexible architecture that is extendible. To be clear, whether the data warehouse database is a bespoke design or implements a packaged solution there is still a large amount of work to be completed. There will always be the need to build a project team and acquire data for the data warehouse.

When ready made solutions provide best results

Generally ready-made solutions provide best results when they offer a high degree of integration for existing operational applications. If using a corporate-wide ERP or CRM solution that provides the backbone tying all corporate applications together then it is possible to utilise the associated data warehouse as the corporate Business Intelligence backbone. The preference for a pre-built database can be strengthened if all of the applications identified can be provided by the same vendor.

Ready-made solutions are becoming increasingly complex as the solution providers learn and adapt their data models based on real business scenarios, these will continue to evolve into the future.

What custom solutions can provide

Custom-made data warehouses are capable of supporting any application that is required to run against it. The primary concern will always be the amount of time necessary to develop any solution. Even where packaged Analytic applications are to be used is it generally necessary to tweak the metadata to enable them to function smoothly against the database. Often these Analytic applications can best be supported through a custom data mart, fed by the data warehouse – the single source of truth.

When custom solutions provide best results

Custom solutions provide the best results when there is a high degree of complexity in the corporate legacy architecture, even if there is an intention trim and simplify the architecture. Custom solutions are also best when the solution itself is likely to be unique in nature, e.g. when your corporation is genuinely the only corporation in the world doing this – however this situation is increasingly rare in today’s business climate.

Five years ago the combination of complex business processes and a complex architecture guaranteed that customised solutions were necessary. Today this is less clear cut. look at some industry models and see whether it is possible to adapt them before making the final decision.

Leveraging the Solution Type Assessment

In the earlier articles the author talked about a Solution Type Assessment. Through this process it is possible to build a score based on firstly mission criticality, and secondly the complexity rating for your organisation. These will act as a weighting factor for the importance of each item covered in the decision process.the weighting factor should be agreed before starting the assessment. In weighting there is a tendency for business to show a low weighting factor for IT issues, this MUST be avoided as the technical obstacles do need to be considered.

The Complexity to Provide column relates to the ability of the current data processing facilities to provide such data in a succinct and meaningful way. It becomes complex if data is stored in many different systems across the world, and in different languages. It is also complex if many types of data storage system are used (e.g. non-relational data stores and relational data bases can provide parts of the data).

Overall taking a solution type assessment will ensure the necessary due diligence considerations are covered. Of course capability has to assessed alongside the overall project costs. Customised solutions can be implemented over a longer period of time whereas ready-made solutions will generally be implemented altogether, even if some data comes on-line at a later date.

-

This series will be taking a break for a few weeks, but when it returns the next article will focus on: “Staffing your BI Solution”

Previous:

Part 1: Financing the Business Intelligence Project (Buy or Build?)

Part 2: BI Project Decisions: Ready Made Database Options

Part 3: The Corporate Architecture and Complexity of Data Links

Part 4: Financing the BI Project: Uniqueness Factors (or USP)

Part 5: Starting your BI Project: Suitable Analytical Applications

This article is the third in the series on Financing the BI Solution and follows from “BI Project Decisions: Ready Made Database Options” and “Financing the Business Intelligence Project“.

The decisions we make about the solutions we buy, which must all be made at the start of any new implementation do require analysis of a wide range of factors. In the case of the BI solution these must consider the existing infrastructure and any proposed imminent changes to it. This does complicate the decision making, but it is an essential part of BI project success.

Within any Business Intelligence implementation it is necessary to understand the impact of the corporate IT architecture. The complexity of the architecture can be a key factor in identifying any plausible ready-made solution. Many organisations have a veritable spiders-web of systems and inter-relationships some of which will defy logic, yet exist they do. Whilst it is primarily the operational applications that will be providing data to the Data Warehouse that must be considered to support the solution must take a more holistic view. Any system may hold key pieces of information and is thus capable of providing data to the Warehouse.

If we examine any two companies we will become immediately aware of architectural differences between them (and these are based on real solutions provided in the past).

In one organisation there may be an evolutionary mix of systems of varying ages and types; with few systems integrated together; there are many overnight batch jobs that transfer data between systems, but no integrated approach. How the organisation was formed may have an impact on the solution needed: here acquisition has meant there are separate systems in each of 17 different countries, some are packaged solutions, but most are bespoke. For the global corporate headquarters to gather worldwide information it has to contend with a complex mix of old and new systems using different data storage standards; all potentially making this a complex solution.

Conversely a second corporations is more integrated: all applications use pre-built solutions, based on relational databases; bringing an integrated architecture, that is well documented, and has been at the foundation of its IT platforms. It should be clear to the reader that this company’s architecture is unlikely to be very complex.

It is essential as a part of the financial decision making process to grade both the business and technical complexity of the solution.

Integration of existing systems is always an essential step especially assessing the corporate legacy application architecture. Integration of systems is almost always ranked as one of the most important architectural factor when deciding whether a pre-built Data Warehouse can be implemented. Highly integrated systems can be an enabler for a ready-made database. They are not the sole reason to implement a pre-built solution.

It is necessary to add a word of warning before diving headlong into such an implementation. When companies introduce new ERP, or CRM systems there is immense pressure to introduce analytics at the same time. This can also be true for SaaS, or Cloud Computing solutions. Often this is the integrated ‘Business Warehouse’. When the new ERP or CRM is introduced it is unproven (from the business viewpoint) and there is an assumption that results taken from the out-of-the-can Analytical Application are valid and correct. This is not always the case. An assessment of data quality is as essential to the ready-made solution as it is to the custom built Warehouse. Generally a new system cannot have its data quality assessed until it is in production and has had the wrinkles ironed out of it.

Next article in this series “What Makes Our Business Unique?”

1. A full data model, usually linked closely with an operational application. SAP, Peoplesoft, Siebel and other software suppliers provide Data Warehouses linked with their ERP and CRM applications.
2. An industry ‘template’ that provides a basic model Data Warehouse relating to the industry sector that the business is trading in.

Remember this section is talking purely about the supporting database, the Data Warehouse, rather that the overall BI solution.

Additionally there are Cloud based BI, or BI as a Service, solutions that tend to have their own internal database, most are linked to cloud based solutions. Generally you should consider these to be included with the first choice here as they are normally related to other operational applications. However some cloud based solutions are not extendible, which may be a factor in the decision making.

This is a continuing article based upon: Financing the Business Intelligence Project (Buy or Build?)…

Full Data Warehouse Model:

Many software companies claim to provide pre-built, fully configurable and extensible Data Warehousing solutions. Few, if any, actually provide an ‘out-of-the-box’ solution for the Data Warehouse that is not linked to another application. Most are linked with other operational software, such as Enterprise Resource Planning or Customer Relationship Management, where the business will have committed a significant investment such technology and associated change programmes.

Solutions are likely to be tailored to industry specific business analysis needs and will probably include pre-configured ETL components that may include extraction software for other commonly used software packages. This can act as a considerable advantage as it can reduce Data Integration efforts.

The advantage offered is that where a business uses the supplier’s solution it can have a ready-made business intelligence offering ‘for a few dollars per head more’. The final solution is still customisable, but is linked into key enterprise systems. This is an excellent goal, but whether it is achievable is a question that must be factored into the decision process.

Industry Templates:

Bill Inmon, used to be a great believer in the use of industry templates, or generic data models. Other companies also provide customisable data models, focused on specific marketplaces. The general aim of these templates is to act as a teaser for a subsequent engagement of the supplier’s consultancy services in order to have that organisation complete the full Data Warehouse and Business Intelligence implementation.

Templates or generic data models are therefore an aid, or kick-start, to custom development rather than a full blown ready-made solution. They are discussed here because generic data models are sometimes seen as a solution rather than a step-up on the development process. The template, or generic data model, can provide a boost to the initial Data Warehouse development project. They can be used to:

♦  Provide a foundation that can be built upon

♦  Identify major data classes

♦  Identify common relationships

♦  Identify some common attributes within the data

♦  Identify some common keys and unique identifiers

♦  Get the modelling exercise off to a fast start

In other words, the generic data model is a good starting point for the creation of a corporate data model, upon which the Data Warehouse will be founded. Using the template or generic data model as the corporate data model is inevitably a mistake. They do not alleviate the need to think about the design of any solution; they simply provide a way to jump-start the data modelling exercise.

The template or generic data model is not really a ready-made solution it is really a short-cut to providing a custom solution. Such a model will not include any pre-configured ETL components so these factors have to enter into any decision made.

The Decision:

At the end of the day having a customisable model is important because the Business Intelligence capability should be able to map onto the organisation’s USP. These are factors that you need to use in order to make your purchase decision. The best models are visible and expansible.

It is easy to assume that there is a ready made solution available and today’s choices are becoming increasingly complex, because vendors are learning the lessons from work they have completed for customers in the past and they have therefore extended their models to include a large array of new capabilities. Some vendors have simply extended default models over time others make these capabilities available only for extra cost. Make sure you know all the cost factors before making any decision.

The Next Decision Factor: Your Corporate Architecture and the complexity of data

1. Whether to use a ready-made solution or have it custom built
2. What degree of external staffing resources are required

This is the first part in a series about financing your BI deployment.

In addition project costs will be impacted by:

◊ The focus of the solution; operational or strategic

◊ The type of architecture selected

◊ The development methodology

These are all issues that the decision maker needs to be aware of whilst working with the business to initiate the project. Financial justification is of-course vital to this process. It is essential that agreement is reached by the project board based upon the holistic needs of the business. How the business interacts with the project and the resulting solution will in the end determine the success of the implementation. This is a corporate-wide decision to make.

Some of the decision may differ in scale when extending an existing Business Intelligence solution, but on the whole the various aspects of this article all have to be considered.

Ready-Made or Custom Built?

This is a decision that impacts the structure of the underlying database, the Data Warehouse. In the past it was a relatively clear-cut decision that few if any solutions provided alongside new operational applications could offer anywhere near the complexity necessary to support strategic decision making. Their only contribution was in respect of operational decision making. The strategic decision making required more data to be aggregated from multiple systems in order to provide the right results.

Each packaged solution requires some customised data and user interfaces to ensure it fits each end-user business. The business is also expected to change its processes in order to work with certain applications. The use of ready-made OLTP solutions has made the task of hiring business community staff and training them easier. E.g. Job advertisements for an Accounts Administrator can specify SAP experience.

So can this experience be applied to the implementation of the Business Intelligence solution? Much depend on the factors making each business unique and the range of applications that will continue to be used within the organisation. Few, if any, of the true BI focused applications feature highly in any business community staff resumes, so training is always essential.

From the systems side a changing IT landscape during the past twenty-odd years where we prefer to buy rather than build encompasses the possible use of packaged solutions for both the Data Warehouse and the BI tool-kit. In order to make such a decision it is necessary to consider the impact of:

◊ The corporate architecture including the complexity of data links

◊ The degree of integration of existing systems

◊ Factors of uniqueness for the business

◊ The suitability of analytical applications

All of which can assist in assessing the suitability of potential ready-made solutions. In all cases it is important to understand the general goals of the business’s market sector, for example in some statutory controls will have a large impact and in others none. It is always necessary to consider what your corporation’s principle competitors are measuring, if only to ensure that you exceed those minimums. It would be great to have a sneak peek at the competitor’s solution, but often industry journals give an expose on what they are doing – these can be analysed to identify the factors being measured.

Another impact is the factors of uniqueness, e.g. what your corporation brings to the marketplace – each company has them, make sure you identify them.

The third factor that need to be considered is the existing legacy IT architecture. Many corporations have already purchased some of the technology that is intended to be used for the solution. Many systems come with data integration elements. From a pure cost effectiveness standpoint you should look at re-use instead of purchasing a new ETL solution. Most companies have them – it is a matter of whether they are fit for the new intended use. Another area to consider is pre-existing and spare licenses for existing analytical tools – many corporations complain about the cost of licensing, yet do not effectively utilise the licenses they already have.

The next article in this series: “Types of Ready Made BI Solutions”

Implementing Business Intelligence solutions it has historically always decided that a data warehouse would simply be re-populated on a new system as part of the disaster recovery plan. In the early days of data warehousing solutions it was possible to simply re-populate the database on a new environment. This is no longer an option for a Business Intelligence solution as the complexity of systems continues to grow. It is essential to consider mission critical the BI system is along with every other corporate application.

In determining how mission critical any system is there are a number of factors that need to be identified. These include:

• Goals generic to the industry sector
• Factors unique to the business being measured
• Applications and their contribution to global business goals
• Corporate legacy architecture

When assessing various parts of the architecture they forget to ask how important is this to the success of the corporation as a whole. E.g. how critical is the success of the passenger check-in system to the success of an airline? The answers will determine a league table of the system criticality.

How mission critical is your BI Solution?

The truth is it will never be the most critical system in the organisation, but it can no-longer simply be repopulated from old data. In our airline example systems relating to safety and ensuring planes continue to depart on-time will always be the most critical of systems. The analytics provided by the BI solution will have become very important in today’s business world.

Closed-loop analytics is becoming increasingly important to many corporations. In one telecom company this capability was so closely tied to the CRM system that Customer Service agents would simply be unable to function correctly without the BI input into the process.

“I want to close my account, and transfer my telephone number to provider X”, takes the agent to a decision based process, that will firstly determine whether the customer is one that the company wishes to retain. Assuming they have retention value the process will provide a set of alternative offers that can be presented to the customer, with the express intent of retaining them. Each step in the process combines the knowledge of the BI system with the capabilities of the CRM. Clearly a mission critical system – central to Disaster Recovery. Not having the two components working smoothly together will damage corporate responsiveness. Agreed the systems are not as mission critical as those necessary to keep the cellular network active, but crucial nonetheless.

According to Wayne Eckerson of TDWI “the litmus test is whether a data warehouse becomes so mission critical that when it goes down people begin to have problems”. This in my view is more the test of when the BI system need to have automatic failover applied to it (as DR is concerned with more than individual machine failure). Many of the database vendors, such as IBM, Oracle and Teradata include failover.

Failover would not have been possible for a server located in the world trade centre on September 11^th 2001, but recovering from the disaster means alternative systems come back on-line within the prescribed time.

Many data warehouses contain derived and aggregated data, which forms the basis upon which specific business decisions can occur. This data needs to be stored and associated with the decision made as an historical check-point. Whenever advanced analytics functions are engaged it is often necessary to update the data warehouse. It is unlikely this data will ever appear in any operational system, placing the data warehouse firmly in the critical path for disaster recovery.

There may be some operations managers that feel adding the data warehouse to the list of critical systems means that other systems are removed. It however important to remember that the role of the data warehouse is to retain historical subject oriented data and is not a backup of associated operational source systems.

The continuity of business ultimately depends on the recovery of the data warehouse as a part of a recovery from a disaster in order to sustain the business and grow from the point of the disaster and moving forward, which often necessitates innovative advances in the marketplace to demonstrate the effectiveness of the corporation.

Peter B. Giblett