Businesses are working their way up the BI ladder, moving from simple reporting tools to complex analytical ones. On this journey towards BI nirvana, one may have to overcome challenges of a BI upgrade. “A BI system builds its foundation on other systems. This fact in itself compounds the process of a BI upgrade manifold”, points out Himanshu Jha, Head of Data Solutions (Business Intelligence) for Yahoo! Asia Pacific Division, who has pulled through many BI upgrades at Yahoo!
Need for a BI upgrade
Ranjit Satyanath, Head - Technology at Shoppers’ Stop and Crossword, who has many BI initiatives credited to his name, remarks, “Most organizations commencing their BI journeys publish static reports. But when BI reaches a maturity level, static reports do not suffice, giving rise to the need to migrate to cubes or data mining.” A well-thought out BI upgrade can make the best of human capital and infrastructure investments.
Some business users may want additional capabilities of their BI solution. To satisfy them an organization will not hurl away everything it has in place and replace it by new systems. This could be an additional plug and play BI upgrade. Here are considered factors that impact enterprise wide BI deployments.
A BI upgrade would be necessary from a technology perspective if the current platform is insufficient and the processes are running slow or are deficient in their reporting and analysis, due to:
- The BI brand being acquired by another vendor, making migration a necessity.
- The vendor ceasing to support the currently running BI version.
- Increase in data volume due to an increase in the number of source systems creating a need for data warehouse upgrade.
From business perspective, a BI upgrade may be required if:
- Changes in the business drivers resulting in requirement of additional analytical capabilities.
- The BI upgrade ventures into light weight BI, which is browser based and also supports mobile reporting.
Factors to consider during BI upgrade
1) Testing and parallel run
Rigorous testing and management are necessary for a smooth BI upgrade. A proof of concept (POC) is desirable and depends on the technology being upgraded. For plug and play BI upgrades, the vendors may willingly run a POC, but a paid POC will have to be undertaken for an enterprise-wide BI upgrade.
In a major BI upgrade, a parallel run may be considered as an option to a POC. The systems would perform the same processes and compare the reports. This will lend time to customize the BI upgrade to the needs of the business users. However, the parallel reports might not match and differences may occur during processes including warehousing, ETL, data migration, and integration.
In case of BI upgrade, the key architecture should be defined first. For instance, if the current BI solution covers seven to eight major areas, including business, information, and technology, it is better to start migrating one area at a time. Running all the systems in parallel and generating many parallel reports may raise questions about the validity of the data itself.
2) DW and ETL challenges
Emphasizing on the importance of having a stable business environment which would entail smooth data governance processes and seamless workflow to ward off the negative effects, Ramesh Babu, BI &DW practice leader at Mphasis and a certified project management professional, says, “The need for stable business environment is even more critical in a DW upgrade. It [i.e. the business environment] isn’t clearly visible to the user as a DW upgrade is not very frequent, unless a great data need arises. A positive impact would be on the speed of the reports generated.”
While upgrading BI, the biggest challenge for the data warehousing system is to deal with changes in the ETL process. It is recommended to address most of the concerns during deployment and identify the inputs (from the source systems) and outputs (organized data in warehouse) and the manipulations and data treatments in between. For instance, if one system uses ‘customer id’ and the BI upgrade uses ‘universal id’, both should be matched.
It is important to ensure that the new ETL process continues from the old one and the impacts on the various fields are communicated clearly to the users.
3) Data quality issues
Data quality is the heart and soul of healthy BI reports. Data quality issues should be tackled during the initial deployment of the BI solution. If not done properly it will come back to haunt you later. If incremental data is added during the BI upgrade, data management laws defined by the organization should be followed.
Satyanath explains, “To avoid issues during migration, there should be data archive and delta reports should be generated.” A delta report would show the expected new report as well as the coordinating older version.
4) Training and support
It is important to train the users on the various features. The real ROI is realized by the user. For the purpose of communication champions may be appointed. These could be trained and responsible for the users. “During a BI upgrade, a proper methodology should be followed. There should be communication, a project pipeline and roadmap,” asserts Jha.
If the BI upgrade is moving into a browser-based environment, an end user training would help minimize the aftershocks. For instance, although reports are generated in spreadsheets like Microsoft Excel, the same browser front end can be used to generate multiple reports. In Excel, people would freeze the top line, which is not possible in a browser. Thus, BI upgrade training and support in the form of helpdesks is essential.
5) Modular approach
If a date has been finalized for the migration, it could have a cascading effect. One wrong report leads to many others going incorrect. “The problem should be divided into parts and one part at a time should be tried to be fixed. During the launch, there should be a clear warning about the data that has issues,” suggests M V Palaniappan, senior software engineer at CA Technologies. Which areas are impacted and why are they impacted should be the goal of this process.
The ground truth that a BI upgrade is instigated by the increase in business demand should never be lost. The future demands and issues should be prognosticated to avoid undertaking a BI upgrade every couple of years.
This was first published in February 2011