Implement a data integration solution in six easy steps

Tip

Implement a data integration solution in six easy steps

This is the first part of a two-part series on BI data integration. See the second part for integration challenges.

 

Today, data integration solutions are deployed to support a plethora of business initiatives and technology implementations. It is important for organizations to understand how to go about implementing these solutions, if they are to avoid problems later. This first part of a two-part series looks at the key considerations to keep in mind when deploying a data integration solution.

The most important consideration when implementing a data integration solution is to ascertain whether there is a strong business need for it. Here are three business scenarios which may call for it:

  1. One-view of data across the group or business: An example would be the situation following a corporate merger or acquisition, when you need to integrate all the enterprise data. At Hypercity, we have multiple group companies specializing in retail: Hypercity, Shoppers Stop, and Crosswords. We needed to integrate customer data across all these businesses to serve our customers better.
  2. Facilitating data flow across systems: There may be situations where you need to integrate multiple data sources and applications to execute a business process; for example our business analytics tool, where the data flows from multiple applications like the merchandise management system or Oracle financials.
  3. Enabling data integration when new applications are deployed: A new enterprise application will need data from all the existing applications. At Hypercity, when we implemented our home delivery application, it needed to source customer- and product-information from the existing systems, making data integration a critical business need.

Move directly to Data integration in parts >>

Key steps involved

  • Work with the software vendor

The first step in deploying a data integration solution is to collaborate with the software vendor. It is in your interest to help the vendor understand your business requirements correctly. This will enable the vendor executives to accurately identify and integrate all the required data points.

  • Define and document integration priorities

List down all the integration tasks required. For instance, if you are deploying a new application, study its deployment schedule. Your aim should be to complete all data integration activity prior to deployment or Go-Live stage. Also, document the frequency at which incremental data should be updated. In addition, assess and document the benefits due to data integration solution, in terms of savings in cost and time.

  • Identify the appropriate integration interface

Data integration solutions provide two kinds of data interfaces: one-way and two-way. Know where to apply which one.

In a one-way interface, data is sent from point A to point B only; there is no return or backward movement. On our B2B portal, our suppliers can track stock movement in the stores. The inventory, payment, and sales information are sent to the B2B portal, but no data is sent back to these data sources.

In a two-way interface, data is sent from one application to another and back. In our case, when a new application such as point of sales (POS) is deployed, the product data is sent from the merchandise management system to POS, and then sales data is sent back again from the POS.

  • Select the right interface medium, may-be not the easiest one

In choosing the right medium to deploy a data integration solution, consider future needs as well as upgrades. Data integration methods include XML, comma-delimited, spreadsheet, direct database connection, and so on. The best method for you may not be the easiest one. Base your decision on scalability requirements, volume of data, and cost of the solution. Text-based integration had been prevalent for years, but XML and direct database connection are the popular modes employed today.

At HyperCity, during our integration of merchandise management systems (MMS) and point of sale (POS) solution, we noticed there was product information being generated by MMS in text format. But the POS solution required it in HTML format. We initially thought the easiest option was to convert the text to HTML. However, the conversion ended up taking longer than desired. We then realized that the better option was to create HTML files at the source system itself.

  • Monitor the process, put various check points

Fine-tune the data that is being integrated. For our B2B portal, data integration occurs at the end of each day, and only the incremental data is pushed to the server. At Hypercity, there are check points at every level, starting with the source system. Sales questions such as “How many quantities were sold?” and “What were the retail prices?” are raised at these check points. The extract and load stages of the B2B portal form the second check point, and the same questions have to be answered here as well. Automated systems check the accuracy of the entered data. In case of error, an email is sent to the concerned user that the data for the day is not uploaded properly, and corrective measures need to be taken.

Your data integration checkpoint design should not consider the current business-needs only as these investments rarely bear fruit in the short term. A three- to five-year information management roadmap for the organization should be factored in when creating the design.

  • Ensure security of the data

Set security policies according to risk level. If you plan to transfer the data on the internal network encryption may not be required. But if you need to send it over the Internet, it may be needed to prevent tampering.

Data integration in parts 

Data integration can be divided into three different parts, based on associated initiatives.

  • Analytic integration
    In analytic integration, one or more data integration techniques are applied in the context of business intelligence (BI) or data warehousing. Data from different applications is brought together at a single location and analyzed. For instance, at Hypercity we have a BI portal from SAS. All the transactional data is gathered and analyzed on this portal. This may include, for example, the analysis of the sales of white shirts in the last three summers. Such data helps us plan the stocks we need to maintain during the following summer.
  • Operational integration
    This involves the access and integration of data between operational applications and databases, whether within the same organization or across multiple group companies. Our home delivery application, for example, uses customer data from our customer database and product information from the product database for its functioning.
  • Hybrid integration
    The hybrid data integration falls between analytic and operational. It includes master data management, customer data integration, and product information management. We have used this concept in our Point of Sale (POS) application, where the data comes from two different sources, customer data and product data. Data integration enables day-today changes in these two data sources to be automatically reflected in the POS application.

<< Back to Top

 

About the authors:

Dipthi Karnad is a senior IT Manager at HyperCITY Retail India. She has over 16 years of experience in IT, of which 13 years are in the Retail IT domain. She has an in-depth understanding of retail business processes and expertise in designing and implementing solutions across platforms. Dipti holds a masters degree in financial management.

 

 

 

Kapil Tulsan works as an IT team lead at HyperCITY Retail India, and has over ten years of IT experience. He has been involved in system requirements study, analysis, design, development, and deployment. He has worked on several projects with multinational companies.

(As told to Mitchelle R Jansen)

This was first published in September 2012

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.