We extract/transform/load into our data warehouse using SQR programs. One of the main reasons why we have not purchased an ETL tool is that we have complicated business logic embedded in lots of fields in many of our tables. Can ETL tools handle this well?

ETL tools provide very robust support for complex business logic. However, I do not want you to think that it is a panacea. At times you will find the need to encapsulate very complex tasks or to solve performance limitations, that you need to use the database's procedural language or a 3GL program to solve these issues.

ETL tools will help you in centralizing your functionality and to simplify your long-term maintenance. I used SQR in much the same way you are today for its support of complex logic, but if I were to choose today, I would suggest an ETL tool as my deployment option.

I would suggest to you that you talk to the leading ETL vendors and have them show you how they would support your complex business rules with their tools.

Hope this helps, good luck.

This was first published in June 2011