Sql-server – Advice for ETL from Oracle to SQL Server

etloraclesql serverssis

I'm building a data mart in SQL Server which needs to extract each month from an Oracle database. Currently this is done by exporting to flat files, then running all the inserts/updates in SQL Server…

There are approximately 10 flat files to export then insert before setting off the updates. Each file contains around 3.5 million rows and 5 to 10 columns per file.

Everything now is done strictly through stored procedures and I'm looking into SSIS assuming the permissions are properly granted this week. I've also come across Rhino ETL.

Does anyone have advice on efficient though relatively painless ETL from Oracle to SQL Server?

Appreciate the help!

Best Answer

SSIS does a good work of exporting data from Oracle to SQL Server. I suggest you create staging tables in SQL Server and use SSIS to extract the data for the 10 Oracle tables. Then either use stored procedures in SQL Server and execute them using SSIS or use Merge syntax and try to do both insert as well as update in Control Flow. Updates tend to be slow in Data Flow task of SSIS using Oledb command for very large number of rows.

Go through this link Using SSIS to get data out of Oracle: A big surprise! which explains the performance of SSIS while loading fairly large amount of data in a short time.

To get started for ETL process from Oracle to SQL Server try to read this documentation.

Related Question