Strategies to import big data into Oracle

migrationnosqloracle

What strategies are available to import about 500 million small transactions every day from an external system?

Oracle becomes very slow with very frequent commits.

The transfer process which is reading the data from the external system can be implemented individually.

A NoSQL based solution is possible.

Update:
The datasource is a 'transaction journal' of a different DB system.

Best Answer

You didn't mention how the data comes in, but I would hazard a guess that it is through flat files. If this is the case, I worked at an order processing company that used Oracle for the back-end of the WMS we used. We regularly received 10 of millions of records a day. We used SQL loader scripts to load the data. The caveat to that is that the data needs to be in the proper format. See here:

http://knoworacle.blogspot.com/2012/05/oracle-sql-loader-with-example.html

And of course from the makers:

http://docs.oracle.com/cd/B10501_01/text.920/a96518/aload.htm

Related Question