Postgresql – How to speed up Oracle to PostgreSQL data transfer

gdalmigrationoracle-12cpostgresqlschema-migration

I have an Oracle schema that I am transferring to PostgreSQL with ogr2ogr.
This particular table has 5 columns with number(22) each. There are some 300M rows. In under 10 minutes I can transfer this table between two oracle servers over the network. ogr2ogr took a day to transfer 10M records and will take a month to transfer the whole table.

I suspect the problem is on the receiving end. Data is transferred to AWS PgSQL with 4 Gb ram. How to debug the bottleneck and speed up the transfer?

Best Answer

It might be worth pushing a subset to a file and loading it with copy, just to get a sense of the throughput. Copy is...legendary....for high-speed bulk loading into an empty table.

https://www.postgresql.org/docs/current/sql-copy.html