I have an Oracle schema that I am transferring to PostgreSQL with ogr2ogr.
This particular table has 5 columns with number(22) each. There are some 300M rows. In under 10 minutes I can transfer this table between two oracle servers over the network. ogr2ogr
took a day to transfer 10M records and will take a month to transfer the whole table.
I suspect the problem is on the receiving end. Data is transferred to AWS PgSQL with 4 Gb ram. How to debug the bottleneck and speed up the transfer?
Best Answer
It might be worth pushing a subset to a file and loading it with copy, just to get a sense of the throughput. Copy is...legendary....for high-speed bulk loading into an empty table.
https://www.postgresql.org/docs/current/sql-copy.html