Postgresql – CSV or DB is quicker to read several lines from a file with 1,5 billion lines

csvdb2performancepostgresql

I have a program which would read several lines according to the indices of lines within a CSV file with 1,5 billion lines(using c fgets, or is there any tool in C faster than fgets?). Compare with storing records into a DB (postgesql or any other) table then reads the ID without any join, would the first propose the better way to do it?

Best Answer

Two great and very fast tools for structured text files are: