Why after the database suffers transaction ID wraparound would
transactions that were in the past appear to be in the future ?
They don't. The quoted text just explains why postgres needs to use modulo 231 arithmatic (which means transactions can wrap around as long as old transactions are 'frozen' early enough):
Normal XIDs are compared using modulo-2^31 arithmetic. This means that
for every normal XID, there are two billion XIDs that are "older" and
two billion that are "newer"
to be specific:
old row versions must be reassigned the XID FrozenXID sometime before
they reach the two-billion-transactions-old mark
or wrapping the XID around would cause things to break. To prevent that, postgres will start to emit warnings, and eventually shut down and refuse to start new transactons if necessary:
If for some reason autovacuum fails to clear old XIDs from a table,
the system will begin to emit warning messages like this when the
database's oldest XIDs reach ten million transactions from the
wraparound point:
WARNING: database "mydb" must be vacuumed within 177009986 transactions
HINT: To avoid a database shutdown, execute a database-wide VACUUM in "mydb".
(A manual VACUUM should fix the
problem, as suggested by the hint; but note that the VACUUM must be
performed by a superuser, else it will fail to process system catalogs
and thus not be able to advance the database's datfrozenxid.) If these
warnings are ignored, the system will shut down and refuse to start
any new transactions once there are fewer than 1 million transactions
left until wraparound
In other words "transactions that were in the past appear to be in the future" and "data loss" are entirely theoretical and will not be caused by transaction ID wraparound in practice.
So, guessing from the above, the first and last case are obvious - they are not visible anymore to transaction 2703 or higher.
The second one has to be looked up somewhere - I suppose it is the commit log, aka clog.
The 2nd one has HEAP_XMAX_INVALID
. That means it doesn't have to consult the clog, because someone has already done so, seen that the xmax
is aborted, and set a "hint bit" so that future processes do not need to visit the clog again for that row.
Which flags (or combination of flags) tell the system to visit the clog?
If there is no heap_xmin_committed
or heap_xmin_invalid
, then you have to visit the clog to see what the disposition of xmin was. If the transaction is still in progress, then the row is not visible to you and you can't set any flags. If the transaction committed or rolled-back, you set the heap_xmin_committed
or heap_xmin_invalid
accordingly (if it is convenient to do that--it is not mandatory) so future people don't need to look it up.
If xmin
is valid and committed, and if xmax
is not zero, and there is no heap_max_committed
or heap_max_invalid
, then you have to visit the clog to see what the disposition of that transaction was.
Is there a way to examine what's inside the clog? There are mentions about clog corruption in earlier versions of Postgres and a hint that one can build a fake file manually. This piece of information would help a lot with it.
I'm not aware of a user-friendly way of doing so. You can use "od" to dump the clog files in a suitable way to inspect them, and figure out where to inspect by using the macros defined in src/backend/access/transam/clog.c
I'm surprised there are no extensions on PGXN that does the work for you, but I couldn't find one. But I think it wouldn't be all that useful, because you really need to be able to do this while your server is not running.
Best Answer
No.
READ COMMITTED
doesn't retry. Neither doesSERIALIZABLE
. The application is expected to retry transactions that suffer deadlocks, serialization failures, etc.That description in docs is very misleading, I'll raise it on the docs list. PostgreSQL doesn't "start over" in
READ COMMITTED
at all, like some other DBs (e.g. Oracle) do. Instead it waits until the row it's waiting for commits or rolls back. If the other tx commits PostgreSQL reads the updated row, checks that it still matches anyWHERE
clause or other predicate, and then continues execution. The details are a bit arcane, seeEvalPlanQual
in the sources.In either
READ COMMITTED
orSERIALIZATION
the application should be able to re-issue a transaction.READ COMMITTED
transactions can fail in ways that will succeed on retry, including:SERIALIZABLE
just adds some more failure cases.Well written applications will cope with query failures and re-issue the transaction.