SQL:2008 Standard – Is CTE Optimization Fence Behavior Specified?

optimizationpostgresqlsql-standard

I see frequent references to WITH queries (common table expressions, or CTEs) acting as an optimisation fence, where the server isn't permitted to push filters down into the CTE queries, pull common expressions up out of the CTE, etc. It's often claimed to be a behaviour required by the SQL standards.

CTEs are definitely an optimisation fence in PostgreSQL … but is this required by the standard, or actually just implementation detail?

For example, these mailing list posts claim or suggest that it's standard:

After mentioning it in a comment I was asked where it's specified – and after a look at the only draft of SQL:2008 I have access to I'm not having much luck finding it.

I haven't yet intensively studied the standard, so I'm hoping for a suggestion from somebody who has: Is the optimisation fencing of CTEs in PostgreSQL actually required by the standard? And if so, where's it specified? Or are the statements on the Pg mailing list in error?

See also the thread CTE optimization fence on the todo list?.

Best Answer

I think it's an implementation detail.

A conforming implementation is not required to perform the exact sequence of actions defined in the General Rules, provided its effect on SQL-data and schemas, on host parameters and host variable, and on SQL parameters and SQL variables is identical to the effect of that sequence. The term effectively is used to emphasize actions whose effect might be achieved in other ways by an implementation.1

I think an implementer could evaluate a common table expression 20 times, even in 20 different ways, and still have a conforming implementation. The only relevant issue is whether "its effect . . . is identical to the effect" of the sequence of actions defined in the General rules.

[1]. Section 6.3.3.3, "Rule evaluation order", in a draft of the SQL 2008 standard, having the local filename 5CD2-01-Framework-2006-01.pdf, p. 41 I have no idea where I got it. Google might know.