PostgreSQL – Using WITH and CREATE TEMPORARY TABLE AS SELECT

postgresqltemporary-tables

I would like to create a temporary table using a select statement that uses temporary views specified in a WITH statement (not the WITH statement of CREATE TEMPORARY TABLE).

e.g. something along the lines of

WITH a AS (
SELECT 1 foo, 2 bar 
), b AS (
SELECT 4 bar, 5 baz
)
CREATE TEMPORARY TABLE foo AS
SELECT * from a JOIN b ON (a.bar=b.bar)

If I comment out the CREATE TEMPORARY TABLE line, this works. How to create a temporary table using the query result of the SELECT, without rewriting the temporary views into a single query?

Best Answer

The WITH goes after the CREATE TABLE

CREATE TEMPORARY TABLE foo 
AS
WITH a AS (
SELECT 1 foo, 2 bar 
), b AS (
SELECT 4 bar, 5 baz
)
SELECT * from a JOIN b ON (a.bar=b.bar)

Note the above will result in "ERROR: column "bar" specified more than once" - but I assume you are using better (distinct) column names in the real query.

Related Solutions

MySQL: Execute CREATE TEMPORARY TABLE SELECT just one time

Here is the nature of table creation with CREATE TEMPORARY TABLE

You can use the TEMPORARY keyword when creating a table. A TEMPORARY table is visible only to the current connection, and is dropped automatically when the connection is closed. This means that two different connections can use the same temporary table name without conflicting with each other or with an existing non-TEMPORARY table of the same name. (The existing table is hidden until the temporary table is dropped.) To create temporary tables, you must have the CREATE TEMPORARY TABLES privilege.

My guess would be you must have done the following sequence of events:

You ran CREATE TEMPORARY TABLE
Terminated DB Connection (which makes the temporary table drop immediately)
You had to run CREATE TEMPORARY TABLE again because he table did not exist

I have seen this problem victimize many developers, especially when it comes to MySQL Replication. For example, if you run STOP SLAVE;, the SQL Thread will close. Any tables create with CREATE TEMPORARY TABLE will be dropped. When you run START SLAVE;, MySQL Replication breaks instantly because the table needed for INSERTs, UPDATEs, or DELETEs no longer exist. I have had to perform mysqlbinlog dumps on relay logs to locate the original CREATE TEMPORARY TABLE statements, run them as CREATE TABLE in order for the table to remain accessible to other DB Connections, including the SQL Thread when START SLAVE; is reissued.

As for your Questions

Is there a way to tell MySQL that the creation of the temporary table and the SELECT query are just executed if the table does not exist?
Maybe with an if-condition?
Or can I check to some information schema if a temp-table exists?

You cannot check the information_schema for temp table existence. In the OS, there is no manifested .frm in datadir. You could find it in the folder defined in tmpdir.

Here is an example: I will create a temporary table called test.junk and try to locate the whereabouts of the table. (Note: I am using MySQL for Windows)

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.5.12-log MySQL Community Server (GPL)

Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use test
Database changed
mysql> create temporary table junk (a int) engine=MyISAM;
Query OK, 0 rows affected (0.02 sec)

mysql> show create table junk\G
*************************** 1. row ***************************
       Table: junk
Create Table: CREATE TEMPORARY TABLE `junk` (
  `a` int(11) DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1
1 row in set (0.01 sec)

mysql> select count(1) from information_schema.tables
-> where table_schema='test' and table_name='junk';
+----------+
| count(1) |
+----------+
|        0 |
+----------+
1 row in set (0.08 sec)

mysql> show variables like 'datadir';
+---------------+-----------------------+
| Variable_name | Value                 |
+---------------+-----------------------+
| datadir       | C:\MySQL_5.5.12\data\ |
+---------------+-----------------------+
1 row in set (0.00 sec)

mysql> show variables like 'tmpdir';
+---------------+--------------------------------------+
| Variable_name | Value                                |
+---------------+--------------------------------------+
| tmpdir        | C:\Users\redwards\AppData\Local\Temp |
+---------------+--------------------------------------+
1 row in set (0.00 sec)

mysql>

The table is not visible in INFORMATION_SCHEMA.TABLES. Let's locate the table in the OS

C:\MySQL_5.5.12\data\test>cd
C:\MySQL_5.5.12\data\test

C:\MySQL_5.5.12\data\test>dir junk.*
 Volume in drive C has no label.
 Volume Serial Number is 2C92-485B

 Directory of C:\MySQL_5.5.12\data\test

File Not Found

C:\MySQL_5.5.12\data\test>dir C:\Users\redwards\AppData\Local\Temp\*.frm
 Volume in drive C has no label.
 Volume Serial Number is 2C92-485B

 Directory of C:\Users\redwards\AppData\Local\Temp

10/19/2012  11:04 AM             8,554 #sql7ac_3_0.frm
               1 File(s)          8,554 bytes
               0 Dir(s)  190,200,049,664 bytes free

C:\MySQL_5.5.12\data\test>dir C:\Users\redwards\AppData\Local\Temp\*.MY*
 Volume in drive C has no label.
 Volume Serial Number is 2C92-485B

 Directory of C:\Users\redwards\AppData\Local\Temp

10/19/2012  11:04 AM                 0 #sql7ac_3_0.MYD
10/19/2012  11:04 AM             1,024 #sql7ac_3_0.MYI
               2 File(s)          1,024 bytes
               0 Dir(s)  190,200,049,664 bytes free

C:\MySQL_5.5.12\data\test>

There is the temp table, in the tmpdir folder. When I closed the mysql client and run dir C:\Users\redwards\AppData\Local\Temp\*.frm, the table vanished.

RECOMMENDATION

You can do one of two things

Create the temp table, load it with data, keep the same DB Connection around, and then access the data with the same ProcessID.
Do CREATE TEMPORARY TABLE IF NOT EXISTS (This is really a band-aid to improper design but may circumvent the problem).

UPDATE 2012-12-18 13:06 EDT

Further Observation #1

If you stayed with the same DB Connection and still have this problem, here something else not previously mentioned: You gave the temp table an index. That should be a UNIQUE INDEX or PRIMARY KEY.

Further Observation #2

You could shift the operation from a CREATE TABLE ... SELECT to CREATE TABLE ... INSERT IGNORE

Further Observation #3

If you want to use temp tables, please specify MyISAM. InnoDB has a tendency to leave pigeon holes in the data dictionary with ibdata1 in the event of a crash or upon DB Connection termination.

With these three(3) observations in mind, here is my recommended change

CREATE TEMPORARY TABLE IF NOT EXISTS cache
(id int(11) NOT NULL, PRIMARY KEY (id)) ENGINE=MyISAM;
INSERT IGNORE INTO cache SELECT id FROM table WHERE xyz;

CREATE TEMPORARY TABLE IF NOT EXISTS cache
(id int(11) NOT NULL, PRIMARY KEY (id)) ENGINE=MyISAM;
INSERT IGNORE INTO cache SELECT id FROM table WHERE xyz;

Give it a Try !!!

Oracle 11g – Using Temporary Table Inside Procedure

It depends on why you are creating the temporary tables in MySQL.

Frequently, people that are creating temporary tables in other databases are doing so in order to work around limitations that don't exist in Oracle where readers don't block writers and writers don't block readers. In other databases, you commonly copy data from a permanent table to a temporary table so that your process doesn't block some other process that needs the same data. Since Oracle provides multi-version read consistency, however, this isn't necessary (or beneficial) in Oracle-- your process can process the data sitting in the real tables without worrying that it is going to block someone else. If that's the situation you're in, the proper response is simply to remove the temporary tables and process the data from the permanent tables.

Assuming that you really need a temporary copy of the data, you can create global temporary tables. You would create these tables outside of your code just like a permanent table and use them just like a permanent table inside your code. The global temporary table ensures that each session can only see the data that session has inserted. The only difference is that you're not dropping and recreating the structure of the table inside your procedure.

Another alternative would be to pull the data into a PL/SQL collection that you work with rather than using a temporary table. PL/SQL collections are stored in the server's PGA (one of Oracle's memory structures) so you generally want to limit the size of the collection particularly if there is a lot of data to process or there could be many sessions processing data simultaneously. You can do something like

DECLARE
  TYPE emp_tbl IS TABLE OF emp%rowtype;
  l_emps emp_tbl;

  CURSOR emp_cur 
      IS SELECT *
           FROM emp;
BEGIN
  OPEN emp_cur;
  LOOP
    -- Fetch 10 rows at a time from the cursor into the collection.
    -- You'd realistically want a larger limit, something between 100 and 1000 generally
    FETCH emp_cur 
     BULK COLLECT INTO l_emps
    LIMIT 10;

    EXIT WHEN l_emps.COUNT = 0;

    -- An example of manipulating the collection in memory
    FOR i IN 1 .. l_emps.COUNT
    LOOP
      l_emps(i).sal := l_emps(i).sal * 2;
    END LOOP;

    -- And an example of using the collection to update a table
    FORALL i IN 1 .. l_emps.COUNT
      UPDATE emp
         SET sal = l_emps(i).sal
       WHERE empno = l_emps(i).empno;
  END LOOP;
END;

Of course, in the example above, it would be far simpler (and more efficient) to simply issue an UPDATE statement that would double every employee's salary.