Postgresql – Search entire database(210 tables) for a unique Id (PK) and get the table name

postgresqlquerystring-searching

Rewritten Question:

I need to look through an entire database, 210 tables for a unique ID that is a number.

I have limited access to my database through things called nodes (Visual programming.)

These nodes allow for simple querys like SELECT x FROM x WHERE x I believe I can use most commands here.

I'm unfamilar with sql and thought I might be able to get some help here.

Below you can see an example of a node where I can type in commands.

and below that my database structure.

What i'm looking to do is look through the entire database for a unique number, this number changes dynamically but can always be found in the "Id" column of every table. This column is also a Primary Key.

By searching through the database and finding this dynamic unique number in one of the 210 tables I then need to be able to get the table name that the data is located in and the row information ie; all the different column entries for that row.

Example:

more info:

Best Answer

In your query:

SELECT table_name 
FROM information_schema.columns 
WHERE (:uniqueid ) IN (SELECT 'Id' 
                       FROM information_schema.columns 
                       WHERE 'Id' = (:uniqueid ))

you are trying to mix data and meta-data. information_schema contains only meta-data so you won't find :uniqueid in there. You will have to extract meta-data first, and then iterate over that some how. Assuming all columns are named id (which is a bad choice, but nothing you can avoid if I get it right), something like the following pseudo-code:

c1.execute("SELECT table_name FROM information_schema.columns WHERE column_name = id")
for row in c1.fetchall():
    t = row[0]
    c2.execute("SELECT 1 FROM %s WHERE id = :uniqueid" % (t) )
    if c2.fetchone():
        print "Found %s in %s" %  (:uniqueid, t)
        sys.exit(0)

will print the first table where there is an id = :uniqueid

Another option is to generate a union all for all tables:

SELECT 't1' from t1 where id = :uniqueid
UNION ALL
SELECT 't2' from t2 where id = :uniqueid
UNION ALL
...
UNION ALL 
SELECT 't210' from t210 where id = :uniqueid

Related Solutions

Postgresql – Merge two huge tables keeping only the unique rows

The basic problem can be solved with various simple queries. Considering all columns:

CREATE TABLE tbl3 AS
TABLE tbl1
UNION TABLE tbl2;

Given this additional information:

All columns except the id column should be considered for the unique check.

And:

I don't need to preserve the ID column.

Just drop the id column, then you can proceed with the simple query above.

I would import to temporary tables (much faster, less overhead) and only write the final result (tbl3) to a regular table - in one session because temporary tables are dropped automatically at the end of the session.

CREATE TEMP TABLE tbl1 ( <columns from above, without id> );
COPY tbl1 FROM '/path/to/file1';

CREATE TEMP TABLE tbl2 ( <columns from above, without id> );
COPY tbl2 FROM '/path/to/file2';

Alternatively, to preserve the input tables across sessions, you could use unlogged tables.

For best performance create and fill the target with CREATE TABLE AS and add the PK constraint in the same transaction:

BEGIN;

CREATE SEQUENCE tbl3_tbl3_id_seq;

CREATE TABLE tbl3 AS 
SELECT nextval('tbl3_tbl3_id_seq'::regclass)::int AS tbl3_id, *
FROM  (TABLE tbl1 UNION TABLE tbl2 ) sub;

ALTER TABLE tbl3
   ADD CONSTRAINT tbl3_pkey PRIMARY KEY(tbl3_id)
 , ALTER COLUMN tbl3_id SET DEFAULT nextval('tbl3_tbl3_id_seq'::regclass);

ALTER SEQUENCE tbl3_tbl3_id_seq OWNED BY tbl3.tbl3_id;    

COMMIT;

Replace all occurrences of "tbl3" with our desired table name.

Detailed explanation in this related answer:

What causes large INSERT to slow down and disk usage to explode?

I added a serial column (tbl3_id) as surrogate PK to the target table. Adding the actual PK constraint at the end (of the same session) is the fastest way.

Optimizing bulk update performance in PostgreSQL

Before you do it, test whether double precision is the best data type for all those columns. Chances are, some of them could be integer (cheaper for whole numbers) or must really be numeric (loss-less). If so, adapt your temp tables to begin with.

Postgresql: there is no unique constraint matching given keys for referenced table

After some clarification (in comments and chat), it seems that:

ChildA and ChildB are subtypes.
a VIPUser has a many-to-many relationship with all Child entities ("VIPUser "uses" Child").
there is a many-to-many relationship between VIPUser and ChildType (stored in Parent), essentially what types of Child a `VIPuser can use ("VIPUser "can use" a child of ChildType").

Then the relationships between entities can be shown in the diagram
(I renamed Parent to VIPCanUse):

               ChildType
VIPUser        /      \
      \       /        \
       \     /          \
      VIPCanUse          \
           \           Child
            \         /    |
             \       /     ------------
              \     /        |        | 
               \   /       ChildA   ChildB
              VipUses

Best Answer

Related Solutions

Postgresql – Merge two huge tables keeping only the unique rows

Postgresql: there is no unique constraint matching given keys for referenced table

Related Question