\df *crypt
in psql reveals the argument types of the pgcrypto encrypt
and decrypt
functions (as do the PgCrypto docs):
List of functions
Schema | Name | Result data type | Argument data types | Type
--------+-----------------+------------------+--------------------------+--------
...
public | decrypt | bytea | bytea, bytea, text | normal
public | encrypt | bytea | bytea, bytea, text | normal
...
so both the encrypt
and decrypt
functions expect the key to be bytea
. As per the error message, "you might need to add explicit type casts".
However, it works fine here on Pg 9.1, so I suspect there's more to it than you've shown. Perhaps you have another function also named encrypt
with three arguments?
Here's how it works on a clean Pg 9.1:
regress=# create table demo(pw bytea);
CREATE TABLE
regress=# insert into demo(pw) values ( encrypt( 'data', 'key', 'aes') );
INSERT 0 1
regress=# select decrypt(pw, 'key', 'aes') FROM demo;
decrypt
------------
\x64617461
(1 row)
regress=# select convert_from(decrypt(pw, 'key', 'aes'), 'utf-8') FROM demo;
convert_from
--------------
data
(1 row)
Awooga! Awooga! Key exposure risk, extreme admin caution required!
BTW, please think carefully about whether PgCrypto is really the right choice. Keys in your queries can be revealed in pg_stat_activity
and the system logs via log_statement
or via crypto statements that fail with an error. IMO it's frequently better to do crypto in the application.
Witness this session, with client_min_messages
enabled so you can see what'd appear in the logs:
regress# SET client_min_messages = 'DEBUG'; SET log_statement = 'all';
regress=# select decrypt(pw, 'key', 'aes') from demo;
LOG: statement: select decrypt(pw, 'key', 'aes') from demo;
LOG: duration: 0.710 ms
decrypt
------------
\x64617461
(1 row)
Whoops, key possibly exposed in the logs if log_min_messages
is low enough. It's now on the server's storage, along with the encrypted data. Fail. Same issue without log_statement
if an error occurs to cause the statement to get logged, or possibly if auto_explain
is enabled.
Exposure via pg_stat_activity
is also possible.. Open two sessions, and:
- S1:
BEGIN;
- S1:
LOCK TABLE demo;
- S2:
select decrypt(pw, 'key', 'aes') from demo;
- S1:
select * from pg_stat_activity where current_query ILIKE '%decrypt%' AND procpid <> pg_backend_pid();
Whoops! There goes the key again. It can be reproduced without the LOCK TABLE
by an unprivileged attacker, it's just harder to time it right. The attack via pg_stat_activity
can be avoided by revoking access to pg_stat_activity
from public
, but it just goes to show that it might not be best to send your key to the DB unless you know your app is the only thing ever accessing it. Even then, I don't like to.
If it's passwords, should you store them at all?
Furthermore, if you're storing passwords, don't two-way encrypt them; if at all possible salt passwords then hash them and store the result. You usually don't need to be able to recover the password cleartext, only confirm that the stored hash matches the password the user sends you to log in when it's hashed with the same salt.
If it's auth, let someone else do it for you
Even better, don't store the password at all, authenticate against LDAP, SASL, Active Directory, an OAuth or OpenID provider, or some other external system that's already designed and working.
Resources
and lots more.
Confusion?
From the docs json_populate_recordset(base anyelement, from_json json)
does not return an ARRAY
, it returns a setof anyelement
. That's fundamentally different from an array.
SELECT * FROM (SELECT ARRAY[1,2]) AS gs(x); -- array (int[])
SELECT * FROM (VALUES (1), (2) ) AS gs(x); -- setof anyelement (setof int)
I am of the opinion that you do not need an array, nor a setof anyelement
.
What you need to do things your way
For comparison this will turn json into foo[]
,
CREATE TYPE foo AS ( id int );
SELECT pg_typeof(
ARRAY(
SELECT *
FROM json_populate_recordset( NULL::foo, '[{"id":1},{"id":2}]' ) )
)
);
Which you can call like this
SELECT test_gps(
ARRAY(
SELECT *
FROM json_populate_recordset( NULL::foo, '[{"id":1},{"id":2}]' ) )
)
);
A better way!
But the better way is to simply not process the whole result set but only a single json value.
CREATE OR REPLACE FUNCTION test_gps
(
gps_points API_GPS_POINT
)
RETURNS BOOLEAN
LANGUAGE plpgsql
AS $$
DECLARE
api_gps_points API_GPS_POINT;
BEGIN
RAISE NOTICE 'API_GPS_POINT : %', API_GPS_POINT;
END;
$$;
And then call it like this..
SELECT test_gps(myjson)
FROM json_populate_recordset( null::foo, '[{"id":1},{"id":2}]' )
AS myjson;
Which you can see returns one type of foo.
SELECT pg_typeof(myjson)
FROM json_populate_recordset( null::foo, '[{"id":1},{"id":2}]' )
AS myjson;
Best Answer
Sample data
You have a table like,
ENUM
typeYou have an enumerated list of colors. So the easy thing here would be to use an ENUM type
Then
Now it's faster, more efficient, and cleaner. MORE WIN. MORE JOY. Etc.
ENUM
are stored as 4-byte internally.Is all you'll need.
Array Sort
But, you've got a God-given right to treat this like any other database that doesn't support ENUM types,
I think this is perfectly fine, but I believe there is a limitation here in the implementation,
Is essentially
This means you need to either (speed doesn't matter as both options perform the same)
Make the
color
column of the nativetext
. This can be done in the call, or you can actually modify the table and should. In PostgreSQL, nothing should bevarchar
without a limit (since that's just a parallel type for text), and very few things should bevarchar
with a limit (since there is no advantage)Or, you can construct the array itself as type
varchar[]
Further notes
varchar
forcolors
. Even if you insist on not using anENUM
here (though I would), that should betext
The
array_position
is a shorthand, I expect it to be substantially slower than a similar operation though