Similar to this previous question for a one-to-one mapping I need a solution for multiple columns in source and destination.
Still working with Postgres 9.4.4, the query and schema are modified and are as below:
Let's say I have these two tables Table1
and Table2
:
Create table Table1(col1 int, col2 varchar(100), col3 varchar(100));
Create table Table2(col1 int, col2 varchar(100), col3 varchar(100));
There is another table Table3
storing a formula for migrating data from Table1
to Table2
:
CREATE TABLE Table3 (
tbl_src character varying(200),
col_src character varying(500),
tbl_des character varying(200),
col_des character varying(100),
condition character varying(500)
);
Insert into Table3(tbl_src, col_src, tbl_des, col_des, condition)
VALUES ('Table1','col1','Table2','col1', 'WHERE col1>=1')
, ('Table1','col2','Table2','col2', NULL)
, ('Table1','col3','Table2','col3', NULL);
How to compile this formula in a dynamic query and insert into the destination table?
The basic query to build the statement for multiple columns dynamically - ignoring the condition
column:
SELECT format(
'INSERT INTO %I (%s) SELECT %s FROM %I'
, tbl_des
, string_agg(quote_ident(col_des), ', ')
, string_agg(quote_ident(col_src), ', ')
, tbl_src) AS sql
FROM table3
WHERE tbl_des = 'Table2'
AND tbl_src = 'Table1'
GROUP BY tbl_des, tbl_src;
Result:
INSERT INTO "Table2" (col1, col2, col3)
SELECT CASE col1, col2, col3 FROM "Table1"
This assumes a single source and a single destination table. Or things get more complicated. I added WHERE
conditions to make that clear. The notes on case-sensitivity I added to my previous answer still apply.
The above still ignores the condition
. First of all, don't include the keyword WHERE
in the condition
column. That's just noise and not helpful:
INSERT INTO Table3(tbl_src, col_src, tbl_des, col_des, condition)
VALUES ('Table1','col1','Table2','col1', 'col1>=1') -- without WHERE!
, ('Table1', ...
Warning
This approach is inherently unsafe. The condition
holds expressions that need to be concatenated "as is", so you are completely open to SQL injection attacks. You need to make sure that untrusted users cannot write to table3
in any way to avoid this.
Building on this, and assuming each condition shall apply to its respective column only, we can solve it by wrapping the column in a CASE
expression:
SELECT format(
'INSERT INTO %I (%s) SELECT %s FROM %I'
, tbl_des
, string_agg(quote_ident(col_des), ', ')
, string_agg(
CASE WHEN condition IS NULL
THEN quote_ident(col_src)
ELSE format('CASE WHEN %s THEN %I END'
, condition, col_src) -- condition is unsafe!
END, ', ')
, tbl_src) AS sql
FROM table3
WHERE tbl_des = 'Table2'
AND tbl_src = 'Table1'
GROUP BY tbl_des, tbl_src;
Produces a statement of the form:
INSERT INTO "Table2" (col1, col2, col3)
SELECT CASE WHEN col1>=1 THEN col1 END, col2, col3 FROM "Table1"
Or, like you added in a later comment, conditions can apply to whole rows. Logically, this is another grey area. The condition is stored with a particular column, but applies to the whole row ...
Be that as it may, you can add universal conditions in a WHERE
clause.
SELECT format(
'INSERT INTO %I (%s) SELECT %s FROM %I%s'
, tbl_des
, string_agg(quote_ident(col_des), ', ')
, string_agg(quote_ident(col_src), ', ')
, tbl_src
, ' WHERE ' || string_agg(condition, ' AND ')) AS sql
FROM table3
WHERE tbl_des = 'Table2'
AND tbl_src = 'Table1'
GROUP BY tbl_des, tbl_src;
Conditions are ANDed, and the WHERE
clause is only appended if there are any conditions - else the resulting NULL values swallows the added keyword in the expression ' WHERE ' || string_agg(condition, ' AND '))
Use this in a DO
command or plpgsql function to execute dynamically like instructed in my previous answer:
- How to INSERT INTO table from dynamic query?
Basic plpgsql function:
CREATE OR REPLACE FUNCTION f_test()
RETURNS void AS
$func$
BEGIN
EXECUTE (
SELECT format(
'INSERT INTO %I (%s) SELECT %s FROM %I%s'
, tbl_des
, string_agg(quote_ident(col_des), ', ')
, string_agg(quote_ident(col_src), ', ')
, tbl_src
, ' WHERE ' || string_agg(condition, ' AND ')) AS sql
FROM table3
WHERE tbl_des = 'Table2'
AND tbl_src = 'Table1'
GROUP BY tbl_des, tbl_src
);
END
$func$ LANGUAGE plpgsql;
SQL Fiddle.