redshift COPY commands can have identity columns
Recently I learned that COPY commandsd can have a flag called EXPLICIT_IDS when
working with Redshift. This allows identity based insert operations to occur,
which is helpful for ingest jobs.
Additionally, the flag can be passed for tables that do not have an identity column,
which will still operate just fine. This is great for if we have lots of tables that
we're doing COPY operations on, but don't want to specifically do some backend logic
to handle which ones need identity based checking explicitly.
The "laziness" factor of this seems to be beneficial for bulk operations.
References
COPYdocumentationEXPLICIT_IDSdocumentationCOPYcommand column mapping documentation - The important bits are below.- If an
IDENTITYcolumn is included in the column list, thenEXPLICIT_IDSmust also be specified; if
IDENTITYcolumn is omitted, thenEXPLICIT_IDScan't be specified. If no column list is specified, the command behaves as if a complete, in-order column list was specified, withIDENTITYcolumns omitted ifEXPLICIT_IDSwas also not specified.- If a column is defined with
GENERATED BY DEFAULT AS IDENTITY, then it can be copied.
EXPLICIT_IDSoption isn't required.COPYdoesn't update the identity high watermark. For more information, seeGENERATED BY DEFAULT AS IDENTITY.- If an