EDB Clone Schema v13
EDB Clone Schema is an extension module for Advanced Server that allows you to copy a schema and its database objects from a local or remote database (the source database) to a receiving database (the target database).
The source and target databases can be the same physical database, or different databases within the same database cluster, or separate databases running under different database clusters on separate database server hosts.
Use the following functions with EDB Clone Schema:
- localcopyschema. This function makes a copy of a schema and its database objects from a source database back into the same database (the target), but with a different schema name than the original. Use this function when the original source schema and the resulting copy are to reside within the same database. See localcopyschema for information on the
localcopyschema
function. - localcopyschema_nb. This function performs the same purpose as
localcopyschema
, but as a background job, thus freeing up the terminal from which the function was initiated. This is referred to as a non-blocking function. See localcopyschema_nb for information on thelocalcopyschema_nb
function. - remotecopyschema. This function makes a copy of a schema and its database objects from a source database to a different target database. Use this function when the original source schema and the resulting copy are to reside in two, separate databases. The separate databases can reside in the same, or in different Advanced Server database clusters. See remotecopyschema for information on the
remotecopyschema
function. - remotecopyschema_nb. This function performs the same purpose as
remotecopyschema
, but as a background job, thus freeing up the terminal from which the function was initiated. This is referred to as a non-blocking function. See remotecopyschema_nb for information on theremotecopyschema_nb
function. - process_status_from_log. This function displays the status of the cloning functions. The information is obtained from a log file that must be specified when a cloning function is invoked. See process_status_from_log for information on the
process_status_from_log
function. - remove_log_file_and_job. This function deletes the log file created by a cloning function. This function can also be used to delete a job created by the non-blocking form of the function. See remove_log_file_and_job for information on the
remove_log_file_and_job
function. - create_clone_log_dir. This function creates a directory to store all the log files.
- grant_clone_schema_privileges. This function grants the privileges to clone the schema to a non-superuser.
- revoke_clone_schema_privileges. This function revokes the privileges to clone the schema from a non-superuser.
The database objects that can be cloned from one schema to another are the following:
- Data types
- Tables including partitioned tables, excluding foreign tables
- Indexes
- Constraints
- Sequences
- View definitions
- Materialized views
- Private synonyms
- Table triggers, but excluding event triggers
- Rules
- Functions
- Procedures
- Packages
- Comments for all supported object types
- Access control lists (ACLs) for all supported object types
The following database objects cannot be cloned:
- Large objects (Postgres
LOBs
andBFILEs
) - Logical replication attributes for a table
- Database links
- Foreign data wrappers
- Foreign tables
- Event triggers
- Extensions (For cloning objects that rely on extensions, see the third bullet point in the following limitations list.)
- Row level security
- Policies
- Operator class
In addition, the following limitations apply:
- EDB Clone Schema is supported on Advanced Server only when a dialect of
Compatible with Oracle
is specified on the Advanced ServerDialect
dialog during installation, or when the--redwood-like
keywords are included during a text mode installation or cluster initialization. - The source code within functions, procedures, triggers, packages, etc., are not modified after being copied to the target schema. If such programs contain coded references to objects with schema names, the programs may fail upon invocation in the target schema if such schema names are no longer consistent within the target schema.
- Cross schema object dependencies are not resolved. If an object in the target schema depends upon an object in another schema, this dependency is not resolved by the cloning functions.
- For remote cloning, if an object in the source schema is dependent upon an extension, then this extension must be created in the public schema of the remote database before invoking the remote cloning function.
- At most, 16 copy jobs can run in parallel to clone schemas, whereas each job can have at most 16 worker processes to copy table data in parallel.
- Queries being run by background workers cannot be cancelled.
The following section describes how to set up EDB Clone Schema on the databases.
Setup Process
Several extensions along with the PL/Perl language must be installed on any database to be used as the source or target database by an EDB Clone Schema function.
In addition, some configuration parameters in the postgresql.conf
file of the database servers may benefit from some modification.
The following is the setup instructions for these requirements.
Installing Extensions and PL/Perl
The following describes the steps to install the required extensions and the PL/Perl language.
These steps must be performed on any database to be used as the source or target database by an EDB Clone Schema function.
Step 1: The log directory is required to store all the log files. Create the log directory as a superuser:
SELECT edb_util.create_clone_log_dir();
When successful, the command returns true
.
Step 2: The following extensions must be installed on the database:
postgres_fdw
dblink
adminpack
pgagent
Ensure that pgAgent is installed before creating the pgagent
extension. On Linux, you can use the edb-asxx-pgagent
RPM package where xx
is the Advanced Server version number to install pgAgent. On Windows, use StackBuilder Plus to download and install pgAgent.
The previously listed extensions can be installed by the following commands if they do not already exist:
CREATE EXTENSION postgres_fdw SCHEMA public; CREATE EXTENSION dblink SCHEMA public; CREATE EXTENSION adminpack; CREATE EXTENSION pgagent;
For more information, see the CREATE EXTENSION command documentation.
Step 3: Modify the postgresql.conf
file.
Modify the postgresql.conf
file by adding $libdir/parallel_clone
to the shared_preload_libraries
configuration parameter as shown by the following example:
shared_preload_libraries = '$libdir/dbms_pipe,$libdir/dbms_aq,$libdir/parallel_clone'
To load the libraries with clone schema support, restart the database server.
Step 4: The Perl Procedural Language (PL/Perl) must be installed on the database and the CREATE TRUSTED LANGUAGE plperl
command must be run. For Linux, install PL/Perl using the edb-asxx-server-plperl
RPM package where xx
is the Advanced Server version number. For Windows, use the EDB Postgres Language Pack. For information, see EDB Language Pack documentation.
Step 5: Connect to the database as a superuser and run the following command:
CREATE TRUSTED LANGUAGE plperl;
For more information on EDB Language Pack, see EDB Postgres Language Pack.
Setting Configuration Parameters
The following sections describe configuration parameters that may need to be altered in the postgresql.conf
file.
Performance Configuration Parameters
You may need to tune the system for copying a large schema as part of one transaction.
Tuning of configuration parameters is for the source database server referenced in a cloning function.
The configuration parameters in the postgresql.conf
file that may need to be tuned include the following:
- work_mem. Specifies the amount of memory to be used by internal sort operations and hash tables before writing to temporary disk files.
- maintenance_work_mem. Specifies the maximum amount of memory to be used by maintenance operations, such as
VACUUM
,CREATE INDEX
, andALTER TABLE ADD FOREIGN KEY
. - max_worker_processes. Sets the maximum number of background processes that the system can support.
- checkpoint_timeout. Maximum time between automatic WAL checkpoints, in seconds.
- checkpoint_completion_target. Specifies the target of checkpoint completion, as a fraction of total time between checkpoints.
- checkpoint_flush_after. Whenever more than
checkpoint_flush_after
bytes have been written while performing a checkpoint, attempt to force the OS to issue these writes to the underlying storage. - max_wal_size. Maximum size to let the WAL grow to between automatic WAL checkpoints.
- max_locks_per_transaction. This parameter controls the average number of object locks allocated for each transaction; individual transactions can lock more objects as long as the locks of all transactions fit in the lock table.
For information about the configuration parameters, see the PostgreSQL core documentation at:
https://www.postgresql.org/docs/current/static/runtime-config.html
Status Logging
Status logging by the cloning functions creates log files in the directory specified by the log_directory
parameter in the postgresql.conf
file for the database server to which you are connected when invoking the cloning function.
The default location is PGDATA/log
as shown by the following:
#log_directory = 'log' # directory where log files are written, # can be absolute or relative to PGDATA
This directory must exist prior to running a cloning function.
The name of the log file is determined by what you specify in the parameter list when invoking the cloning function.
To display the status from a log file, use the process_status_from_log
function.
To delete a log file, use the remove_log_file_and_job
function, or simply navigate to the log directory and delete it manually.
Installing EDB Clone Schema
The following are the directions for installing EDB Clone Schema.
These steps must be performed on any database to be used as the source or target database by an EDB Clone Schema function.
Step 1: If you had previously installed an older version of the edb_cloneschema
extension, then you must run the following command:
DROP EXTENSION parallel_clone CASCADE;
This command also drops the edb_cloneschema
extension.
Step 2: Install the extensions using the following commands:
CREATE EXTENSION parallel_clone SCHEMA public; CREATE EXTENSION edb_cloneschema;
Make sure you create the parallel_clone
extension before creating the edb_cloneschema
extension.
Creating Log directory
The Log directory is required to store all the log files.
After creating the extensions, to create the log directory, execute the following statement as a superuser:
SELECT edb_util.create_clone_log_dir();
When successful, the command returns true
.
Creating the Foreign Servers and User Mappings
When using one of the local cloning functions, localcopyschema
or localcopyschema_nb
, one of the required parameters includes a single, foreign server for identifying the database server along with its database that is the source and the receiver of the cloned schema.
When using one of the remote cloning functions, remotecopyschema
or remotecopyschema_nb
, two of the required parameters include two foreign servers. The foreign server specified as the first parameter identifies the source database server along with its database that is the provider of the cloned schema. The foreign server specified as the second parameter identifies the target database server along with its database that is the receiver of the cloned schema.
For each foreign server, a user mapping must be created. When a selected database user invokes a cloning function, that database user who invokes the function must have been mapped to a database user name and password that has access to the foreign server that is specified as a parameter in the cloning function.
For general information about foreign data, foreign servers, and user mappings, see the PostgreSQL core documentation at:
https://www.postgresql.org/docs/current/static/ddl-foreign-data.html
The following two sections describe how these foreign servers and user mappings are defined.
Foreign Server and User Mapping for Local Cloning Functions
For the localcopyschema
and localcopyschema_nb
functions, the source and target schemas are both within the same database of the same database server. Thus, only one foreign server must be defined and specified for these functions. This foreign server is also referred to as the local server.
This server is referred to as the local server because this server is the one to which you must be connected when invoking the localcopyschema
or localcopyschema_nb
function.
The user mapping defines the connection and authentication information for the foreign server.
This foreign server and user mapping must be created within the database of the local server in which the cloning is to occur.
When invoking an EDB Clone Schema function, the database user for whom the user mapping is defined must have the required privileges and be connected to the local server.
The following example creates the foreign server for the database containing the schema to be cloned, and to receive the cloned schema as well.
CREATE SERVER local_server FOREIGN DATA WRAPPER postgres_fdw OPTIONS( host 'localhost', port '5444', dbname 'edb' );
For more information about using the CREATE SERVER
command, see the PostgreSQL core documentation at:
https://www.postgresql.org/docs/current/static/sql-createserver.html
The user mapping for this server is the following: