Tutorial PPSS xDB Replication Server

How to Set Up Postgres Plus xDB Replication ServerHow to Set Up Postgres Plus® xDB Replication Server A Postgres Evaluation Quick Tutorial From EnterpriseDB February 9, 2010 EnterpriseDB Corporation, 235 Littleton Road, Westford, MA 01866, USA T +1 978 589 5700 F +1 978 589 5701 E [email protected] www.enterprisedb.com Copyright © 2010 EnterpriseDB Corporation. All rights reserved. 1 How to Set Up Postgres Plus xDB Replication Server Introduction This EnterpriseDB Quick Tutorial helps you get started with the Postgres Plus Standard Server database product. It is assumed that you have already downloaded and installed Postgres Plus Standard Server on your desktop or laptop computer. This Quick Tutorial is designed to help you expedite your Technical Evaluation of Postgres Plus Standard Server. In this Quick Tutorial you will learn how to set up the Postgres Plus xDB Replication Server. For more informational assets on conducting your evaluation of Postgres Plus, visit the self-service web site, Postgres Plus Open Source Adoption. In this Quick Tutorial you will learn the following: • • • • • • Basic features of the xDB Replication Server Fundamental replication concepts and vocabulary The physical and logical components of a replication system How to install and configure the xDB Replication Server How to register publication and subscription servers How to start replication Feature Description Postgres Plus xDB Replication Server is an asynchronous, master-to-slave replication system enabling replication of tables from an Oracle® database to a Postgres Plus Standard Server database. The following are some of the features of xDB Replication Server: • Different replication methods are available for different needs. Two types of replication methods are supported. Snapshot replication takes a copy of the entire table. Synchronization replication takes only the modifications made to the table data since the last replication. Replication data can be filtered. A filter can be defined specifying a subset of rows in a source table or view to include during replication. All other rows are excluded from replication. Replication occurs asynchronously. Once xDB Replication Server is started, it can run unattended. Replication can be scheduled. Replication can be scheduled to run repeatedly at a specific time interval, daily at a specific time of day, weekly on specific days of the week, or monthly on specific months. Replication history can be viewed. Two levels of replication history are available. The date and time when a replication occurred is recorded. In addition, 2 • • • • Copyright © 2010 EnterpriseDB Corporation. All rights reserved. How to Set Up Postgres Plus xDB Replication Server the specific inserts, updates, and deletions made to synchronize a replicated table can be viewed. Replication can be distributed. The source and target databases can reside on different hosts, as well as the individual components of xDB Replication Server to optimize performance. Replication can run from the command line. In addition to a graphical user interface, a command line interface allows scripts to embed commands to configure and run replication. • • The following sections introduce you to the concepts and components of xDB Replication Server. Replication Concepts A publish and subscribe architecture is used by xDB Replication Server. The data to be made available for copying is made public as a publication. To get a copy of that data, you must subscribe to that publication. A publication is defined as a named set of tables and views within a database. The database that contains the publication is called the publication database of that publication. To get a copy of a publication, you must create a subscription. A subscription is a named association with a publication. Along with its association with a publication, a subscription is associated with a database to which the publication is to be copied. This database is called the subscription database. Replication is said to occur when xDB Replication Server initiates and completes either of the following processes: 1) applies changes that have been made to rows in the publication since the last replication occurred, to rows in tables of the subscription database (called synchronization); or 2) copies rows of the publication to empty tables of the subscription database (called a snapshot). The subscription tables are the tables in the subscription database where each subscription table is created from a corresponding table or view in the publication. Note: xDB Replication Server creates tables in the subscription database for views in the publication. The following illustrates a basic replication system: Copyright © 2010 EnterpriseDB Corporation. All rights reserved. 3 The following diagram shows the programs and database components of xDB Replication Server and how they are used to form a complete replication system.How to Set Up Postgres Plus xDB Replication Server xDB Replication Server Physical Components xDB Replication Server is not a single executable program. All rights reserved. but rather a set of programs that work together to form a replication system. Copyright © 2010 EnterpriseDB Corporation. 4 . 5 . and recovery. xDB Control database. The program that configures the publication database for replication and initiates replication. xDB Replication Server consists of the following software components: • • • • xDB Replication Console. monitoring. The Postgres Plus database containing the metadata of the replication system. (Commands written using the xDB Replication Server CLI can be used in place of the xDB Replication Console. replication. Graphical user interface that allows you to configure and run a replication system.) Publication server.How to Set Up Postgres Plus xDB Replication Server xDB Replication Server is the engine that enables replication system configuration. All rights reserved. The entire replication system is completed with the addition of the following Copyright © 2010 EnterpriseDB Corporation. scheduling. Subscription server. The program that configures the subscription database for replication. maintenance. It is suggested that you record the values you enter on these screens as they will be needed during the publication and subscription server registration process. Certain screens where you will be choosing or entering configuration options are described by the following. All rights reserved. The screens are self-explanatory.How to Set Up Postgres Plus xDB Replication Server components: • • One or more publication databases One or more subscription databases xDB Replication Server Logical Components The logical components are created when you build a replication system using the xDB Replication Console or the xDB Replication Server CLI. Creating a replication system requires the following steps: • • • • • • Register the publication server Create a publication database definition Create a publication Register the subscription server Create a subscription database definition Create a subscription Each of these steps creates a logical component that is represented by a node in the replication tree window of the xDB Replication Console. Tutorial Steps This section provides directions for installing and configuring xDB Replication Server. and then performing your first replication. Installing xDB Replication Server Use StackBuilder Plus to install xDB Replication Server. A series of setup screens leads you though the installation process. The first setup screen appears as follows: Copyright © 2010 EnterpriseDB Corporation. The logical components are stored as part of the replication system metadata in the xDB Control database. 6 . A detailed description of each of these steps is presented later in this tutorial. leave all components selected. the publication server.How to Set Up Postgres Plus xDB Replication Server When you are presented with the Select Components screen. This will install the xDB Replication Console. All rights reserved. and the subscription server. Copyright © 2010 EnterpriseDB Corporation. 7 . it will be created. If the database does not already exist. This is the xDB Control database of the replication system. enter the name of the Postgres Plus database that you want the installation process to use to store the replication system metadata. Copyright © 2010 EnterpriseDB Corporation. All rights reserved. You must also use this user name and password when you register a publication server or a subscription server running on this host.How to Set Up Postgres Plus xDB Replication Server In the following screen. 8 . The database name xdb is chosen in this example. The user name and password entered on this screen will be used by the publication server and subscription server to connect to the xDB Control database. All rights reserved.How to Set Up Postgres Plus xDB Replication Server Enter an available port on which the publication server will run. 9 . Port number 9011 is selected in this example. Copyright © 2010 EnterpriseDB Corporation. Copyright © 2010 EnterpriseDB Corporation. 10 . Port number 9012 is selected in this example. All rights reserved.How to Set Up Postgres Plus xDB Replication Server Enter an available port on which the subscription server will run. enter postgres.How to Set Up Postgres Plus xDB Replication Server For the operating system account under which the publication and subscription server are to run. All rights reserved. Copyright © 2010 EnterpriseDB Corporation. 11 . Note: On some Linux systems. 12 . Description xDB Control database connection /etc edb-repl.) The application menu contains a new item for the xDB Replication Console. the publication server and subscription server run as services named Publication Service and Subscription Service.conf (Windows) %APPDATA%\EnterpriseDB information File Name Location Copyright © 2010 EnterpriseDB Corporation. (On Microsoft Windows® systems. you will have a publication server daemon and a subscription server daemon running on your host computer.conf (Linux) information xDB Control database connection edb-repl. All rights reserved.How to Set Up Postgres Plus xDB Replication Server When the installation process is finished. You will also have a new Postgres Plus database named xdb as your xDB Control database. The following files are created that you will use during the configuration process. you may have to restart the server before you can see the xDB Replication Console choice in the application menu. ojdbc14.4-701. or restart the publication server Start. must be accessible to the Java virtual machine (JVM) running xDB Replication Server. from the Oracle download site and copy it to the lib/jdbc subdirectory of the xDB Replication Server home directory. GRANT SELECT ON edb. The Oracle system identifier (SID) is xe. $ ls edb-jdbc14. Step 3: Create a subscription database user name in the Postgres Plus database server.jar. ojdbc14. Download the Oracle JDBC driver.d xDB_Home/bin/logs xDB_Home/bin/logs Description Start.log subserver.How to Set Up Postgres Plus xDB Replication Server File Name edb-xdbpubserver edb-xdbsubserver pubserver. stop. The user name subuser is created for this purpose in this example. or restart the subscription server Publication server log file Subscription server log file Note: xDB_Home is the directory where you installed xDB Replication Server. GRANT CREATE ANY TRIGGER TO pubuser.jar.d /etc/init.jdbc3.jar Step 2: Create a publication database user name in the Oracle database. Step 1: The JDBC driver for Oracle.dept TO pubuser. The publication database user name owns the triggers and other database objects that the publication server creates in the Oracle database to manage replication. click the secondary mouse button on the Login Roles node and choose New Login Role. 13 .log Location /etc/init. Log in to the host running xDB Replication Server with a user name that has write permission to the xDB Replication Server home directory.jar . Fill in the Properties tab of the New Login Role dialog box.jar postgresql-8. GRANT SELECT ON edb.jar ojdbc14. Copyright © 2010 EnterpriseDB Corporation. In pgAdmin. $ cd /opt/PostgresPlus/EnterpriseDB-xDBReplicationServer/lib/jdbc $ cp /home/user/Downloads/ojdbc14. CREATE USER pubuser IDENTIFIED BY password. a replication system is created based upon a publication consisting of two tables named dept and emp belonging to schema edb in an Oracle database. GRANT RESOURCE TO pubuser. GRANT CONNECT TO pubuser. All rights reserved. The subscription database user name owns the database in which the replication tables reside. The following SQL statements create the user name pubuser and grant the required privileges.emp TO pubuser. stop. Prerequisite Steps For the example in this tutorial. All rights reserved. Click the OK button.How to Set Up Postgres Plus xDB Replication Server Click the Role Privileges tab and select the check boxes Superuser and Can Modify Catalog Directly. Copyright © 2010 EnterpriseDB Corporation. 14 . In pgAdmin. The database subdb is created for this purpose in this example.How to Set Up Postgres Plus xDB Replication Server Step 4: Create a Postgres Plus subscription database owned by the subscription database user. All rights reserved. Be sure to select subuser in the Owner field then click the OK button. click the secondary mouse button on the Databases node and choose New Database. 15 . Copyright © 2010 EnterpriseDB Corporation. The subscription database contains the replicated tables. Fill in the Properties tab of the New Database dialog box. 102 instead of the loopback address 127.10.10.168.168. The following example shows the modified /etc/hosts file so that the host name opensuse-vm is now associated with the network IP address 192.0.2 opensuse-vm.0.0.net opensuse-vm On some Linux systems. The hostname -i command returns the IP address associated with the host name: $ hostname -i 127.vmplanet. you may need to restart the network service after you have Copyright © 2010 EnterpriseDB Corporation.2 If the loopback address 127.How to Set Up Postgres Plus xDB Replication Server Step 5 (For Linux only): Modifications may be necessary to the /etc/hosts file.vmplanet. 16 .0.net opensuse-vm opensuse-vm. edit the /etc/hosts file so that the network IP address is associated with the host name instead.2: 192. All rights reserved.0.0.102 #127.x is returned such as in the preceding example.0.x. 168.How to Set Up Postgres Plus xDB Replication Server modified the /etc/hosts file. The values you substitute for pub_ipaddr and sub_ipaddr are the network IP addresses where the publication server and the subscription server are running. the following entries are needed to allow the publication server and the subscription server access to the xDB Control database: host control_dbname control_dbuser host control_dbname control_dbuser pub_ipaddr/32 sub_ipaddr/32 md5 md5 The values you substitute for control_dbname and control_dbuser are the entries made in the Control Database and User Name fields.d/networking restart The following example illustrates the service network command: $ su root Password: $ service network restart The hostname -i command now returns the network IP address of the host: $ hostname -i 192. this file is located in the data subdirectory of the Postgres Plus home directory.102 Step 6: Add entries to the host-based authentication file. This may be done a number of different ways depending upon the Linux system you are using as shown by the following variations: service network restart /etc/init. In general. respectively.d/networking restart sudo /etc/init. on the Postgres Plus Standard Server Installation Details screen during xDB Replication Server installation. On Linux systems. use the network IP address returned by the hostname -i command you used in Step 5.10. On Windows systems. was created in Step 4. In general. the following entries are needed to allow the publication server and the subscription server access to the subscription database: host sub_dbname host sub_dbname sub_dbuser sub_dbuser pub_ipaddr/32 sub_ipaddr/32 md5 md5 The value you substitute for sub_dbname. All rights reserved. 17 . pg_hba. In a default Postgres Plus installation. to allow access to the xDB Control database and the subscription database from the publication server and subscription server.conf. open a Command Prompt window and use the ipconfig command to obtain the host’s network IP address. The value you Copyright © 2010 EnterpriseDB Corporation. The publication server runs as a service named Publication Service. Reload the configuration file after making the modifications.0.1/32 # IPv6 local connections: host all all ::1/128 Note: Since the publication server and the subscription server are running on the same host. and then Services. only one entry is needed per database to allow access by both the publication server and the subscription server.7 2.0 3068 736 pts/3 S+ 15:53 0:00 grep pubserver For Windows hosts open Control Panel. Copyright © 2010 EnterpriseDB Corporation. you must restart the publication server.conf file appears as follows: # TYPE DATABASE USER CIDR-ADDRESS METHOD md5 md5 md5 md5 md5 # "local" is for Unix domain socket connections only local all all # IPv4 local connections: host xdb postgres 192.168. All rights reserved.0-sun-1.6. to the xDB Replication Server lib/jdbc subdirectory.u6/jre/bin/java -jar edb-repserver. ojdbc14. was created in Step 3.102/32 host all all 127.2 192444 23188 pts/3 Sl 15:50 0:01 /usr/lib/jvm/java-1.0 0.How to Set Up Postgres Plus xDB Replication Server substitute for sub_dbuser.10.jar pubserver 9011 postgres 4895 0.10. Note: If the publication server has not been restarted since copying the Oracle JDBC driver.102/32 host subdb subuser 192.jar. the resulting pg_hba. Administrative Tools. 18 . This will put the modified pg_hba. Registering the Publication Server and Creating a Publication Step 1: Start the publication server if it is not already running. On Linux hosts you can verify the publication server is running by using the following command: ps aux | grep pubserver This is shown by the following: $ ps aux | grep pubserver postgres 4820 0.0.168. For this example.6.conf file into effect. Choose Reload Configuration from the Postgres Plus application menu.0. $ su postgres Password: $ /etc/init. which is operating system user postgres in this example. Use the Start or Restart link for the service. Open the xDB Replication Console from the system’s application menu. run the edb-xdbpubserver script with the start option as shown by the following: $ /etc/init. Step 2: Register the publication server. 19 . and then Services. Run the script as the publication/subscription service account that you chose during installation.d/edb-xdbpubserver start Password: For Windows hosts open Control Panel. run the edbxdbpubserver script located in the /etc/init. If the publication server is not running.d directory with the restart option. Administrative Tools. All rights reserved. Copyright © 2010 EnterpriseDB Corporation.How to Set Up Postgres Plus xDB Replication Server If the publication server is running and you wish to restart it.d/edb-xdbpubserver restart Password: Enter the password for operating system user name postgres when prompted. 20 . The Register Publication Server dialog box appears. All rights reserved. Copyright © 2010 EnterpriseDB Corporation.How to Set Up Postgres Plus xDB Replication Server Step 3: Click the secondary mouse button on the Replication Servers node and choose Register Publication Server. Copyright © 2010 EnterpriseDB Corporation. Database superuser name that the publication server uses to connect to the xDB Control database. This is the network IP address determined in Step 5 of Prerequisite Steps. User Name. 21 . Save login information.) Port. Password you specified on the Postgres Plus Standard Server Installation Details screen for the database superuser. This is the user name you specified on the Postgres Plus Standard Server Installation Details screen. This is the port number you specified on the Publication Server Details screen. Port number the publication server is using.How to Set Up Postgres Plus xDB Replication Server Enter the values you supplied during the installation of xDB Replication Server unless otherwise specified. All rights reserved. • • • • • Host. Check this box so you do not have to re-enter the User Name and Password each time you restart the xDB Replication Console. A Publication Server node appears in the replication tree in the xDB Replication Console window. Click the Register button after you have filled in the fields. (Do not use localhost for this field. Password. Network IP address of the host running the publication server. Copyright © 2010 EnterpriseDB Corporation. A Publication Database node is added to the replication tree under the Publication Server node. IP address of the host on which the Oracle database server is running. If Test Result: Success appears. Fill in the following fields: • • • • • • Database Type. Password of the database user. All rights reserved. The publication database user name created in Step 2 of Prerequisite Steps.How to Set Up Postgres Plus xDB Replication Server Step 4: Create a publication database definition. Be sure Oracle is selected. The Oracle service ID (SID) of the publication database. The Add Database dialog box appears. Note: The Oracle database server must be running and accepting client connections for this and subsequent steps. Port. User. Step 5: Click the Test button. click the OK button. Service ID. Port on which the Oracle database server is listening for connections. Password. Click the secondary mouse button on the Publication Server node and choose Add Database. 22 . Host. then click the Save button. If the Snapshot-Only Replication box was checked. Check this box if you want to include all tables and views in the Publish list in the publication. Check the box if replication is to be done by snapshot only.How to Set Up Postgres Plus xDB Replication Server Step 6: Click the secondary mouse button on the Publication Database node and choose Create Publication. Tables included in publications on which synchronization replication is to be used must have primary keys. Enter a name that is unique amongst all publications. All rights reserved. • • Copyright © 2010 EnterpriseDB Corporation. Check the boxes next to the tables that are to be included in the publication. Publish. 23 . Tables included in a snapshot-only publication do not require a primary key. Fill in the following fields: • • Publication Name. then views would appear in the Publish list as well. Select All. Snapshot-only replication. The Create Publication dialog box appears. A Publication node is added to the replication tree. Registering the Subscription Server and Creating a Subscription Step 1: Start the subscription server if it is not already running. Repeat the same process Copyright © 2010 EnterpriseDB Corporation. All rights reserved.How to Set Up Postgres Plus xDB Replication Server Click the Create button. 24 . d directory to restart or start the subscription server.u6/jre/bin/java -jar edb-repserver. 25 .6. Enter the values you supplied during the installation of xDB Replication Server unless otherwise specified.jar subserver 9012 vmplanet 23507 0. Use the Start or Restart link for the service named Subscription Service.2 192236 23300 ? Sl 15:02 0:02 /usr/lib/jvm/java-1. to the xDB Replication Server lib/jdbc subdirectory. Step 2: Open the xDB Replication Console.0-sun-1. Copyright © 2010 EnterpriseDB Corporation. Note: If the subscription server has not been restarted since copying the Oracle JDBC driver.1 2. Click the secondary mouse button on the Replication Servers node and choose Register Subscription Server.0.6. you must restart the subscription server. and then Services.d/edb-xdbsubserver restart Password: Enter the password for operating system user name postgres when prompted.0 0. Administrative Tools. Use script /etc/init. This is illustrated by the following: $ ps aux | grep subserver postgres 22614 0.0 3068 740 pts/3 S+ 15:25 0:00 grep subserver $ su postgres Password: $ /etc/init.d/edb-xdbsubserver located in the /etc/init. ojdbc14.How to Set Up Postgres Plus xDB Replication Server as in Step 1 of Registering the Publication Server and Creating a Publication. The Register Subscription Server dialog box appears. All rights reserved. For Windows hosts open Control Panel.jar. Copyright © 2010 EnterpriseDB Corporation.) Port. This is the user name you specified on the Postgres Plus Standard Server Installation Details screen.How to Set Up Postgres Plus xDB Replication Server • • • • • Host. Password you specified on the Postgres Plus Standard Server Installation Details screen for the database superuser. Check this box so you do not have to re-enter the User Name and Password each time you restart the xDB Replication Console. Password. (Do not use localhost for this field. This is the network IP address determined in Step 5 of Prerequisite Steps. All rights reserved. This is the port number you specified on the Subscription Server Details screen. 26 . A Subscription Server node appears in the replication tree. Click the Register button after you have filled in the fields. Network IP address of the host running the subscription server. Database superuser name that the subscription server uses to connect to the xDB Control database. Save login information. User Name. Port number the subscription server is using. 27 . The name of the subscription database created in Step 4 of Prerequisite Steps. Password. Be sure Postgres Plus is selected. Database. IP address of the host on which the subscription database server is running. A Subscription Database node is added to the replication tree under the Subscription Server node. Fill in the following fields: • • • • • • Database Type.How to Set Up Postgres Plus xDB Replication Server Step 3: Create a subscription database definition. The Add Database dialog box appears. Click the secondary mouse button on the Subscription Server node and choose Add Database. User. Port. Host. Password of the database user. Step 4: Click the Test button. click the OK button. Copyright © 2010 EnterpriseDB Corporation. All rights reserved. Port on which the database server is listening for connections. The subscription database user name created in Step 3 of Prerequisite Steps. then click the Save button. If Test Result: Success appears. Port used by the publication server. 28 . Password. User Name. Choose the publication to which to subscribe. User name of the registered publication server. Enter a name for the subscription that is unique amongst all subscription names. Unless otherwise noted. Port. All rights reserved. enter the same values you gave in Step 3 of Registering the Publication Server and Creating a Publication. Click the Load button to get a list of available publications. Host. Password of the user.How to Set Up Postgres Plus xDB Replication Server Step 5: Click the secondary mouse button on the Subscription Database node and choose Create Subscription. The Create Subscription dialog box appears. • • • • • • Subscription Name. Publication Name. Network IP address of the publication server that appears as the parent node of the publication to be subscribed to when viewed in the replication tree. Copyright © 2010 EnterpriseDB Corporation. A Subscription node is added to the replication tree. All rights reserved. 29 . Snapshot Replication The very first replication from a publication to a subscription must be done using snapshot replication. Once the first snapshot is successfully taken.How to Set Up Postgres Plus xDB Replication Server Click the Create button. subsequent Copyright © 2010 EnterpriseDB Corporation. Copyright © 2010 EnterpriseDB Corporation. The Snapshot dialog box appears.How to Set Up Postgres Plus xDB Replication Server replications can be done using either the synchronization method (if the publication was not initially defined as a snapshot-only publication) or the snapshot method. Step 2: Click the Snapshot button. Step 1: Click the secondary mouse button on the Subscription node and choose Snapshot. All rights reserved. 30 . How to Set Up Postgres Plus xDB Replication Server Step 3: Snapshot Taken Successfully will appear if the snapshot was successful. Click the OK button. 31 .log Copyright © 2010 EnterpriseDB Corporation. All rights reserved. The status messages of each snapshot are also saved in log files named as follows: xDB_HOME/bin/logs/mtk_yyyymmddhhmiss. If the snapshot was not successful. scroll through the messages in the Snapshot dialog box window. How to Set Up Postgres Plus xDB Replication Server The publication has now been replicated to the subscription database. 32 . The Synchronize dialog box appears. Copyright © 2010 EnterpriseDB Corporation. Synchronization Replication Step 1: Click the secondary mouse button on the Subscription node and choose Synchronize. All rights reserved. which is edb in this example. The subscription tables appear under a schema with the same name as the Oracle schema. Click the OK button. Copyright © 2010 EnterpriseDB Corporation. All rights reserved. If the synchronization was not successful. Step 3: Subscription Synchronized Successfully will appear if the synchronization was successful. scroll through the messages in the Synchronize dialog box window.How to Set Up Postgres Plus xDB Replication Server Step 2: Click the Synchronize button. 33 . How to Set Up Postgres Plus xDB Replication Server Modifications have now been applied to the rows in the subscription tables reflecting changes made to the publication since the last replication. Conclusion In this Quick Tutorial you learned how to set up Postgres Plus xDB Replication Server. 34 . You should now be able to proceed confidently with a Technical Evaluation of Postgres Plus knowing that your data can be easily and efficiently replicated from an Oracle database to a Postgres Plus database. All rights reserved. The following resources should help you move on with this step: • • • • • • • • • • PostgreSQL Replication Solutions Replication Training Classes from EnterpriseDB High Availability Database Solutions for PostgreSQL and Postgres Plus Replication Packaged Services from EnterpriseDB Postgres Plus Technical Evaluation Guide Postgres Plus Getting Started resources Postgres Plus Quick Tutorials Postgres Plus User Forums Postgres Plus Documentation Postgres Plus Webinars Copyright © 2010 EnterpriseDB Corporation.