Professional Documents
Culture Documents
12c (12.3.0.1)
E81607-07
September 2018
Oracle Fusion Middleware Administering Oracle GoldenGate, 12c (12.3.0.1)
E81607-07
Copyright © 2013, 2018, Oracle and/or its affiliates. All rights reserved.
This software and related documentation are provided under a license agreement containing restrictions on
use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your
license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify,
license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means.
Reverse engineering, disassembly, or decompilation of this software, unless required by law for
interoperability, is prohibited.
The information contained herein is subject to change without notice and is not warranted to be error-free. If
you find any errors, please report them to us in writing.
If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on
behalf of the U.S. Government, then the following notice is applicable:
U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software,
any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are
"commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-
specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the
programs, including any operating system, integrated software, any programs installed on the hardware,
and/or documentation, shall be subject to license terms and license restrictions applicable to the programs.
No other rights are granted to the U.S. Government.
This software or hardware is developed for general use in a variety of information management applications.
It is not developed or intended for use in any inherently dangerous applications, including applications that
may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you
shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its
safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this
software or hardware in dangerous applications.
Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of
their respective owners.
Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are
used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron,
the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro
Devices. UNIX is a registered trademark of The Open Group.
This software or hardware and documentation may provide access to or information about content, products,
and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly
disclaim all warranties of any kind with respect to third-party content, products, and services unless otherwise
set forth in an applicable agreement between you and Oracle. Oracle Corporation and its affiliates will not be
responsible for any loss, costs, or damages incurred due to your access to or use of third-party content,
products, or services, except as set forth in an applicable agreement between you and Oracle.
Contents
Preface
Audience xvii
Documentation Accessibility xvii
Related Information xvii
Conventions xviii
iii
3.8.1 Stopping Manager on UNIX and Linux 3-5
3.8.2 Stopping Manager on Windows 3-5
iv
4.5.4.3 Applying Data from Multiple Containers or Catalogs 4-21
4.5.4.4 Specifying a Default Container or Catalog 4-21
4.5.5 Specifying Case-Sensitive Database Object Names 4-21
4.5.6 Using Wildcards in Database Object Names 4-22
4.5.6.1 Rules for Using Wildcards for Source Objects 4-23
4.5.6.2 Rules for Using Wildcards for Target Objects 4-24
4.5.6.3 Fallback Name Mapping 4-24
4.5.6.4 Wildcard Mapping from Pre-11.2.1 Trail Version 4-25
4.5.6.5 Asterisks or Question Marks as Literals in Object Names 4-25
4.5.6.6 How Wildcards are Resolved 4-25
4.5.6.7 Excluding Objects from a Wildcard Specification 4-25
4.5.7 Differentiating Case-Sensitive Column Names from Literals 4-25
v
6.2.4 Additional Information 6-2
6.3 Creating a Data Distribution Configuration 6-2
6.3.1 Source System 6-3
6.3.2 Target Systems 6-5
vi
9 Configuring Oracle GoldenGate for Active-Active High Availability
9.1 Overview of an Active-Active Configuration 9-1
9.2 Considerations for an Active-Active Configuration 9-1
9.2.1 TRUNCATES 9-2
9.2.2 Application Design 9-2
9.2.3 Keys 9-2
9.2.4 Triggers and Cascaded Deletes 9-3
9.2.5 Database-Generated Values 9-3
9.2.6 Database Configuration 9-4
9.3 Preventing Data Looping 9-4
9.3.1 Preventing the Capture of Replicat Operations 9-4
9.3.1.1 Preventing the Capture of Replicat Transactions (Oracle) 9-4
9.3.1.2 Preventing Capture of Replicat Transactions (Other Databases) 9-5
9.3.2 Identifying Replicat Transactions 9-5
9.3.2.1 DB2 z/OS, DB2 LUW, and DB2 for i 9-5
9.3.2.2 MySQL 9-5
9.3.2.3 Oracle 9-5
9.3.2.4 SQL Server 9-6
9.3.3 Replicating DDL in a Bi-directional Configuration 9-6
9.4 Managing Conflicts 9-6
9.5 Additional Information 9-8
9.6 Creating an Active-Active Configuration 9-8
9.6.1 Prerequisites on Both Systems 9-9
9.6.2 Configuration from Primary System to Secondary System 9-9
9.6.3 Configuration from Secondary System to Primary System 9-12
vii
10.2.4.2 GGSCI 10-8
10.2.4.3 Column-conversion Functions 10-8
10.3 CDR Example 1: All Conflict Types with USEMAX, OVERWRITE, DISCARD 10-8
10.3.1 Table Used in this Example 10-9
10.3.2 MAP Statement with Conflict Resolution Specifications 10-9
10.3.3 Description of MAP Statement 10-9
10.3.4 Error Handling 10-10
10.3.5 INSERTROWEXISTS with the USEMAX Resolution 10-10
10.3.6 UPDATEROWEXISTS with the USEMAX Resolution 10-11
10.3.7 UPDATEROWMISSING with OVERWRITE Resolution 10-12
10.3.8 DELETEROWMISSING with DISCARD Resolution 10-13
10.3.9 DELETEROWEXISTS with OVERWRITE Resolution 10-14
10.4 CDR Example 2: UPDATEROWEXISTS with USEDELTA and USEMAX 10-15
10.4.1 Table Used in this Example 10-15
10.4.2 MAP Statement 10-16
10.4.3 Description of MAP Statement 10-16
10.4.4 Error Handling 10-16
10.5 CDR Example 3: UPDATEROWEXISTS with USEDELTA, USEMAX, and
IGNORE 10-18
10.5.1 Table Used in this Example 10-18
10.5.2 MAP Statement 10-18
10.5.3 Description of MAP Statement 10-18
10.5.4 Error Handling 10-19
viii
11.6.2.2 Using USEDEFAULTS to Enable Default Column Mapping 11-7
11.6.2.3 Determining Whether COLMAP Requires a Data-definitions File 11-9
11.6.3 Configuring Global Column Mapping with COLMATCH 11-9
11.6.4 Understanding Default Column Mapping 11-12
11.6.5 Mapping Data Types from Column to Column 11-13
11.6.5.1 Numeric Columns 11-13
11.6.5.2 Character-type Columns 11-13
11.6.5.3 Datetime Columns 11-13
11.7 Selecting and Filtering Rows 11-14
11.7.1 Selecting Rows with a FILTER Clause 11-14
11.7.2 Selecting Rows with a WHERE Clause 11-17
11.7.3 Considerations for Selecting Rows with FILTER and WHERE 11-17
11.7.3.1 Ensuring Data Availability for Filters 11-18
11.7.3.2 Comparing Column Values 11-18
11.7.3.3 Testing for NULL Values 11-18
11.8 Retrieving Before and After Values 11-19
11.9 Selecting Columns 11-19
11.10 Selecting and Converting SQL Operations 11-20
11.11 Using Transaction History 11-20
11.12 Testing and Transforming Data 11-21
11.12.1 Handling Column Names and Literals in Functions 11-23
11.12.2 Using the Appropriate Function 11-23
11.12.3 Transforming Dates 11-23
11.12.4 Performing Arithmetic Operations 11-24
11.12.4.1 Omitting @COMPUTE 11-24
11.12.5 Manipulating Numbers and Character Strings 11-25
11.12.6 Handling Null, Invalid, and Missing Data 11-25
11.12.6.1 Using @COLSTAT 11-25
11.12.6.2 Using @COLTEST 11-26
11.12.6.3 Using @IF 11-26
11.12.7 Performing Tests 11-26
11.12.7.1 Using @CASE 11-26
11.12.7.2 Using @VALONEOF 11-27
11.12.7.3 Using @EVAL 11-27
11.13 Using Tokens 11-27
11.13.1 Defining Tokens 11-27
11.13.2 Using Token Data in Target Tables 11-28
ix
12 Associating Replicated Data with Metadata
12.1 Overview 12-1
12.2 Understanding Data Definition Files 12-1
12.2.1 Contents of the Definitions File 12-1
12.2.2 Which Definitions File Type to Use, and Where 12-2
12.2.3 Understanding the Effect of Character Sets on Definitions Files 12-2
12.2.3.1 Confining Data Mapping and Conversion to the Replicat Process 12-2
12.2.3.2 Avoiding File Corruptions Due to Operating System Character
Sets 12-3
12.2.3.3 Changing the Character Set of Existing Definitions Files 12-3
12.2.3.4 Downloading from a z/OS system to another platform 12-3
12.2.4 Using a Definitions Template 12-3
12.2.5 Configuring Oracle GoldenGate to Capture Data-definitions 12-4
12.2.5.1 Configure DEFGEN 12-4
12.2.5.2 Run DEFGEN 12-6
12.2.5.3 Transfer the Definitions File to the Remote System 12-6
12.2.5.4 Specify the Definitions File 12-6
12.2.6 Adding Tables that Satisfy a Definitions Template 12-7
12.2.7 Examples of Using a Definitions File 12-7
12.2.7.1 Creating a Source-definitions file for Use on a Target System 12-7
12.2.7.2 Creating Target-definitions Files for Use on a Source System 12-8
12.2.7.3 Creating Multiple Source Definition Files for Use on a Target
System 12-9
12.3 Using Automatic Trail File Recovery 12-9
12.4 Configuring Oracle GoldenGate to Use Self-Describing Trail Files 12-10
12.4.1 Support Considerations 12-12
12.4.2 Using Self-Describing Trail Files 12-12
12.4.3 Examples of Parameter Files 12-13
12.5 Configuring Oracle GoldenGate to Assume Identical Metadata 12-14
12.5.1 Rules for Tables to be Considered Identical 12-14
12.6 Configuring Oracle GoldenGate to Assume Dissimilar Metadata 12-15
12.7 Configuring Oracle GoldenGate to Use a Combination of Similar and
Dissimilar Definitions 12-15
x
13.3 Creating a Checkpoint Table 13-3
13.3.1 Options for Creating the Checkpoint Table 13-3
13.3.2 Adjusting for Coordinated Replicat in Oracle RAC 13-5
13.4 Creating an Online Extract Group 13-5
13.5 Creating a Trail 13-8
13.5.1 Assigning Storage for Oracle GoldenGate Trails 13-8
13.5.2 Estimating Space for the Trails 13-9
13.5.3 Adding a Trail 13-9
13.6 Creating a Parameter File for Online Extraction 13-10
13.7 Creating an Online Replicat Group 13-12
13.7.1 About Classic Replicat Mode 13-13
13.7.2 About Coordinated Replicat Mode 13-14
13.7.2.1 About Barrier Transactions 13-15
13.7.2.2 How Barrier Transactions are Processed 13-16
13.7.2.3 About the Global Watermark 13-16
13.7.3 About Integrated Replicat Mode 13-16
13.7.4 Understanding Replicat Processing in Relation to Parameter Changes 13-17
13.7.5 Creating the Replicat Group 13-17
13.8 Creating a Parameter File for Online Replication 13-19
xi
15.1.2.3 Configure the Manager Process 15-2
15.1.2.4 Create a Data-definitions File 15-2
15.1.2.5 Create Change-synchronization Groups 15-2
15.1.2.6 Sharing Parameters between Process Groups 15-3
15.2 Initial Load in Classic Architecture 15-3
15.2.1 Loading Data with a Database Utility 15-3
15.2.2 Loading Data with Oracle Data Pump 15-6
15.2.2.1 Using Automatic Per Table Instantiation 15-6
15.2.2.2 Using Oracle Data Pump Table Instantiation 15-7
15.2.3 Loading Data from File to Replicat 15-7
15.2.4 Loading Data with an Oracle GoldenGate Direct Load 15-13
15.2.5 Loading Data with a Direct Bulk Load to SQL*Loader 15-18
15.2.6 Loading Data with Teradata Load Utilities 15-23
xii
16.3.4 Supporting Character-set Conversion in User Exits 16-18
16.3.5 Using Macros to Check Name Metadata 16-19
16.3.6 Describing the Character Format 16-19
16.3.7 Upgrading User Exits 16-21
16.3.8 Viewing Examples of How to Use the User Exit Functions 16-21
16.4 Using the Oracle GoldenGate Event Marker System to Raise Database
Events 16-22
16.4.1 Case Studies in the Usage of the Event Marker System 16-23
16.4.1.1 Trigger End-of-day Processing 16-23
16.4.1.2 Simplify Transition from Initial Load to Change Synchronization 16-23
16.4.1.3 Stop Processing When Data Anomalies are Encountered 16-23
16.4.1.4 Trace a Specific Order Number 16-24
16.4.1.5 Execute a Batch Process 16-24
16.4.1.6 Propagate Only a SQL Statement without the Resultant
Operations 16-24
16.4.1.7 Committing Other Transactions Before Starting a Long-running
Transaction 16-25
16.4.1.8 Execute a Shell Script to Validate Data 16-25
xiii
17.11 Getting Help with Performance Tuning 17-16
xiv
19.4 Changing Database Attributes 19-3
19.4.1 Changing Database Metadata 19-4
19.4.2 Adding Tables to the Oracle GoldenGate Configuration 19-5
19.4.3 Coordinating Table Attributes between Source and Target 19-6
19.4.4 Performing an ALTER TABLE to Add a Column on DB2 z/OS Tables 19-8
19.4.5 Dropping and Recreating a Source Table 19-9
19.4.6 Changing the Number of Oracle RAC Threads when Using Classic
Capture 19-9
19.4.7 Changing the ORACLE_SID 19-10
19.4.8 Purging Archive Logs 19-10
19.4.9 Reorganizing a DB2 Table (z/OS Platform) 19-11
19.5 Adding Process Groups to an Active Configuration 19-11
19.5.1 Before You Start 19-11
19.5.2 Adding Another Extract Group to an Active Configuration 19-12
19.5.3 Adding Another Data Pump to an Active Configuration 19-14
19.5.4 Adding Another Replicat Group to an Active Configuration 19-16
19.6 Changing the Size of Trail Files 19-18
19.7 Switching Extract from Classic Mode to Integrated Mode 19-18
19.8 Switching Extract from Integrated Mode to Classic Mode 19-20
19.9 Switching Replicat from Nonintegrated Mode to Integrated Mode 19-21
19.10 Switching Replicat from Integrated Mode to Nonintegrated Mode 19-22
19.11 Switching Replicat to Coordinated Mode 19-23
19.11.1 Procedure Overview 19-24
19.11.2 Performing the Switch to Coordinated Replicat 19-24
19.12 Administering a Coordinated Replicat Configuration 19-26
19.12.1 Performing a Planned Re-partitioning of the Workload 19-27
19.12.2 Recovering Replicat After an Unplanned Re-partitioning 19-27
19.12.2.1 Reprocessing From the Low Watermark with
HANDLECOLLISIONS 19-27
19.12.2.2 Using the Auto-Saved Parameter File 19-28
19.12.3 Synchronizing Threads After an Unclean Stop 19-29
19.13 Restarting a Primary Extract after System Failure or Corruption 19-30
19.13.1 Details of This Procedure 19-30
19.13.2 Performing the Recovery 19-30
xv
A Supported Character Sets
A.1 Supported Character Sets - Oracle A-1
A.2 Supported Character Sets - Non-Oracle A-8
B Supported Locales
E About Checkpoints
E.1 Extract Checkpoints E-1
E.1.1 About Extract read checkpoints E-3
E.1.1.1 Startup Checkpoint E-3
E.1.1.2 Recovery Checkpoint E-3
E.1.1.3 Current Checkpoint E-3
E.1.2 About Extract Write Checkpoints E-3
E.2 Replicat Checkpoints E-4
E.2.1 About Replicat Checkpoints E-5
E.2.1.1 Startup Checkpoint E-5
E.2.1.2 Current Checkpoint E-5
E.3 Internal Checkpoint Information E-5
E.4 Oracle GoldenGate Checkpoint Tables E-6
xvi
Preface
This guide contains instructions for:
• Working with the interface components that control Oracle GoldenGate.
• Monitoring and troubleshooting Oracle GoldenGate performance.
• Perform other administrative operations.
Audience
This guide is intended for the person or persons who are responsible for operating
Oracle GoldenGate and maintaining its performance. This audience typically includes,
but is not limited to, systems administrators and database administrators.
Documentation Accessibility
For information about Oracle's commitment to accessibility, visit the Oracle
Accessibility Program website at http://www.oracle.com/pls/topic/lookup?
ctx=acc&id=docacc.
Related Information
The Oracle GoldenGate Product Documentation Libraries are found at
Oracle GoldenGate
Oracle GoldenGate Application Adapters
Oracle GoldenGate for Big Data
Oracle GoldenGate Plug-in for EMCC
Oracle GoldenGate Monitor
Oracle GoldenGate for HP NonStop (Guardian)
Oracle GoldenGate Veridata
Oracle GoldenGate Studio
xvii
Preface
Conventions
The following text conventions are used in this document:
Convention Meaning
boldface Boldface type indicates graphical user interface elements associated
with an action, such as "From the File menu, select Save." Boldface
also is used for terms defined in text or in the glossary.
italic Italic type indicates placeholder variables for which you supply
italic particular values, such as in the parameter statement: TABLE
table_name. Italic type also is used for book titles and emphasis.
monospace Monospace type indicates code components such as user exits and
MONOSPACE scripts; the names of files and database objects; URL paths; and input
and output text that appears on the screen. Uppercase monospace
type is generally used to represent the names of Oracle GoldenGate
parameters, commands, and user-configurable functions, as well as
SQL commands and keywords.
UPPERCASE Uppercase in the regular text font indicates the name of a utility unless
the name is intended to be a specific case.
{} Braces within syntax enclose a set of options that are separated by
pipe symbols, one of which must be selected, for example: {option1 |
option2 | option3}.
[] Brackets within syntax indicate an optional element. For example in this
syntax, the SAVE clause is optional: CLEANUP REPLICAT group_name [,
SAVE count]. Multiple options within an optional element are separated
by a pipe symbol, for example: [option1 | option2].
xviii
1
Oracle GoldenGate Administration
Overview
Administrative decisions for Oracle GoldenGate processes depend on the type of
architecture you are using.
As an administrator for Oracle GoldenGate, you need to ensure that the configuration
and processes are relevant to the type of architecture that’s implemented in your
production environment. This book is divided into two parts to describe the
configurations, processes, and tasks that are specific to the Classic Architecture and
the Microservices Architecture.
See Getting Started with the Oracle GoldenGate Architectures for conceptual details
about the Oracle GoldenGate architectures.
This book is divided into two parts to describe processes that are specific to each
architecture:
• Part 1: Administering Oracle GoldenGate Classic Architecture (page 1)
• Part 2: Administering Oracle GoldenGate Microservices Architecture (page 1)
1-1
2
Oracle GoldenGate Globalization Support
This chapter describes Oracle GoldenGate globalization support, which enables the
processing of data in its native language encoding.
Topics:
See Mapping and Manipulating Data (page 11-1) for more information about
character sets, conversion between them, and data mapping.
2-1
Chapter 2
Using Unicode and Native Characters
• Console
• Command-line input and output
• FORMATASCII, FORMATSQL, FORMATXML parameters, text files such as parameter files,
data-definitions files, error log, process reports, discard files, and other human-
readable files that are used by Oracle GoldenGate users to configure, run, and
monitor the Oracle GoldenGate environment.
In the event that the platform does not support a required character set as the default
in the operating system, you can use the following parameters to specify a character
set:
• CHARSET parameter to specify a character set to be used by processes to read their
parameter files.
• CHARSET option of the DEFSFILE parameter to generate a data-definitions file in a
specific character set.
The GGSCI command console always operates in the character set of the local
operating system for both keyboard and OBEY file input and console output.
2-2
Part I
Administering Oracle GoldenGate Classic
Architecture
Oracle GoldenGate Classic Architecture provides the processes and files required to
effectively move data across a variety of topologies. These processes and files form
the main components of the classic architecture and was the product design until this
release.
3
Configuring Manager and Network
Communications
This chapter describes how to configure the Manager process and specify ports for
local and remote network communications.
Topics:
3-1
Chapter 3
Maintaining Ports for Remote Connections through Firewalls
• Manager reports an error in its process report and in the Oracle GoldenGate
ggserr log.
• Collector retries based on the rules in the Oracle GoldenGate tcperrs file. For
more information about the tcperrs file, see Handling TCP/IP Errors (page 14-5).
See DYNAMICPORTLIST in Reference for Oracle GoldenGate for more information.
3-2
Chapter 3
Using the Recommended Manager Parameters
Note:
If Oracle GoldenGate resides in a cluster, configure the Manager process
within the cluster application as directed by the vendor's documentation, so
that Oracle GoldenGate fails over properly with other applications.For more
information about installing Oracle GoldenGate in a cluster, see the Oracle
GoldenGate documentation for your database.
3-3
Chapter 3
Starting Manager
1. From the Oracle GoldenGate directory, run the GGSCI program to open the
Oracle GoldenGate Software Command Interface (GGSCI).
2. In GGSCI, issue the following command to edit the Manager parameter file.
EDIT PARAMS MGR
3. Add the parameters that you want to use for the Manager process, each on one
line.
4. Save, then close the file.
Example 3-1 Sample manager file on a UNIX system
PORT 7809
DYNAMICPORTLIST 7810-7820, 7830
AUTOSTART ER t*
AUTORESTART ER t*, RETRIES 4, WAITMINUTES 4
STARTUPVALIDATIONDELAY 5
USERIDALIAS mgr1
PURGEOLDEXTRACTS /ogg/dirdat/tt*, USECHECKPOINTS, MINKEEPHOURS 2
The following is a sample Manager parameter file on a UNIX system using required
and recommended parameters.
The reportfile argument is optional and can be used to store the Manager process
report in a location other than the default of the dirrpt directory in the Oracle
GoldenGate installation location.
3-4
Chapter 3
Stopping Manager
Note:
When starting Manager from the command line or GGSCI with User Account
Control enabled, you will receive a UAC prompt requesting you to allow or
deny the program to run.
Where:
! stops Manager without user confirmation.
In a UNIX or Linux cluster, refer to the documentation provided by the cluster vendor
to determine whether Manager should be stopped from the cluster or by using GGSCI.
In a Windows cluster, you must take the Manager resource offline from the Cluster
Administrator. If you attempt to stop Manager from the GGSCI interface, the cluster
monitor interprets it as a resource failure and attempts to bring the resource online
again. Multiple start requests through GGSCI eventually will exceed the start threshold
of the Manager cluster resource, and the cluster monitor will mark the Manager
resource as failed.
3-5
4
Getting Started with the Oracle
GoldenGate Process Interfaces
This chapter describes how Oracle GoldenGate users provide instructions to the
processes through the GGSCI (Oracle GoldenGate Software Command Interface),
batch and shell scripts, and parameter files.
Topics:
For more information, see Support for Escape Sequences (page 11-4) for more
information about using Unicode notation.
Note:
Oracle GoldenGate group names are case-insensitive.
4-1
Chapter 4
Using the GGSCI Command-line Interface
To use OBEY
1. Create and save a text file that contains the commands, one command per line.
This is your OBEY file. The name can be anything supported by the operating
system. You can nest other OBEY files within an OBEY file.
2. Run GGSCI.
3. (Optional) If using an OBEY file that contains nested OBEY files, issue the following
command. This command enables the use of nested OBEY files for the current
session of GGSCI and is required whenever using nested OBEY files. See
Reference for Oracle GoldenGate for more information.
ALLOWNESTED
4. In GGSCI, call the OBEY file by using the OBEY command.
OBEY file_name
Where:
file_name is the relative or fully qualified name of the OBEY file.
The following example illustrates an OBEY command file for use with the OBEY command.
It creates and starts Extract and Replicat groups and retrieves processing information.
See Reference for Oracle GoldenGate for more information about the OBEY command.
4-2
Chapter 4
Controlling Oracle GoldenGate Processes
To Stop Manager
1. From the Oracle GoldenGate directory, run GGSCI.
2. In GGSCI, issue the following command.
{START | STOP [!]} MANAGER
Where:
The ! bypasses the prompt that confirms the intent to shut down Manager.
Note:
When starting Manager from the command line or GGSCI with User Account
Control enabled, you will receive a UAC prompt requesting you to allow or
deny the program to run.
Where:
group_name is the name of the Extract or Replicat group or a wildcard set of groups (for
example, * or fin*).
Where:
4-3
Chapter 4
Controlling Oracle GoldenGate Processes
group_name is the name of the Extract or Replicat group or a wildcard set of groups (for
example, * or fin*).
The current transaction is aborted and the process stops immediately. You cannot
stop Extract forcefully.
Killing a process does not shut it down gracefully, and checkpoint information can be
lost.
Where:
• command is: KILL, START, or STOP
4-4
Chapter 4
Automating Commands
2. Issue one of the following commands from GGSCI to log into the database.
DBLOGIN [SOURCEDB dsn] {USERID user, PASSWORD password [encryption_options] |
USERIDALIAS alias [DOMAIN domain]}
Where:
• SOURCEDB dsn supplies the data source name, if required as part of the
connection information.
• USERID user, PASSWORD password specifies an explicit database login credential.
Deleting a Replicat group preserves the checkpoints in the checkpoint table (if being
used). Deleting a process group also preserves the parameter file. You can create the
same group again, using the same parameter file, or you can delete the parameter file
to remove the group's configuration permanently.
To Input a Script
Use the following syntax from the command line of the operating system.
ggsci < input_file
Where:
• The angle bracket (<) character pipes the file into the GGSCI program.
• input_file is a text file, known as an OBEY file, containing the commands that you
want to issue, in the order they are to be issued.
For detailed documentation of Oracle GoldenGate commands, see Reference for
Oracle GoldenGate.
4-5
Chapter 4
Using Oracle GoldenGate Parameter Files
Note:
To stop the Manager process from a batch file, make certain to add the !
argument to the end of the STOP MANAGER command. Otherwise, GGSCI
issues a prompt that requires a response and causes the process to enter
into a loop. See Stopping Manager (page 3-5) for more information about
stopping Manager.
EXTRACT
GGSCI
KEYGEN
LOGDUMP
MGR
REPLICAT
For more information about these commands, see Reference for Oracle GoldenGate
for Windows and UNIX.
4-6
Chapter 4
Using Oracle GoldenGate Parameter Files
server where Oracle GoldenGate is installed and where the operating system
character set is different. Oracle GoldenGate provides some tools to help with
character set incompatibilities if you must create the parameter file on a different
system:
• You can use the CHARSET parameter to specify a compatible character set for the
parameter file. This parameter must be placed on the first line of the parameter file
and allows you to write the file in the specified character set. After the file is
transferred to the other system, do not edit the file on that system.
• You can use Unicode notation to substitute for characters that are not compatible
with the character set of the operating system where the file will be used. See
Support for Escape Sequences (page 11-4) for more information about Unicode
notation.
See Reference for Oracle GoldenGate for more information about the CHARSET
parameter.
Note:
The ./ portion of this command must be used, because the GLOBALS file
must reside at the root of the Oracle GoldenGate installation file.
4-7
Chapter 4
Using Oracle GoldenGate Parameter Files
4-8
Chapter 4
Using Oracle GoldenGate Parameter Files
4-9
Chapter 4
Using Oracle GoldenGate Parameter Files
directory other than dirprm, but you also must specify the full path name with the
PARAMS option of the ADD EXTRACT or ADD REPLICAT command when you create your
process groups. Once paired with an Extract or Replicat group, a parameter file must
remain in its original location for Oracle GoldenGate to operate properly once
processing has started.
The EDIT PARAMS command launches the following text editors within the GGSCI
interface:
• Notepad on Microsoft Windows systems
• The vi editor on UNIX and Linux systems. DB2 for i only supports vi when
connected with SSH or xterm. For more information, see Creating a Parameter
File with a Text Editor (page 4-11).
Note:
You can change the default editor through the GGSCI interface by using
the SET EDITOR command. See Reference for Oracle GoldenGate.
Where:
group_name is either mgr (for the Manager process) or the name of the Extract or
Replicat group for which the file is being created. The name of an Extract or
Replicat parameter file must match that of the process group.
The following creates or edits the parameter file for an Extract group named
extora.
The following creates or edits the parameter file for the Manager process.
EDIT PARAMS MGR
3. Using the editing functions of the text editor, enter as many comment lines as you
want to describe this file, making certain that each comment line is preceded with
two hyphens (--).
4. On non-commented lines, enter the Oracle GoldenGate parameters, starting a
new line for each parameter statement.
Oracle GoldenGate parameters have the following syntax:
PARAMETER_NAME argument [, option] [&]
Where:
• PARAMETER_NAME is the name of the parameter.
4-10
Chapter 4
Using Oracle GoldenGate Parameter Files
– TABLE
– SEQUENCE
– FILE
– QUERY
Note:
The RMTHOST and RMTHOSTOPTIONS parameters can be specified together;
the RMTHOST parameter is not required for RMTHOSTOPTIONS if the dynamic IP
assignment is properly configured. When RMTHOSTOPTIONS is used, the
MGRPORT option is ignored.
4-11
Chapter 4
Using Oracle GoldenGate Parameter Files
environment. It can provide either a simple PASS/FAIL or with optional details about how
the values of each parameter are stored and interpreted.
The input to checkprm is case insensitive. If a value string contains spaces, it does not
need to be quoted because checkprm can recognize meaningful values. If no mode is
specified to checkprm, then all parameters applicable to any mode of the component
will be accepted.
The output of checkprm is assembled with four possible sections:
• help messages
• pre-validation error
• validation result
• parameter details
A pre-validation error is typically an error that prevents a normal parameter validation
from executing, such as missing options or an inaccessible parameter file. If an option
value is specified incorrectly, a list of possible inputs for that option is provided. If the
result is FAIL, each error is in the final result message. If the result is PASS, a message
that some of the parameters are subject to further runtime validation. The parameter
detailed output contains the validation context, the values read from GLOBALS (if it is
present), and the specified parameters. The parameter and options are printed with
proper indentation to illustrate these relationships.
Table 4-1 (page 4-12) describes all of the arguments that you can use with the
checkprm commands. When you use checkprm and do not use any of these arguments,
then checkprm attempts to automatically detect Extract or Replicat and the platform and
database of the Oracle GoldenGate installation.
4-12
Chapter 4
Using Oracle GoldenGate Parameter Files
4-13
Chapter 4
Using Oracle GoldenGate Parameter Files
The process audits the syntax, writes the results to the report file or the screen,
and then stops.
3. Do either of the following:
• If the syntax is correct, remove the CHECKPARAMS parameter before starting the
process to process data.
• If the syntax is wrong, correct it based on the findings in the report. You can
run another test to verify the changes, if desired. Remove CHECKPARAMS before
starting the process to process data.
For more information about the report file, see Monitoring Oracle GoldenGate
Processing (page 17-1).
For more information about CHECKPARAMS, see Reference for Oracle GoldenGate.
Where:
group_name is either mgr (for Manager) or the name of the Extract or Replicat group that
is associated with the parameter file.
Caution:
Do not use VIEW PARAMS to view an existing parameter file that is in a
character set other than that of the local operating system (such as one
where the CHARSET option was used to specify a different character set). The
contents may become corrupted. View the parameter file from outside
GGSCI.
If the parameter file was created in a location other than the dirprm sub-directory of the
Oracle GoldenGate directory, specify the full path name as shown in the following
example.
4-14
Chapter 4
Using Oracle GoldenGate Parameter Files
Caution:
Do not use the EDIT PARAMS command to view or edit an existing parameter
file that is in a character set other than that of the local operating system
(such as one where the CHARSET option was used to specify a different
character set). The contents may become corrupted. View the parameter file
from outside GGSCI.
To Change Parameters:
1. Stop the process by issuing the following command in GGSCI. To stop Manager in
a Windows cluster, use the Cluster Administrator.
STOP {EXTRACT | REPLICAT | MANAGER} group_name
2. Open the parameter file by using a text editor or the EDIT PARAMS command in
GGSCI.
EDIT PARAMS mgr
3. Make the edits, and then save the file.
4. Start the process by issuing the following command in GGSCI. Use the Cluster
Administrator if starting Manager in a Windows cluster.
START {EXTRACT | REPLICAT | MANAGER} group_name
4-15
Chapter 4
Using Oracle GoldenGate Parameter Files
OBEY file_name
Where:
file_name is the relative or full path name of the file.
Upon encountering an OBEY parameter in the active parameter file, Oracle GoldenGate
processes the parameters from the referenced file and then returns to the active file to
process any remaining parameters. OBEY is not supported for the GLOBALS parameter
file.
If using the CHARSET parameter in a parameter file that includes an OBEY parameter, the
referenced parameter file does not inherit the CHARSET character set. The CHARSET
character set is used to read wildcarded object names in the referenced file, but you
must use an escape sequence (\uX) for all other multibyte specifications in the
referenced file.
See Reference for Oracle GoldenGate for more information about OBEY.
See Reference for Oracle GoldenGate for more information about CHARSET.
4-16
Chapter 4
Specifying Object Names in Oracle GoldenGate Input
UNIX is case-sensitive, so the parameter declaration in the parameter file must be the
same case as the shell variable assignments.
Using the GETPARAMINFO, you can query the runtime parameter values of a running
instance, including Extract, Replicat, and Manager. This command is similar to using
checkprm -v, see Validating a Parameter File (page 4-11). The default behavior is to
display all that has ever been queried by the application, parameters and their current
values. If a particular parameter name is specified, then the output is filtered by that
name. Optionally, the output can be redirect to a file specified by the -FILE option. For
example:
SEND ext1pmp GETPARAMINFO
For more information about these and all Oracle GoldenGate parameters including
exact syntax, see the Reference for Oracle GoldenGate.
4-17
Chapter 4
Specifying Object Names in Oracle GoldenGate Input
For more information, see Support for Escape Sequences (page 11-4).
Character Description
/ Forward slash (See Specifying Names that Contain Slashes (page 4-19))
* Asterisk (Must be escaped by a backward slash when used in parameter file, as
in: \*)
? Question mark (Must be escaped by a backward slash when used in parameter
file, as in: \?)
@ At symbol (Supported, but is often used as a resource locator by databases.
May cause problems in object names)
# Pound symbol
$ Dollar symbol
% Percent symbol (Must be %% when used in parameter file)
4-18
Chapter 4
Specifying Object Names in Oracle GoldenGate Input
Character Description
^ Caret symbol
() Open and close parentheses
_ Underscore
- Dash
Space
Character Description
\ Backward slash (Must be \\ when used in parameter file)
{} Begin and end curly brackets (braces)
[] Begin and end brackets
= Equal symbol
+ Plus sign
! Exclamation point
~ Tilde
| Pipe
& Ampersand
: Colon
; Semi-colon
, Comma
'' Single quotes
"" Double quotes
' Accent mark (Diacritical mark)
. Period
< Less-than symbol (or beginning angle bracket)
> Greater-than symbol (or ending angle bracket)
If the name contains a forward slash that is not enclosed within double quotes, Oracle
GoldenGate treats it as a name that originated on the IBM i platform (from a DB2 for i
database). The forward slash in the name is interpreted as a separator character.
4-19
Chapter 4
Specifying Object Names in Oracle GoldenGate Input
Where:
owner is a schema or database, depending on how the database defines a logical
namespace that contains database objects. object is a table or other supported
database object.
The databases for which Oracle GoldenGate supports two-part names are as follows,
shown with their appropriate two-part naming convention:
• DB2 for i: schema.object and library/file(member)
• DB2 LUW: schema.object
• DB2 on z/OS: schema.object
• MySQL: database.object
• Oracle Database (non-CDB databases): schema.object
• SQL Server: schema.object
• Teradata: database.object
4-20
Chapter 4
Specifying Object Names in Oracle GoldenGate Input
For more information about Oracle container databases, see Oracle Database
Administrator’s Guide.
4-21
Chapter 4
Specifying Object Names in Oracle GoldenGate Input
Note:
For all supported databases, passwords are always treated as case-sensitive
regardless of whether the associated object name is quoted or unquoted.
4-22
Chapter 4
Specifying Object Names in Oracle GoldenGate Input
Where appropriate, Oracle GoldenGate parameters permit the use of two wildcard
types to specify multiple objects in one statement:
• A question mark (?) replaces one character. For example in a schema that
contains tables named TABn, where n is from 0 to 9, a wildcard specification of
HQ.TAB? returns HQ.TAB0, HQ.TAB1, HQ.TAB2, and so on, up to HQ.TAB9, but no others.
This wildcard is not supported for the DB2 LUW database nor for DEFGEN. This
wildcard can only be used to specify source objects in a TABLE or MAP parameter. It
cannot be used to specify target objects in the TARGET clause of TABLE or MAP.
• An asterisk (*) represents any number of characters (including zero sequence).
For example, the specification of HQ.T* could return such objects as HQ.TOTAL,
HQ.T123, and HQ.T. This wildcard is valid for all database types throughout all
Oracle GoldenGate commands and parameters where a wildcard is allowed.
• In TABLE and MAP statements, you can combine the asterisk and question-mark
wildcard characters in source object names only.
• TABLE PDB*.HQ.*;
• MAP HQ.T_*;
The TABLE, MAP and SEQUENCE parameters take the case-sensitivity and locale of the
database into account for wildcard resolution. For databases that are created as case-
sensitive or case-insensitive, the wildcard matches the exact name and case. For
example, if the database is case-sensitive, SCHEMA.TABLE is matched to SCHEMA.TABLE,
Schema.Table is matched to Schema.Table, and so forth. If the database is case-
insensitive, the matching is not case-sensitive.
For databases that can have both case-sensitive and case-insensitive object names in
the same database instance, with the use of quote marks to enforce case-sensitivity,
the wildcarding works differently. When used alone for a source name in a TABLE
statement, an asterisk wildcard matches any character, whether or not the asterisk is
within quotes. The following statements produce the same results:
TABLE hr.*;
TABLE hr."*";
Similarly, a question mark wildcard used alone matches any single character, whether
or not it is within quotes. The following produce the same results:
TABLE hr.?;
TABLE hr."?";
4-23
Chapter 4
Specifying Object Names in Oracle GoldenGate Input
applied. It captures table names that include "abcA" and "abca" because the
wildcard matches both case-sensitive and case-insensitive characters.
TABLE hr."abc*";
TABLE hr."abc?";
• The following TABLE statements capture any table name that begins with upper-
case ABC, because the partial name is case-insensitive (no quotes) and is stored in
upper case by this database. However, because the wildcard matches both case-
sensitive and case-insensitive characters, this example captures table names that
include ABCA and "ABCa".
TABLE hr.abc*;
TABLE hr.abc?;
The preceding mappings produce incorrect results, because the wildcard in the target
specification is replaced with T_TEST (the name of a source object), making the whole
target name T_T_TESTn. The following illustrates the incorrect results:
4-24
Chapter 4
Specifying Object Names in Oracle GoldenGate Input
The following example maps a case-insensitive table name abc to target table name
ABC. Previous releases of Oracle GoldenGate stored case-insensitive object names to
the trail in upper case; thus the target table name is always upper cased. For case-
insensitive name conversion, the comparison is in uppercase, A to Z characters only,
in US-ASCII without taking locale into consideration.
MAP hq.abc, TARGET hq.*;
4-25
Chapter 4
Specifying Object Names in Oracle GoldenGate Input
"columnA"
Literals must be enclosed within single quotes. In the following example, Product_Code
is a case-sensitive column name in an Oracle database, and the other strings are
literals.
@CASE ("Product_Code", 'CAR', 'A car', 'TRUCK', 'A truck')
4-26
5
Using Oracle GoldenGate for Live
Reporting
This chapter describes the usage of Oracle GoldenGate for live reporting.
Topics:
Oracle GoldenGate supports different reporting topologies that enable you to custom-
configure the processes based on your requirements for scalability, availability, and
performance. This section contains things to take into consideration when choosing a
reporting configuration.
5-1
Chapter 5
Creating a Standard Reporting Configuration
• A user exit from the Extract or Replicat process that applies rules from an external
transformation solution, then returns the manipulated data to Oracle GoldenGate
• Replicat to deliver data directly to an ETL solution or other transformation engine
For more information about Oracle GoldenGate's filtering and conversion support, see:
• Mapping and Manipulating Data (page 11-1)
• Customizing Oracle GoldenGate Processing (page 16-1)
5-2
Chapter 5
Creating a Standard Reporting Configuration
See Reference for Oracle GoldenGate for detailed information about these and
other ADD EXTRACT options that may be required for your installation.
2. On the source, use the ADD RMTTRAIL command to specify a remote trail to be
created on the target system.
ADD RMTTRAIL remote_trail, EXTRACT ext
Use the EXTRACT argument to link this trail to the Extract group.
See Reference for Oracle GoldenGate for additional ADD RMTTRAIL options.
3. On the source, use the EDIT PARAMS command to create a parameter file for the
Extract group. Include the following parameters plus any others that apply to your
database environment. For possible additional required parameters, see the
Oracle GoldenGate installation and setup guide for your database.
-- Identify the Extract group:
EXTRACT ext
-- Specify database login information as needed for the database:
[SOURCEDB dsn_1][, USERIDALIAS alias]
-- Log all of the supplementally logged columns if using integrated Replicat
LOGALLSUPCOLS
-- Valid for Oracle. Specify the name or IP address of the target system and
-- optional encryption across TCP/IP:
RMTHOSTOPTIONS target, MGRPORT port_number, ENCRYPT encryption_options
-- Specify the remote trail and encryption algorithm on the target system:
5-3
Chapter 5
Creating a Standard Reporting Configuration
ENCRYPTTRAIL algorithm
RMTTRAIL remote_trail
-- Specify tables and sequences to be captured:
SEQUENCE [container.|catalog.]owner.sequence;
TABLE [container.|catalog.]owner.table;
Use the EXTTRAIL argument to link the Replicat group to the remote trail.
See Reference for Oracle GoldenGate for detailed information about these and
other options that may be required for your installation.
3. On the target, use the EDIT PARAMS command to create a parameter file for the
Replicat group. Include the following parameters plus any others that apply to your
database environment. For possible additional required parameters, see the
Oracle GoldenGate installation and setup guide for your database.
-- Identify the Replicat group:
REPLICAT rep
-- Specify database login information as needed for the database:
[TARGETDB dsn_2][, USERIDALIAS alias]
-- Specify error handling rules:
REPERROR (error, response)
-- Specify tables for delivery and threads if using coordinated Replicat:
MAP [container.|catalog.]owner.table, TARGET owner.table[, DEF template]
[, THREAD (thread_ID)][, THREADRANGE (thread_range[, column_list])]
;
5-4
Chapter 5
Creating a Reporting Configuration with a Data Pump on the Source System
Note:
For DB2 for i, you may need to use the ADD TRANDATA command on the
target tables if they are not already journaled. Alternatively, you could
use the STRJRNPF command to assign the tables to the appropriate
journal. If the target tables are not required to be replicated by Oracle
GoldenGate, the IMAGES(*AFTER) option can be used with STRJRNPF. Since
Oracle GoldenGate operates using transactions, all tables must be
journaled to support transactions and this is not the default with DB2 for
i.
Figure 5-2 Configuration Elements for Replicating to One Target with a Data
Pump
5-5
Chapter 5
Creating a Reporting Configuration with a Data Pump on the Source System
See Reference for Oracle GoldenGate for detailed information about these and
other ADD EXTRACT options that may be required for your installation.
2. On the source, use the ADD EXTTRAIL command to create a local trail. The primary
Extract writes to this trail, and the data-pump Extract reads it.
ADD EXTTRAIL local_trail, EXTRACT ext
Use the EXTRACT argument to link this trail to the primary Extract group. The primary
Extract group writes to this trail, and the data pump group reads it.
3. On the source, use the EDIT PARAMS command to create a parameter file for the
primary Extract group. Include the following parameters plus any others that apply
to your database environment. For possible additional required parameters, see
the Oracle GoldenGate installation and setup guide for your database.
-- Identify the Extract group:
EXTRACT ext
-- Specify database login information as needed for the database:
[SOURCEDB dsn_1][,USERIDALIAS alias]
-- Log all scheduling columns if using integrated Replicat
LOGALLSUPCOLS
-- Specify the local trail that this Extract writes to and
-- encryption algorithm:
ENCRYPTTRAIL algorithm
EXTTRAIL local_trail
-- Specify tables and sequences to be captured:
SEQUENCE [container.|catalog.]owner.sequence;
TABLE [container.|catalog.]owner.table;
Use EXTTRAILSOURCE as the data source option, and specify the name of the local
trail.
2. On the source, use the ADD RMTTRAIL command to specify a remote trail that will be
created on the target system.
ADD RMTTRAIL remote_trail, EXTRACT pump
5-6
Chapter 5
Creating a Reporting Configuration with a Data Pump on the Source System
Use the EXTRACT argument to link the remote trail to the data pump group. The
linked data pump writes to this trail.
See Reference for Oracle GoldenGate for additional ADD RMTTRAIL options.
3. On the source, use the EDIT PARAMS command to create a parameter file for the
data pump. Include the following parameters plus any others that apply to your
database environment.
-- Identify the data pump group:
EXTRACT pump
-- Specify database login information as needed for the database:
[SOURCEDB dsn_1][, USERIDALIAS alias]
-- Decrypt the data only if the data pump must process it.
-- DECRYPTTRAIL
-- Specify the name or IP address of the target system
-- and optional encryption of data over TCP/IP:
RMTHOSTOPTIONS target, MGRPORT port_number, ENCRYPT encryption_options
-- Specify the remote trail and encryption algorithm on the target system:
ENCRYPTTRAIL alogrithm
RMTTRAIL remote_trail
-- Specify tables and sequences to be captured:
SEQUENCE [container.|catalog.]owner.sequence;
TABLE [container.|catalog.]owner.table;
Use the EXTTRAIL argument to link the Replicat group to the remote trail.
See Reference for Oracle GoldenGate for detailed information about these and
other options that may be required for your installation.
3. On the target, use the EDIT PARAMS command to create a parameter file for the
Replicat group. Include the following parameters plus any others that apply to your
database environment. For possible additional required parameters, see the
Oracle GoldenGate installation and setup guide for your database.
-- Identify the Replicat group:
REPLICAT rep
5-7
Chapter 5
Creating a Reporting Configuration with a Data Pump on an Intermediary System
Note:
For DB2 for i, you may need to use the ADD TRANDATA command on the
target tables if they are not already journaled. Alternatively, you could
use the STRJRNPF command to assign the tables to the appropriate
journal. If the target tables are not required to be replicated by Oracle
GoldenGate, the IMAGES(*AFTER) option can be used with STRJRNPF. Since
Oracle GoldenGate operates using transactions, all tables must be
journaled to support transactions and this is not the default with DB2 for
i.
5-8
Chapter 5
Creating a Reporting Configuration with a Data Pump on an Intermediary System
5-9
Chapter 5
Creating a Reporting Configuration with a Data Pump on an Intermediary System
See Reference for Oracle GoldenGate for detailed information about these and
other ADD EXTRACT options that may be required for your installation.
2. On the source, use the ADD EXTTRAIL command to create a local trail. The primary
Extract writes to this trail, and the data-pump Extract reads it.
ADD EXTTRAIL local_trail, EXTRACT ext
Use the EXTRACT argument to link this trail to the primary Extract group. The primary
Extract group writes to this trail, and the data pump group reads it.
3. On the source, use the EDIT PARAMS command to create a parameter file for the
primary Extract group. Include the following parameters plus any others that apply
to your database environment. For possible additional required parameters, see
the Oracle GoldenGate installation and setup guide for your database.
-- Identify the Extract group:
EXTRACT ext
-- Specify database login information as needed for the database:
[SOURCEDB dsn_1][, USERIDALIAS alias]
-- Log all scheduling columns if using integrated Replicat
LOGALLSUPCOLS
-- Specify the local trail that this Extract writes to and
5-10
Chapter 5
Creating a Reporting Configuration with a Data Pump on an Intermediary System
-- encryption algorithm:
ENCRYPTTRAIL algorithm
EXTTRAIL local_trail
-- Specify tables and sequences to be captured:
SEQUENCE [container.|catalog.]owner.sequence;
TABLE [container.|catalog.]owner.table;
Use EXTTRAILSOURCE as the data source option, and specify the name of the local
trail. For a local Extract, you must use EXTTRAIL not RMTTRAIL.
2. On the source, use the ADD RMTTRAIL command to specify a remote trail that will be
created on the intermediary system.
ADD RMTTRAIL remote_trail_1, EXTRACT pump_1
Use the EXTRACT argument to link the remote trail to the pump_1 data pump group.
The linked data pump writes to this trail.
See Reference for Oracle GoldenGate for additional ADD RMTTRAIL options.
3. On the source, use the EDIT PARAMS command to create a parameter file for the
pump_1 data pump. Include the following parameters plus any others that apply to
your database environment.
-- Identify the data pump group:
EXTRACT pump_1
-- Specify database login information:
[SOURCEDB dsn_1][, USERIDALIAS alias]
-- Decrypt the data only if the data pump must process it.
-- DECRYPTTRAIL
-- Specify the name or IP address of the intermediary system
-- and optional encryption of data over TCP/IP:
RMTHOSTOPTIONS target_1, MGRPORT port_number, ENCRYPT encryption_options
-- Specify remote trail and encryption algorithm on intermediary system:
ENCRYPTTRAIL algorithm
RMTTRAIL remote_trail_1
-- Specify tables and sequences to be captured:
SEQUENCE [container.|catalog.]owner.sequence;
TABLE [container.|catalog.]owner.table;
5-11
Chapter 5
Creating a Reporting Configuration with a Data Pump on an Intermediary System
Use EXTTRAILSOURCE as the data source option, and specify the name of the trail
that you created on this system
2. On the intermediary system, use the ADD RMTTRAIL command to specify a remote
trail on the target system.
ADD RMTTRAIL remote_trail_2, EXTRACT pump_2
Use the EXTRACT argument to link the remote trail to the pump_2 data pump. The
linked data pump writes to this trail.
See Reference for Oracle GoldenGate for additional ADD RMTTRAIL options.
3. On the intermediary system, use the EDIT PARAMS command to create a parameter
file for the pump_2 data pump. Include the following parameters plus any others
that apply to your database environment.
-- Identify the data pump group:
EXTRACT pump_2
-- Note that no database login parameters are required in this case.
-- Specify the target definitions file if SOURCEDEFS was used:
TARGETDEFS full_pathname
-- Decrypt the data only if the data pump must process it.
-- DECRYPTTRAIL
-- Specify the name or IP address of the target system
-- and optional encryption of data over TCP/IP:
RMTHOSTOPTIONS target_2, MGRPORT port_number, ENCRYPT encryption_options
-- Specify the remote trail and encryption algorithm on the target system:
ENCRYPTTRAIL algorithm
RMTTRAIL remote_trail_2
-- Specify tables and sequences to be captured:
SEQUENCE [container.|catalog.]owner.sequence;
TABLE [container.|catalog.]owner.table;
5-12
Chapter 5
Creating a Cascading Reporting Configuration
2. On the target, use the ADD REPLICAT command to create a Replicat group. For
documentation purposes, this group is called rep.
ADD REPLICAT rep
[, INTEGRATED | COORDINATED [MAXTHREADS number]]
, EXTTRAIL remote_trail_2,
, BEGIN time
Use the EXTTRAIL argument to link the Replicat group to the trail on this system.
See Reference for Oracle GoldenGate for detailed information about these and
other options that may be required for your installation.
3. On the target, use the EDIT PARAMS command to create a parameter file for the
Replicat group. Include the following parameters plus any others that apply to your
database environment. For possible additional required parameters, see the
Oracle GoldenGate installation and setup guide for your database.
-- Identify the Replicat group:
REPLICAT rep
-- Specify database login information as needed for the database:
[TARGETDB dsn_2][, USERIDALIAS alias]
-- Specify error handling rules:
REPERROR (error, response)
-- Specify tables for delivery and threads if using coordinated Replicat:
MAP [container.|catalog.]owner.table, TARGET owner.table[, DEF template]
[, THREAD (thread_ID)][, THREADRANGE (thread_range[, column_list])]
;
Note:
For DB2 for i, you may need to use the ADD TRANDATA command on the
target tables if they are not already journaled. Alternatively, you could
use the STRJRNPF command to assign the tables to the appropriate
journal. If the target tables are not required to be replicated by Oracle
GoldenGate, the IMAGES(*AFTER) option can be used with STRJRNPF. Since
Oracle GoldenGate operates using transactions, all tables must be
journaled to support transactions and this is not the default with DB2 for
i.
5-13
Chapter 5
Creating a Cascading Reporting Configuration
Note:
See Creating a Reporting Configuration with a Data Pump on an
Intermediary System (page 5-8) if you do not need to apply the
replicated changes to a database on the secondary system.
5-14
Chapter 5
Creating a Cascading Reporting Configuration
5-15
Chapter 5
Creating a Cascading Reporting Configuration
See Reference for Oracle GoldenGate for detailed information about these and
other ADD EXTRACT options that may be required for your installation.
2. On the source, use the ADD EXTTRAIL command to create a local trail.
ADD EXTTRAIL local_trail_1, EXTRACT ext_1
Use the EXTRACT argument to link this trail to the ext_1 Extract group.
3. On the source, use the EDIT PARAMS command to create a parameter file for the
ext_1 Extract group. Include the following parameters plus any others that apply to
5-16
Chapter 5
Creating a Cascading Reporting Configuration
your database environment. For possible additional required parameters, see the
Oracle GoldenGate installation and setup guide for your database.
-- Identify the Extract group:
EXTRACT ext_1
-- Specify database login information as needed for the database:
[SOURCEDB dsn_1][, USERIDALIAS alias]
-- Log all scheduling columns if using integrated Replicat
LOGALLSUPCOLS
-- Specify the local trail that this Extract writes to
-- and encryption algorithm:
ENCRYPTTRAIL algorithm
EXTTRAIL local_trail
-- Specify tables and sequences to be captured:
SEQUENCE [container.|catalog.]owner.sequence;
TABLE [container.|catalog.]owner.table;
Use EXTTRAILSOURCE as the data source option, and specify the name of the local
trail.
2. On the source, use the ADD RMTTRAIL command to specify a remote trail that will be
created on the second system in the cascade.
ADD RMTTRAIL remote_trail_1, EXTRACT pump_1
Use the EXTRACT argument to link the remote trail to the pump_1 data pump group.
The linked data pump writes to this trail.
See Reference for Oracle GoldenGate for additional ADD RMTTRAIL options.
3. On the source, use the EDIT PARAMS command to create a parameter file for the
pump_1 data pump. Include the following parameters plus any others that apply to
your database environment.
-- Identify the data pump group:
EXTRACT pump_1
-- Specify database login information if using NOPASSTHROUGH:
[SOURCEDB dsn_1][, USERIDALIAS alias]
-- Decrypt the data only if the data pump must process it.
-- DECRYPTTRAIL
-- Specify the name or IP address of second system in cascade
-- and optional encryption of data over TCP/IP:
RMTHOSTOPTIONS target_1, MGRPORT port_number, ENCRYPT encryption_options
-- Specify the remote trail and encryption algorithm on the second system:
ENCRYPTTRAIL algorithm
RMTTRAIL remote_trail_1
-- Specify tables and sequences to be captured:
SEQUENCE [container.|catalog.]owner.sequence;
TABLE [container.|catalog.]owner.table;
5-17
Chapter 5
Creating a Cascading Reporting Configuration
Use the EXTTRAIL option to link the rep_1 group to the remote trail remote_trail_1
that is on the local system.
See Reference for Oracle GoldenGate for detailed information about these and
other options that may be required for your installation.
3. On the second system, use the EDIT PARAMS command to create a parameter file
for the Replicat group. Include the following parameters plus any others that apply
to your database environment. For possible additional required parameters, see
the Oracle GoldenGate installation and setup guide for your database.
-- Identify the Replicat group:
REPLICAT rep_1
-- Specify database login information as needed for the database:
[TARGETDB dsn_2][, USERIDALIAS alias]
-- Specify error handling rules:
REPERROR (error, response)
-- Specify tables for delivery and threads if using coordinated Replicat:
MAP [container.|catalog.]owner.table, TARGET owner.table[, DEF template]
[, THREAD (thread_ID)][, THREADRANGE (thread_range[, column_list])]
;
Note:
For DB2 for i, you may need to use the ADD TRANDATA command on the
target tables if they are not already journaled. Alternatively, you could
use the STRJRNPF command to assign the tables to the appropriate
journal. If the target tables are not required to be replicated by Oracle
GoldenGate, the IMAGES(*AFTER) option can be used with STRJRNPF. Since
Oracle GoldenGate operates using transactions, all tables must be
journaled to support transactions and this is not the default with DB2 for
i.
5-18
Chapter 5
Creating a Cascading Reporting Configuration
See Reference for Oracle GoldenGate for detailed information about these and
other ADD EXTRACT options that may be required for your installation.
2. On the second system, use the ADD EXTTRAIL command to specify a local trail that
will be created on the third system.
ADD EXTTRAIL local_trail_2, EXTRACT ext_2
Use the EXTRACT argument to link this local trail to the ext_2 Extract group.
3. On the second system, use the EDIT PARAMS command to create a parameter file
for the ext_2 Extract group. Include the following parameters plus any others that
apply to your database environment. For possible additional required parameters,
see the Oracle GoldenGate installation and setup guide for your database.
-- Identify the Extract group:
EXTRACT ext_2
-- Specify database login information as needed for the database:
[SOURCEDB dsn_2][, USERIDALIAS alias]
-- Log all scheduling columns if using integrated Replicat
LOGALLSUPCOLS
-- Specify the local trail that this Extract writes to
-- and encryption algorithm:
ENCRYPTTRAIL algorithm
EXTTRAIL local_trail_2
-- Ignore local DML, capture Replicat DML:
IGNOREAPPLOPS, GETREPLICATES
-- Specify tables and sequences to be captured:
SEQUENCE [container.|catalog.]owner.sequence;
TABLE [container.|catalog.]owner.table;
Note:
If replicating DDL operations, IGNOREAPPLOPS, GETREPLICATES functionality is
controlled by the DDLOPTIONS parameter.
Use EXTTRAILSOURCE as the data source option, and specify the name of the local
trail.
2. On the second system, use the ADD RMTTRAIL command to specify a remote trail
that will be created on the third system in the cascade.
ADD RMTTRAIL remote_trail_2, EXTRACT pump_2
5-19
Chapter 5
Creating a Cascading Reporting Configuration
Use the EXTRACT argument to link the remote trail to the pump_2 data pump group.
The linked data pump writes to this trail.
See Reference for Oracle GoldenGate for additional ADD RMTTRAIL options.
3. On the second system, use the EDIT PARAMS command to create a parameter file
for the pump_2 data pump. Include the following parameters plus any others that
apply to your database environment.
-- Identify the data pump group:
EXTRACT pump_2
[SOURCEDB dsn_2][, USERIDALIAS alias]
-- Decrypt the data only if the data pump must process it.
-- DECRYPTTRAIL
-- Specify the name or IP address of third system in cascade
-- and optional encryption of data over TCP/IP:
RMTHOSTOPTIONS target_2, MGRPORT port_number, ENCRYPT encryption_options
-- Specify the remote trail and encryption algorithm on the third system:
ENCRYPTTRAIL algorithm
RMTTRAIL remote_trail_2
-- Specify tables and sequences to be captured:
SEQUENCE [container.|catalog.]owner.sequence;
TABLE [container.|catalog.]owner.table;
Use the EXTTRAIL option to link the rep_2 group to the remote_trail_2 trail.
See Reference for Oracle GoldenGate for detailed information about these and
other options that may be required for your installation.
3. On the third system, use the EDIT PARAMS command to create a parameter file for
the Replicat group. Include the following parameters plus any others that apply to
your database environment. For possible additional required parameters, see the
Oracle GoldenGate installation and setup guide for your database.
5-20
Chapter 5
Creating a Cascading Reporting Configuration
Note:
For DB2 for i, you may need to use the ADD TRANDATA command on the
target tables if they are not already journaled. Alternatively, you could
use the STRJRNPF command to assign the tables to the appropriate
journal. If the target tables are not required to be replicated by Oracle
GoldenGate, the IMAGES(*AFTER) option can be used with STRJRNPF. Since
Oracle GoldenGate operates using transactions, all tables must be
journaled to support transactions and this is not the default with DB2 for
i.
5-21
6
Using Oracle GoldenGate for Real-time
Data Distribution
This chapter describes the usage of Oracle GoldenGate for real-time data distribution.
Topics:
6-1
Chapter 6
Creating a Data Distribution Configuration
6-2
Chapter 6
Creating a Data Distribution Configuration
See Reference for Oracle GoldenGate for detailed information about these and
other ADD EXTRACT options that may be required for your installation.
2. On the source, use the ADD EXTTRAIL command to create a local trail.
ADD EXTTRAIL local_trail, EXTRACT ext
6-3
Chapter 6
Creating a Data Distribution Configuration
Use the EXTRACT argument to link this trail to the primary Extract group. The primary
Extract group writes to this trail, and the data pump groups read it
3. On the source, use the EDIT PARAMS command to create a parameter file for the
primary Extract group. Include the following parameters plus any others that apply
to your database environment. For possible additional required parameters, see
the Oracle GoldenGate installation and setup guide for your database.
-- Identify the Extract group:
EXTRACT ext
-- Specify database login information as needed for the database:
[SOURCEDB dsn_1][, USERIDALIAS alias]
-- Log all scheduling columns if using integrated Replicat
LOGALLSUPCOLS
-- Specify the local trail that this Extract writes to
-- and encryption algorithm:
ENCRYPTTRAIL algorithm
EXTTRAIL local_trail
-- Specify tables and sequences to be captured:
SEQUENCE [container.|catalog.]owner.sequence;
TABLE [container.|catalog.]owner.table;
Use EXTTRAIL to specify the local trail.
Use EXTTRAILSOURCE as the data source option, and supply the name of the local
trail.
2. On the source, use the ADD RMTTRAIL command to specify a remote trail that will be
created on each of the target systems.
ADD RMTTRAIL remote_trail_1, EXTRACT pump_1
ADD RMTTRAIL remote_trail_2, EXTRACT pump_2
Use the EXTRACT argument to link each remote trail to a different data pump group.
The linked data pump writes to this trail.
See Reference for Oracle GoldenGate for additional ADD RMTTRAIL options.
3. On the source, use the EDIT PARAMS command to create a parameter file for each of
the data pumps. Include the following parameters plus any others that apply to
your database environment.
Parameter file for pump_1:
-- Identify the data pump group:
EXTRACT pump_1
-- Specify database login information:
[SOURCEDB dsn_1][, USERIDALIAS alias]
-- Decrypt the data only if the data pump must process it.
-- DECRYPTTRAIL
-- Specify the name or IP address of the first target system
-- and optional encryption of data over TCP/IP:
RMTHOSTOPTIONS target_1, MGRPORT port_number, ENCRYPT encryption_options
-- Specify remote trail and encryption algorithm on first target system:
6-4
Chapter 6
Creating a Data Distribution Configuration
ENCRYPTTRAIL algorithm
RMTTRAIL remote_trail_1
-- Specify tables and sequences to be captured:
SEQUENCE [container.|catalog.]owner.sequence;
TABLE [container.|catalog.]owner.table;
Command on target_2:
ADD REPLICAT rep_2
[, INTEGRATED | COORDINATED [MAXTHREADS number]]
, EXTTRAIL remote_trail_2, BEGIN time
Use the EXTTRAIL argument to link the Replicat group to the correct trail.
See Reference for Oracle GoldenGate for detailed information about these and
other options that may be required for your installation.
6-5
Chapter 6
Creating a Data Distribution Configuration
3. On each target, use the EDIT PARAMS command to create a parameter file for the
Replicat group. Use the following parameters plus any others that apply to your
database environment. For possible additional required parameters, see the
Oracle GoldenGate installation and setup guide for your database.
Parameter file for rep_1:
-- Identify the Replicat group:
REPLICAT rep_1
-- Specify database login information as needed for the database:
[TARGETDB dsn_2][, USERIDALIAS alias]
-- Specify error handling rules:
REPERROR (error, response)
-- Specify tables for delivery and threads if using coordinated Replicat:
MAP [container.|catalog.]owner.table, TARGET owner.table[, DEF template]
[, THREAD (thread_ID)][, THREADRANGE (thread_range[, column_list])]
;
You can use any number of MAP statements for any given Replicat group. All MAP
statements for a given Replicat group must specify the same objects that are
contained in the trail that is linked to the group.
6-6
7
Configuring Oracle GoldenGate for Real-
time Data Warehousing
This chapter describes how to configure Oracle GoldenGate for real-time data
warehousing.
Topics:
7-1
Chapter 7
Creating a Data Warehousing Configuration
7-2
Chapter 7
Creating a Data Warehousing Configuration
7-3
Chapter 7
Creating a Data Warehousing Configuration
2. In each Manager parameter file, use the PURGEOLDEXTRACTS parameter to control the
purging of files from the trail on the local system.
Command on source_2:
ADD EXTRACT ext_2, {TRANLOG | INTEGRATED TRANLOG}, BEGIN time [option[, ...]]
See Reference for Oracle GoldenGate for detailed information about these and
other ADD EXTRACT options that may be required for your installation.
2. On each source, use the ADD EXTTRAIL command to create a local trail.
Command on source_1:
ADD EXTTRAIL local_trail_1, EXTRACT ext_1
Command on source_2:
ADD EXTTRAIL local_trail_2, EXTRACT ext_2
Use the EXTRACT argument to link each Extract group to the local trail on the same
system. The primary Extract writes to this trail, and the data-pump reads it.
3. On each source, use the EDIT PARAMS command to create a parameter file for the
primary Extract. Include the following parameters plus any others that apply to
your database environment. For possible additional required parameters, see the
Oracle GoldenGate installation and setup guide for your database.
Parameter file for ext_1:
-- Identify the Extract group:
EXTRACT ext_1
-- Specify database login information as needed for the database:
[SOURCEDB dsn_1][, USERIDALIAS alias]
-- Log all scheduling columns if using integrated Replicat
LOGALLSUPCOLS
-- Specify the local trail that this Extract writes to
-- and the encryption algorithm:
ENCRYPTTRAIL algorithm
EXTTRAIL local_trail_1
-- Specify tables and sequences to be captured:
SEQUENCE [container.|catalog.]owner.sequence;
TABLE [container.|catalog.]owner.table;
7-4
Chapter 7
Creating a Data Warehousing Configuration
ENCRYPTTRAIL algorithm
EXTTRAIL local_trail_2
-- Specify tables and sequences to be captured:
SEQUENCE [container.|catalog.]owner.sequence;
TABLE [container.|catalog.]owner.table;
Command on source_2:
ADD EXTRACT pump_2, EXTTRAILSOURCE local_trail_2, BEGIN time
Use EXTTRAILSOURCE as the data source option, and specify the name of the trail on
the local system
2. On each source, use the ADD RMTTRAIL command to create a remote trail on the
target.
Command on source_1:
ADD RMTTRAIL remote_trail_1, EXTRACT pump_1
Command on source_2:
ADD RMTTRAIL remote_trail_2, EXTRACT pump_2
Use the EXTRACT argument to link each remote trail to a different data pump. The
data pump writes to this trail over TCP/IP, and a Replicat reads from it.
See Reference for Oracle GoldenGate for additional ADD RMTTRAIL options.
3. On each source, use the EDIT PARAMS command to create a parameter file for the
data pump group. Include the following parameters plus any others that apply to
your database environment.
Parameter file for pump_1:
-- Identify the data pump group:
EXTRACT pump_1
-- Specify database login information as needed for the database:
[SOURCEDB dsn_1][, USERIDALIAS alias]
-- Decrypt the data only if the data pump must process it.
-- DECRYPTTRAIL
-- Specify the name or IP address of the target system
-- and optional encryption of data over TCP/IP:
RMTHOSTOPTIONS target, MGRPORT port_number, ENCRYPT encryption_options
-- Specify the remote trail and encryption algorithm on the target system:
ENCRYPTTRAIL algorithm
RMTTRAIL remote_trail_1
-- Specify tables and sequences to be captured:
SEQUENCE [container.|catalog.]owner.sequence;
TABLE [container.|catalog.]owner.table;
7-5
Chapter 7
Creating a Data Warehousing Configuration
Use the EXTTRAIL argument to link the Replicat group to the trail.
See Reference for Oracle GoldenGate for detailed information about these and
other options that may be required for your installation.
3. On the target, use the EDIT PARAMS command to create a parameter file for each
Replicat group. Include the following parameters plus any others that apply to your
database environment. For possible additional required parameters, see the
Oracle GoldenGate installation and setup guide for your database.
Parameter file for rep_1:
7-6
Chapter 7
Creating a Data Warehousing Configuration
You can use any number of MAP statements for any given Replicat group. All MAP
statements for a given Replicat group must specify the same objects that are
contained in the trail that is linked to the group.
7-7
8
Configuring Oracle GoldenGate to Maintain
a Live Standby Database
This chapter describes how to configure Oracle GoldenGate to maintain a live standby
database.
Topics:
8-1
Chapter 8
Considerations for a Live Standby Configuration
In the case of a failure of the primary system, the Oracle GoldenGate Manager and
Replicat processes work in conjunction with a database instantiation taken from the
standby to restore parity between the two systems after the primary system is
recovered. At the appropriate time, users are moved back to the primary system, and
Oracle GoldenGate is configured in ready mode again, in preparation for future
failovers.
8-2
Chapter 8
Creating a Live Standby Configuration
8-3
Chapter 8
Creating a Live Standby Configuration
8-4
Chapter 8
Creating a Live Standby Configuration
ADD EXTRACT ext_1, {TRANLOG | INTEGRATED TRANLOG}, BEGIN time [option[, ...]]
See Reference for Oracle GoldenGate for detailed information about these and
other ADD EXTRACT options that may be required for your installation.
2. Use the ADD EXTTRAIL command to add a local trail. For documentation purposes,
this trail is called local_trail_1.
ADD EXTTRAIL local_trail_1, EXTRACT ext_1
For EXTRACT, specify the pump_1 data pump to write to this trail.
See Reference for Oracle GoldenGate for additional ADD RMTTRAIL options.
3. Use the EDIT PARAMS command to create a parameter file for the pump_1 group.
Include the following parameters plus any others that apply to your database
environment.
-- Identify the data pump group:
EXTRACT pump_1
-- Specify database login information as needed for the database:
[SOURCEDB dsn_1][, USERIDALIAS alias]
-- Decrypt the data only if the data pump must process it.
-- DECRYPTTRAIL
-- Specify the name or IP address of the standby system
8-5
Chapter 8
Configuration from Standby to Active Source
For EXTTRAIL, specify remote_trail_1 as the trail that this Replicat reads.
See Reference for Oracle GoldenGate for detailed information about these and
other options that may be required for your installation.
3. Use the EDIT PARAMS command to create a parameter file for the rep_1 group.
Include the following parameters plus any others that apply to your database
environment. For possible additional required parameters, see the Oracle
GoldenGate installation and setup guide for your database.
-- Identify the Replicat group:
REPLICAT rep_1
-- State that source and target definitions are identical:
ASSUMETARGETDEFS
-- Specify database login information as needed for the database:
[TARGETDB dsn_2][, USERIDALIAS alias]
-- Specify error handling rules:
REPERROR (error, response)
-- Specify tables for delivery and threads if using coordinated Replicat:
MAP [container.|catalog.]owner.table, TARGET owner.table[, DEF template]
[, THREAD (thread_ID)][, THREADRANGE (thread_range[, column_list])]
;
Note:
This is a reverse image of the configuration that you just created.
8-6
Chapter 8
Configuration from Standby to Active Source
See Reference for Oracle GoldenGate for detailed information about these and
other ADD EXTRACT options that may be required for your installation.
2. Use the ADD EXTTRAIL command to add a local trail. For documentation purposes,
this trail is called local_trail_2.
ADD EXTTRAIL local_trail_2, EXTRACT ext_2
For EXTRACT, specify the pump_2 data pump to write to this trail.
See Reference for Oracle GoldenGate for additional ADD RMTTRAIL options.
3. Use the EDIT PARAMS command to create a parameter file for the pump_2 group.
Include the following parameters plus any others that apply to your database
environment.
8-7
Chapter 8
Moving User Activity in a Planned Switchover
For EXTTRAIL, specify remote_trail_2 as the trail that this Replicat reads.
See Reference for Oracle GoldenGate for detailed information about these and
other options that may be required for your installation.
2. Use the EDIT PARAMS command to create a parameter file for the rep_2 group.
Include the following parameters plus any others that apply to your database
environment. For possible additional required parameters, see the Oracle
GoldenGate installation and setup guide for your database.
-- Identify the Replicat group:
REPLICAT rep_2
-- State that source and target definitions are identical:
ASSUMETARGETDEFS
-- Specify database login information as needed for the database:
[TARGETDB dsn_1][, USERIDALIAS alias]
-- Handle collisions between failback data copy and replication:
HANDLECOLLISIONS
-- Specify error handling rules:
REPERROR (error, response)
-- Specify tables for delivery and threads if using coordinated Replicat:
MAP [container.|catalog.]owner.table, TARGET owner.table[, DEF template]
[, THREAD (thread_ID)][, THREADRANGE (thread_range[, column_list])]
;
8-8
Chapter 8
Moving User Activity in a Planned Switchover
Note:
Since capture continues to read REDO, the non-production workload
continues to work. In this case, there is possibility that At EOF is never
returned even though the production workload has already
stopped8.5.1..
8-9
Chapter 8
Moving User Activity in a Planned Switchover
Note:
Do not start the data pump on the live standby system, and do not start
the Replicat on the primary system. Data must be stored in the local trail
on the live standby until the primary database is ready for user activity
again.
8-10
Chapter 8
Moving User Activity in an Unplanned Failover
6. On the live standby system, issue the following command for the data pump until
it returns "At EOF, no more records to process." This indicates that the pump sent
all of the captured data to the primary system.
LAG EXTRACT pump_2
7. On the live standby system, stop the data pump.
STOP EXTRACT pump_2
8. On the primary system, issue the STATUS REPLICAT command until it returns "At
EOF (end of file)." This confirms that Replicat applied all of the data from the trail
to the database.
STATUS REPLICAT rep_2
9. On the primary system, stop Replicat.
STOP REPLICAT rep_2
10. On the primary system, do the following:
• Run the script that grants insert, update, and delete permissions to the users
of the business applications.
• Run the script that enables triggers and cascade delete constraints.
• Run the scripts that switch over the application server, start applications, and
copy essential files that are not part of the replication environment.
11. On the primary system, alter the primary Extract to begin capturing data based on
the current timestamp. Otherwise, Extract will spend unnecessary time looking for
operations that were already captured and replicated while users were working on
the standby system.
ALTER EXTRACT ext_1, BEGIN NOW
12. On the primary system, start the primary Extract so that it is ready to capture
transactional changes.
START EXTRACT ext_1
13. Switch user activity to the primary system.
14. (Optional) If system maintenance must be done on the live standby system, you
can do it now, before starting the data pump on the primary system. Note that
captured data will be accumulating on the primary system while the standby is
offline.
15. On the primary system, start the data pump.
8-11
Chapter 8
Moving User Activity in an Unplanned Failover
Note:
Do not start the data pump group on the standby. The user transactions
must accumulate there until just before user activity is moved back to the
primary system.
8-12
Chapter 8
Moving User Activity in an Unplanned Failover
• For TRANLOG and INTEGRATED TRANLOG, see Reference for Oracle GoldenGate.
INTEGRATED TRANLOG enables integrated capture for an Oracle database.
6. On the primary system, add the local trail again, using the same name as before.
For documentation purposes, this trail is called local_trail_1.
ADD EXTTRAIL local_trail_1, EXTRACT ext_1
8-13
Chapter 8
Moving User Activity in an Unplanned Failover
copy stopped at 12:05, make sure that change replication has posted data up to
that point.
INFO REPLICAT rep_2
4. On the primary system, issue the following command to turn off the
HANDLECOLLISIONS parameter and disable the initial-load error handling.
8-14
9
Configuring Oracle GoldenGate for Active-
Active High Availability
This chapter describes how to configure Oracle GoldenGate for active-active high
availability.
Topics:
9-1
Chapter 9
Considerations for an Active-Active Configuration
database to see if there are any other limitations or requirements to support a bi-
directional configuration.
9.2.1 TRUNCATES
Bi-directional replication of TRUNCATES is not supported, but you can configure these
operations to be replicated in one direction, while data is replicated in both directions.
To replicate TRUNCATES (if supported by Oracle GoldenGate for the database) in an
active-active configuration, the TRUNCATES must originate only from one database, and
only from the same database each time.
Configure the environment as follows:
• Configure all database roles so that they cannot execute TRUNCATE from any
database other than the one that is designated for this purpose.
• On the system where TRUNCATE will be permitted, configure the Extract and Replicat
parameter files to contain the GETTRUNCATES parameter.
• On the other system, configure the Extract and Replicat parameter files to contain
the IGNORETRUNCATES parameter. No TRUNCATES should be performed on this system
by applications that are part of the Oracle GoldenGate configuration.
9.2.3 Keys
For accurate detection of conflicts, all records must have a unique, not-null identifier. If
possible, create a primary key. If that is not possible, use a unique key or create a
substitute key with a KEYCOLS option of the MAP and TABLE parameters. In the absence of
a unique identifier, Oracle GoldenGate uses all of the columns that are valid in a WHERE
clause, but this will degrade performance if the table contains numerous columns.
To maintain data integrity and prevent errors, the following must be true of the key that
you use for any given table:
• contain the same columns in all of the databases where that table resides.
• contain the same values in each set of corresponding rows across the databases.
9-2
Chapter 9
Considerations for an Active-Active Configuration
Note:
For MySQL targets, cascade delete queries result in the deletion of the
child of the parent operation.
Note:
For Oracle Database targets, if Replicat is in integrated mode,
constraints are handled automatically without special configuration.
Note:
See other IDENTITY limitations for SQL Server in Configuring a Replicat
Database ConnectionUsing Oracle GoldenGate for Heterogeneous
Databases.
9-3
Chapter 9
Preventing Data Looping
9-4
Chapter 9
Preventing Data Looping
This parameter statement marks all DDL and DML transactions that are generated by
this user as Replicat transactions. The user name is included in the transaction record
that is read by Extract.
9.3.2.2 MySQL
Identify the name of the Replicat checkpoint table by using the following parameter
statement in the Extract parameter file.
TRANLOGOPTIONS FILTERTABLE table_name
Replicat writes a checkpoint to the checkpoint table at the end of each of its
transactions as part of its checkpoint procedure. (This is the table that is created with
the ADD CHECKPOINTTABLE command.) Because every Replicat transaction includes a
write to this table, it can be used to identify Replicat transactions in a bidirectional
configuration. FILTERTABLE identifies the name of the checkpoint table, so that Extract
ignores transactions that contain any operations on it.
9.3.2.3 Oracle
There are multiple ways to identify Replicat transaction in an Oracle environment.
When Replicat is in classic or integrated mode, you use the following parameters:
• Use DBOPTIONS with the SETTAG option in the Replicat parameter file. Replicat tags
the transactions being applied with the specified value, which identifies those
transactions in the redo stream. The default SETTAG value is 00. Valid values are a
single TAG value consisting of hexadecimal digits. For more information about
tags, see Reference for Oracle GoldenGate.
9-5
Chapter 9
Managing Conflicts
• Use the TRANLOGOPTIONS parameter with the EXCLUDETAG option in the Extract
parameter file. The logmining server associated with that Extract excludes redo
that is tagged with the SETTAG value.
The following shows how SETTAG can be set in the Replicat parameter file:
DBOPTIONS SETTAG 0935
The following shows how EXCLUDETAG can be set in the Extract parameter file:
TRANLOGOPTIONS EXCLUDETAG 0935
If you are excluding multiple tags, each must have a separate TRANLOGOPTIONS
EXCLUDETAG statement specified.
You can also use the transaction name or userid of the Replicat user to identify
Replicat transactions. You can choose which of these to ignore when you configure
Extract. See Preventing the Capture of Replicat Transactions (Oracle) (page 9-4).
For more information, see Reference for Oracle GoldenGate.
Replicat writes a checkpoint to the checkpoint table at the end of each of its
transactions as part of its checkpoint procedure. (This is the table that is created with
the ADD CHECKPOINTTABLE command). Because every Replicat transaction includes a
write to this table, it can be used to identify Replicat transactions in a bi-directional
configuration. FILTERTABLE identifies the name of the checkpoint table, so that Extract
ignores transactions that contain any operations on it.
(Classic Extract) By default, Extract ignores the Replicat's transactions, however, if
you modify the Replicat's transaction name with the DBOPTIONS TRANSNAME parameter,
then you must exclude those transactions by using the following parameter statement
in the Extract parameter file.
TRANLOGOPTIONS EXCLUDETRANS transaction_name
This parameter statement is only required if the Replicat transaction name is set to
something other than the default of ggs_repl.
9-6
Chapter 9
Managing Conflicts
9-7
Chapter 9
Additional Information
9-8
Chapter 9
Creating an Active-Active Configuration
9-9
Chapter 9
Creating an Active-Active Configuration
Note:
The VAM parameter in the examples is used only for heterogeneous
databases and does not apply to Oracle Database.
9-10
Chapter 9
Creating an Active-Active Configuration
2. Use the ADD RMTTRAIL command to add a remote trail that will be created on the
secondary system. For documentation purposes, this trail is called remote_trail_1.
ADD RMTTRAIL remote_trail_1, EXTRACT pump_1
For EXTRACT, specify the pump_1 data pump to write to this trail.
See Reference for Oracle GoldenGate for additional ADD RMTTRAIL options.
Use the EDIT PARAMS command to create a parameter file for the pump_1 group.
Include the following parameters plus any others that apply to your database
environment.
-- Identify the data pump group:
EXTRACT pump_1
-- Specify database login information as needed for the database:
[SOURCEDB dsn_1][, USERIDALIAS alias]
-- Decrypt the data only if the data pump must process it.
-- DECRYPTTRAIL
-- Specify the name or IP address of the secondary system
-- and optional encryption of data over TCP/IP:
RMTHOSTOPTIONS system_2, MGRPORT port_number, ENCRYPT encryption_options
-- Specify remote trail and encryption algorithm on secondary system:
ENCRYPTTRAIL algorithm
RMTTRAIL remote_trail_1
-- Specify tables to be captured:
TABLE [container.|catalog.]owner.*;
For EXTTRAIL, specify remote_trail_1 as the trail that this Replicat reads.
2. Use the EDIT PARAMS command to create a parameter file for the rep_1 group.
Include the following parameters plus any others that apply to your database
environment. For possible additional required parameters, see the Oracle
GoldenGate installation and setup guide for your database.
-- Identify the Replicat group:
REPLICAT rep_1
-- State that source and target definitions are identical:
ASSUMETARGETDEFS
-- Specify database login information as needed for the database:
[TARGETDB dsn_2][, USERIDALIAS alias]
-- Specify error handling rules:
REPERROR (error, response)
-- Set redo tag for Oracle only replicat via settag
-- Default is 00.
SETTAG tag_value
-- Specify tables for delivery, threads if coordinated Replicat
-- and conflict-resolution:
MAP [container.|catalog.]owner.*, TARGET owner.*, COMPARECOLS (ON operation
{ALL | KEY | KEYINCLUDING (col_list) | ALLEXCLUDING (col_list)}),
RESOLVECONFLICT (conflict type (resolution_name, resolution_type COLS
9-11
Chapter 9
Creating an Active-Active Configuration
(col[,...]))
[, THREAD (thread_ID)]
[, THREADRANGE (thread_range[, column_list])]
;
-- Specify mapping of exceptions to exceptions table:
MAP [container.|catalog.]owner.*, TARGET owner.exceptions, EXCEPTIONSONLY;
Note:
This is a reverse image of the configuration that you just created.
1. Use the ADD EXTRACT command to create a primary Extract group. For
documentation purposes, this group is called ext_2.
ADD EXTRACT ext_2, {TRANLOG | INTEGRATED TRANLOG}, BEGIN time
2. Use the ADD EXTTRAIL command to add a local trail. For documentation purposes,
this trail is called local_trail_2.
ADD EXTTRAIL local_trail_2, EXTRACT ext_2
9-12
Chapter 9
Creating an Active-Active Configuration
-- Oracle:
-- TRACETABLE trace_table_name
-- Log all scheduling columns for CDR and if using integrated Replicat
LOGALLSUPCOLS
-- Capture before images for conflict resolution:
GETBEFORECOLS (ON operation {ALL | KEY | KEYINCLUDING (col_list) | ALLEXCLUDING
(col_list)})
-- Specify tables to be captured and (optional) columns to fetch:
TABLE [container.|catalog.]owner.* [, FETCHCOLS cols | FETCHCOLSEXCEPT cols];
Note:
To replicate Oracle DBFS data, specify the internally generated local
read-write DBFS tables in the TABLE statement on each node. For more
information on identifying these tables and configuring DBFS for
propagation by Oracle GoldenGate, see Applying the Required Patch in
Using Oracle GoldenGate for Oracle Database.
For EXTRACT, specify the pump_2 data pump to write to this trail.
3. Use the EDIT PARAMS command to create a parameter file for the pump_2 group.
Include the following parameters plus any others that apply to your database
environment.
-- Identify the data pump group:
EXTRACT pump_2
-- Specify database login information as needed for the database:
[SOURCEDB dsn_2][, USERIDALIAS alias]
-- Decrypt the data only if the data pump must process it.
-- DECRYPTTRAIL
-- Specify the name or IP address of the primary system
-- and optional encryption of data over TCP/IP:
RMTHOSTOPTIONS system_1, MGRPORT port_number, ENCRYPT encryption_options
-- Specify the remote trail and encryption algorithm on the primary system:
ENCRYPTTRAIL algorithm
RMTTRAIL remote_trail_2
-- Specify tables to be captured:
TABLE [container.|catalog.]owner.*;
9-13
Chapter 9
Creating an Active-Active Configuration
Note:
To replicate Oracle DBFS data, specify the internally generated local
read-write DBFS tables in the TABLE statement on each node. For more
information on identifying these tables and configuring DBFS for
propagation by Oracle GoldenGate, see Configuring the DBFS File
System Using Oracle GoldenGate for Oracle Database.
For EXTTRAIL, specify remote_trail_2 as the trail that this Replicat reads.
See ADD RMTTRAIL in Reference for Oracle GoldenGatefor detailed information
about these and other options that may be required for your installation.
2. Use the EDIT PARAMS command to create a parameter file for the rep_2 group.
Include the following parameters plus any others that apply to your database
environment.
-- Identify the Replicat group:
REPLICAT rep_2
-- State that source and target definitions are identical:
ASSUMETARGETDEFS
-- Specify database login information as needed for the database:
[TARGETDB dsn_1][, USERIDALIAS alias]
-- Specify error handling rules:
REPERROR (error, response)
-- Specify tables for delivery, threads if coordinated Replicat
-- and conflict-resolution:
MAP [container.|catalog.]owner.*, TARGET owner.*, COMPARECOLS (ON operation
{ALL | KEY | KEYINCLUDING (col_list) | ALLEXCLUDING (col_list)}),
RESOLVECONFLICT (conflict type (resolution_name, resolution_type COLS
(col[,...]))
[, THREAD (thread_ID)]
[, THREADRANGE (thread_range[, column_list])]
;
-- Specify mapping of exceptions to exceptions table:
MAP [container.|catalog.]owner.*, TARGET owner.exceptions, EXCEPTIONSONLY;
Note:
To replicate Oracle DBFS data, specify the internally generated local
read-write DBFS tables in the TABLE statement on each node.
9-14
10
Configuring Conflict Detection and
Resolution
This chapter contains instructions for using the Oracle GoldenGate Conflict Detection
and Resolution (CDR) feature. Conflict detection and resolution is required in active-
active configurations, where Oracle GoldenGate must maintain data synchronization
among multiple databases that contain the same data sets.
Topics:
• DATE
• TIMESTAMP
• CHAR/NCHAR
• VARCHAR/ NVARCHAR
This means that these column types can be used with the COMPARECOLS parameter, the
GETBEFORECOLS parameter, and as the resolution column in the USEMIN and USEMAX
options of the RESOLVECONFLICT parameter. Only NUMERIC columns can be used for the
USEDELTA option of RESOLVECONFLICT. Do not use CDR for columns that contain LOBs,
abstract data types (ADT), or user-defined types (UDT).
Conflict resolution is not performed when Replicat operates in BATCHSQL mode. If a
conflict occurs in BATCHSQL mode, Replicat reverts to GROUPTRANSOPS mode, and then to
single-transaction mode. Conflict detection occurs in all three modes. For more
information, see Reference for Oracle GoldenGate.
10-1
Chapter 10
Configuring Oracle GoldenGate CDR
2. Use the COMPARECOLS option of the MAP parameter in the Replicat parameter file to
specify columns that are to be used with before values in the Replicat WHERE
clause. The before values are compared with the current values in the target
database to detect update and delete conflicts. (By default, Replicat only uses the
primary key in the WHERE clause; this may not be enough for conflict detection).
3. Use the RESOLVECONFLICT option of the MAP parameter to specify conflict resolution
routines for different operations and conflict types. You can use RESOLVECONFLICT
multiple times in a MAP statement to specify different resolutions for different conflict
types. However, you cannot use RESOLVECONFLICT multiple times for the same type
of conflict. Use identical conflict-resolution procedures on all databases, so that
10-2
Chapter 10
Configuring Oracle GoldenGate CDR
the same conflict produces the same end result. One conflict-resolution method
might not work for every conflict that could occur. You might need to create
several routines that can be called in a logical order of priority so that the risk of
failure is minimized.
Note:
Additional consideration should be given when a table has a primary key and
additional unique indexes or unique keys. The automated routines provided
with the COMPARECOLS and RESOLVECONFLICT parameters require a consistent
way to uniquely identify each row. Failure to consistently identify a row will
result in an error during conflict resolution. In these situations the additional
unique keys should be disabled or you can use the SQLEXEC feature to handle
the error thrown and resolve the conflict.
For detailed information about these parameters, see Reference for Oracle
GoldenGate. See the examples starting on CDR Example 1: All Conflict Types with
USEMAX, OVERWRITE, DISCARD (page 10-8), for more information on these
parameters.
10-3
Chapter 10
Configuring Oracle GoldenGate CDR
• The before image of the trail record. This is a duplicate set of the target
columns with names such as col1_before, col2_before, and so forth.
• The current values of the target columns. This also is a duplicate set of the
target columns with names such as col1_current, col2_current, and so forth.
• The name of the target table
• The timestamp of the conflict
• The operation type
• The database error number
• (Optional) The database error message
• Whether the conflict was resolved or not
3. Create an exceptions MAP statement to map the exceptions data to the exceptions
table. An exceptions MAP statement contains:
• (Required) The INSERTALLRECORDS option. This parameter converts all mapped
operations to INSERTs so that all column values are mapped to the exceptions
table.
• (Required) The EXCEPTIONSONLY option. This parameter causes Replicat to map
operations that generate an error, but not those that were successful.
• (Optional) A COLMAP clause. If the names and definitions of the columns in the
exceptions table are identical to those of the source table, and the exceptions
table only contains those columns, no COLMAP is needed. However, if any
names or definitions differ, or if there are extra columns in the exceptions table
that you want to populate with additional data, use a COLMAP clause to map all
columns.
10-4
Chapter 10
Configuring Oracle GoldenGate CDR
See Reference for Oracle GoldenGate for Windows and UNIX for the usage and
syntax of the parameters and column-conversion functions shown in these examples.
10.2.3.2 Sample Exceptions Mapping with Source and Target Columns Only
The following is a sample parameter file that shows error handling and simple
exceptions mapping for the source and target tables that are used in the CDR
examples that begin. This example maps source and target columns, but no extra
columns. For the following reasons, a COLMAP clause is not needed in the exceptions
MAP statement in this example:
• The source and target exceptions columns are identical in name and definition.
• There are no other columns in the exceptions table.
Note:
This example intentionally leaves out other parameters that are required
in a Replicat parameter file, such as process name and login credentials,
as well as any optional parameters that may be required for a given
database type. When using line breaks to split a parameter statement
into multiple lines, use an ampersand (&) at the end of each line.
10-5
Chapter 10
Configuring Oracle GoldenGate CDR
INSERTALLRECORDS
;
Note:
This example intentionally leaves out other parameters that are required in a
Replicat parameter file, such as process name and login credentials, as well
as any optional parameters that may be required for a given database type.
When using line breaks to split a parameter statement into multiple lines, use
an ampersand (&) at the end of each line.
10-6
Chapter 10
Configuring Oracle GoldenGate CDR
USEDEFAULTS, &
-- captures the timestamp when the resolution was performed.
res_date = @DATENOW (), &
-- captures and maps the DML operation type.
optype = @GETENV ('LASTERR', 'OPTYPE'), &
-- captures and maps the database error number that was returned.
dberrnum = @GETENV ('LASTERR', 'DBERRNUM'), &
-- captures and maps the database error that was returned.
dberrmsge = @GETENV ('LASTERR', 'DBERRMSG'), &
-- captures and maps the name of the target table
tabname = @GETENV ('GGHEADER', 'TABLENAME'), &
-- If the names and definitions of the source columns and the target
-- exceptions columns were not identical, the columns would need to
-- be mapped in the COLMAP clause instead of using USEDEFAULTS, as
-- follows:
-- name_after = name, &
-- phone_after = phone, &
-- address_after = address, &
-- salary_after = salary, &
-- balance_after = balance, &
-- comment_after = comment, &
-- last_mod_time_after = last_mod_time &
-- maps the before image of each column from the trail to a column in the
-- exceptions table.
name_before = @BEFORE (name), &
phone_before = @BEFORE (phone), &
address_before = @BEFORE (address), &
salary_before = @BEFORE (salary), &
balance_before = @BEFORE (balance), &
comment_before = @BEFORE (comment), &
last_mod_time_before = @BEFORE (last_mod_time), &
-- maps the results of the SQLEXEC query to rows in the exceptions table
-- to show the current image of the row in the target.
name_current = qry.name, &
phone_current = qry.phone, &
address_current = qry.address, &
salary_current = qry.salary, &
balance_current = qry.balance, &
comment_current = qry.comment, &
last_mod_time_current = qry.last_mod_time)
;
For more information about creating an exceptions table and using exceptions
mapping, see Handling Replicat Errors during DML Operations (page 14-1).
Once you are confident that your routines work as expected in all situations, you can
reduce the amount of data that is logged to the exceptions table to reduce the
overhead of the resolution routines.
10-7
Chapter 10
CDR Example 1: All Conflict Types with USEMAX, OVERWRITE, DISCARD
10.2.4.2 GGSCI
You can view CDR statistics from GGSCI by using the STATS REPLICAT command with
the REPORTCDR option:
STATS REPLICAT group, REPORTCDR
You can return these statistics for all of the tables in all of the MAP statements in the
Replicat parameter file:
@GETENV ('STATS','CDR_CONFLICTS')
@GETENV ('STATS','CDR_RESOLUTIONS_SUCCEEDED')
@GETENV ('STATS','CDR_RESOLUTIONS_FAILED')
For more information about @GETENV, see Reference for Oracle GoldenGate.
10-8
Chapter 10
CDR Example 1: All Conflict Types with USEMAX, OVERWRITE, DISCARD
• Per COMPARECOLS, use the before image of all columns in the trail record in the
Replicat WHERE clause for updates and deletes.
• Per DEFAULT, use all columns as the column group for all conflict types; thus the
resolution applies to all columns.
• For an INSERTROWEXISTS conflict, use the USEMAX resolution: If the row exists during
an insert, use the last_mod_time column as the resolution column for deciding
which is the greater value: the value in the trail or the one in the database. If the
value in the trail is greater, apply the record but change the insert to an update. If
the database value is higher, ignore the record.
• For an UPDATEROWEXISTS conflict, use the USEMAX resolution: If the row exists during
an update, use the last_mod_time column as the resolution column: If the value in
the trail is greater, apply the update.
• If you use USEMIN or USEMAX, and the values are exactly the same, then
RESOLVECONFLICT isn't triggered and the incoming row is ignored. If you use USEMINEQ
or USEMAXEQ, and the values are exactly the same, then the resolution is triggered.
• For a DELETEROWEXISTS conflict, use the OVERWRITE resolution: If the row exists during
a delete operation, apply the delete.
• For an UPDATEROWMISSING conflict, use the OVERWRITE resolution: If the row does not
exist during an update, change the update to an insert and apply it.
10-9
Chapter 10
CDR Example 1: All Conflict Types with USEMAX, OVERWRITE, DISCARD
• For a DELETROWMISSING conflict use the DISCARD resolution: If the row does not exist
during a delete operation, discard the trail record.
Note:
As an alternative to USEMAX, you can use the USEMAXEQ resolution to apply
a >= condition. For more information, see Reference for Oracle
GoldenGate.
10-10
Chapter 10
CDR Example 1: All Conflict Types with USEMAX, OVERWRITE, DISCARD
UPDATE applied by Replicat SQL bind variables: Because USEMAX is specified for
to resolve the conflict INSERTROWEXISTS, Replicat converts the
1)'1234567890'
insert to an update, and it compares the
2)'Oracle Pkwy'
value of last_mod_time in the trail record
3)100
with the value in the database. The value in
4)100
the record is greater, so the after images for
5)NULL
columns in the trail file are applied to the
6)'9/1/10 3:00'
target.
7)'Mary'
8)'9/1/10 3:00'
10-11
Chapter 10
CDR Example 1: All Conflict Types with USEMAX, OVERWRITE, DISCARD
Initial UPDATE applied by SQL bind variables: This SQL returns a no-data-found error
Replicat that detects the because the values for the balance,
conflict 1)'222222' comment, and last_mod_time are different
2)'Holly'
in the target.
3)'9/1/10 5:00'
4)'Mary' All columns are used in the WHERE clause
5)'1234567890' because the COMPARECOLS statement is
6)'Oracle Pkwy' set to ALL.
7)100
8)100
9)NULL
10)'9/1/10 3:00'
UPDATE applied by Replicat to SQL bind variables: Because the after value of last_mod_time
resolve the conflict in the trail record is less than the current
1)'Mary' value in the database, the database value
2)'222222' is retained. Replicat applies the operation
3)'Holly' with a WHERE clause that contains the
4)100
primary key plus a last_mod_time value
5)100
set to less than 9/1/10 5:00. No rows
6)NULL
match this criteria, so the statement fails
7)'9/1/10 5:00'
with a "data not found" error, but Replicat
8)'Mary'
ignores the error because a USEMAX
9)'9/1/10 5:00'
resolution is expected to fail if the
condition is not satisfied.
10-12
Chapter 10
CDR Example 1: All Conflict Types with USEMAX, OVERWRITE, DISCARD
Initial UPDATE applied by SQL bind variables: This SQL returns a no-data-found error.
Replicat that detects the All columns are used in the WHERE clause
conflict 1)'4444' because the COMPARECOLS statement is
2)'Holly'
set to ALL.
3)'9/1/10 8:00'
4)'Jane'
5)'333'
6)'Oracle Pkwy'
7)200
8)200
9)NULL
10)'9/1/10 7:00'
INSERT applied by Replicat SQL bind variables: The update is converted to an insert
to resolve the conflict because OVERWRITE is the resolution.
1)'Jane' The after image of a column is used if
2)'4444' available; otherwise the before image is
3)'Holly' used.
4)200
5)200
6)NULL
7)'9/1/10 8:00'
10-13
Chapter 10
CDR Example 1: All Conflict Types with USEMAX, OVERWRITE, DISCARD
Initial DELETE applied by SQL bind variables: This SQL returns a no-data-found error.
Replicat that detects the All columns are used in the WHERE clause
conflict 1)'Jane' because the COMPARECOLS statement is
2)'4444'
set to ALL.
3)'Holly'
4)200
5)200
6)NULL
7)'9/1/10 8:00'
10-14
Chapter 10
CDR Example 2: UPDATEROWEXISTS with USEDELTA and USEMAX
Initial DELETE applied by SQL bind variables: All columns are used in the WHERE clause
Replicat that detects the because the COMPARECOLS statement is
conflict 1)'Mary'
set to ALL.
2)'222222'
3)'Holly' A no-data-found error occurs because of
4)100 the difference between the before and
5)100d current values.
6)NULL
7)'9/1/10 5:00'
DELETE applied by Replicat SQL bind variables: Because OVERWRITE is the resolution. the
to resolve the conflict DELETE is applied using only the primary
1)'Mary'
key (to avoid an integrity error).
10-15
Chapter 10
CDR Example 2: UPDATEROWEXISTS with USEDELTA and USEMAX
Per COMPARECOLS, use the primary key (name column) plus the address, phone, salary,
and last_mod_time columns as the comparison columns for conflict detection for UPDATE
and DELETE operations. (The balance and comment columns are not compared.)
Note:
As an alternative to USEMAX, you can use the USEMAXEQ resolution to apply a >=
condition. For more information, see Reference for Oracle GoldenGate.
10-16
Chapter 10
CDR Example 2: UPDATEROWEXISTS with USEDELTA and USEMAX
Initial UPDATE applied by SQL bind variables: This SQL returns a no-data-found error
Replicat that detects the because the values for the salary and
conflict 1)'222222' last_mod_time are different. (The values
2)'Holly'
for comment and balance are also
3)200
different, but these columns are not
4)'new'
compared.)
5)'9/1/10 5:00'
6)'Mary'
7)'1234567890'
8)'Oracle Pkwy'
9)100
10)'9/1/10 3:00'
UPDATE applied by Replicat SQL bind variables: Per USEDELTA, the difference between
to resolve the conflict for the after image of salary (200) in the
salary, using USEDELTA. 1)200
trail and the before image of salary
2)100
(100) in the trail is added to the current
3)'Mary'
value of salary in the target (600). The
result is 700.
600 + (200 - 100) = 700
UPDATE applied by Replicat SQL bind variables: Per USEMAX, because the after value of
to resolve the conflict for the last_mod_time in the trail record is
default columns, using 1)'222222'
greater than the current value in the
2)'Holly'
USEMAX. database, the row is updated with the
3)'new'
after values from the trail record.
4)'9/1/10 5:00'
5)'Mary' Note that the salary column is not set
6)'9/1/10 5:00' here, because it is resolved with the
UPDATE from the USEDELTA resolution.
10-17
Chapter 10
CDR Example 3: UPDATEROWEXISTS with USEDELTA, USEMAX, and IGNORE
10-18
Chapter 10
CDR Example 3: UPDATEROWEXISTS with USEDELTA, USEMAX, and IGNORE
– Per DEFAULT, use the IGNORE resolution logic for the remaining columns (phone
and comment) in the table (the default column group). Changes to these
columns will always be ignored by Replicat.
• Per COMPARECOLS, use all columns except the comment column as the comparison
columns for conflict detection for UPDATE operations. Comment will not be used in
the WHERE clause for updates, but all other columns that have a before image in the
trail record will be used.
Note:
As an alternative to USEMAX, you can use the USEMAXEQ resolution to apply
a >= condition. For more information, see Reference for Oracle
GoldenGate.
10-19
Chapter 10
CDR Example 3: UPDATEROWEXISTS with USEDELTA, USEMAX, and IGNORE
UPDATE applied by Replicat SQL bind variables: For salary, there is a difference of 100,
to resolve the conflict for but there was no change in value for
salary, using USEDELTA. 1)200 balance, so it is not needed in the
2)100
update SQL. Per USEDELTA, the
3)'Mary'
difference (delta) between the after
(200) image and the before image (100)
of salary in the trail is added to the
current value of salary in the target
(600). The result is 700.
UPDATE applied by Replicat SQL bind variables: Because the after value of
to resolve the conflict for last_mod_time in the trail record is
USEMAX. 1)'Holly' greater than the current value in the
2)'9/1/10 5:00' database, that column plus the address
3)'Mary'
column are updated with the after values
4)'9/1/10 5:00'
from the trail record.
Note that the salary column is not set
here, because it is resolved with the
UPDATE from the USEDELTA resolution.
UPDATE applied by Replicat SQL bind variables: IGNORE is specified for the DEFAULT
for IGNORE. column group (phone and comment), so
1)'222222'
no resolution SQL is applied.
2)'new'
3)'Mary'
10-20
11
Mapping and Manipulating Data
This chapter describe how you can integrate data between source and target tables.
Topics:
11-1
Chapter 11
Globalization Considerations when Mapping Data
11-2
Chapter 11
Globalization Considerations when Mapping Data
Note:
Oracle GoldenGate supports timestamp data from 0001-01-03 00:00:00 to
9999-12-31 23:59:59. If a timestamp is converted from GMT to local time,
these limits also apply to the resulting timestamp. Depending on the
timezone, conversion may add or subtract hours, which can cause the
timestamp to exceed the lower or upper supported limit.
11-3
Chapter 11
Globalization Considerations when Mapping Data
• An object name
• WHERE clause
Note:
To specify an actual backslash in the parameter file, specify a double
backslash. For example, the following finds a backslash in COL1: @STRFIND
(COL1, '\\' ).
11-4
Chapter 11
Mapping Columns
– A to F (U+0041 to U+0046)
– a to f (U+0061 to U+0066)
\u20ac is the Unicode escape sequence for the Euro currency sign.
Note:
For reliable cross-platform support, use the Unicode escape sequence. Octal
and hexadecimal escape sequences are not standardized on different
operating systems.
– A to F (U+0041 to U+0046)
– a to f (U+0061 to U+0066)
\x80 is the hexadecimal escape sequence for the Euro currency sign on Microsoft
Windows 1252 Latin1 code page.
11-5
Chapter 11
Mapping Columns
• map individual source columns to target columns that have different names.
• specify default column mapping when an explicit column mapping is not needed.
• Provide instructions for selecting, mapping, translating, and moving data from a
source column into a target column.
In this syntax, target_column is the name of the target column, and source_expression
can be any of the following, allowing you to map the source column by name, so as to
pass the source value exactly as recorded in the trail, or to transform the data before
passing it to the target column:
• The name of a source column, such as ORD_DATE.
• Numeric constant, such as 123.
• String constant enclosed within single quotes, such as 'ABCD'.
• An expression using an Oracle GoldenGate column-conversion function. Within a
COLMAP statement, you can employ any of the Oracle GoldenGate column-
conversion functions to transform data for the mapped columns, for example:
@STREXT (COL1, 1, 3)
If the column mapping involves case-sensitive columns from different database types,
specify each column as it is stored in the database.
• If the database requires double quotes to enforce case-sensitivity, specify the
case-sensitive column name within double quotes.
• If the database is case-sensitive without requiring double quotes, specify the
column name as it is stored in the database.
The following shows a mapping between a target column in an Oracle database and a
source column in a case-sensitive SQL Server database.
COLMAP ("ColA" = ColA)
11-6
Chapter 11
Mapping Columns
See Specifying Object Names in Oracle GoldenGate Input (page 4-17) for more
information about specifying names to Oracle GoldenGate.
See Globalization Considerations when Mapping Data (page 11-2) for globalization
considerations when mapping source and target columns in databases that have
different character sets and locales.
Avoid using COLMAP to map a value to a key column (which causes the operation to
become a primary key update), The WHERE clause that Oracle GoldenGate uses to
locate the target row will not use the correct before image of the key column. Instead,
it will use the after image. This will cause errors if you are using any functions based
on that key column, such as a SQLEXEC statement, as shown in the following example.
SQLEXEC (id mytest, query 'select city from TCUSTMER1 WHERE state = 'CA'',
noparams, ERROR RAISE)
• COLMAP statement in the MAP statement:
INSERT into TCUSTMER1 values (Cust = '1234', Name = 'Ace', City = 'SF', State =
'CA');
Commit;
The SQLEXEC query returns the correct value, and the target table also has a value
of SF for City and CA for State.
mytest.city = 'SF'
2. UPDATE statement changes City from SF to LA on the source. This does not succeed
on the target. The SQLEXEC query looks up the City column in TCUSTMER1 and returns
a value of LA. Based on the COLMAP clause, the before and after versions of City
both are now LA. This generates SQL error 1403 when executing the target WHERE
clause, because a value of LA does not exist for the City column in the target table.
11-7
Chapter 11
Mapping Columns
Default mapping causes Oracle GoldenGate to map those columns and, if required,
translate the data types based on the data-definitions file (see Determining Whether
COLMAP Requires a Data-definitions File (page 11-9)). Do not specify default
mapping for columns that are mapped already with an explicit mapping statement.
The following example of a column mapping illustrates the use of both default and
explicit column mapping for a source table ACCTBL and a target table ACCTTAB. Most
columns are the same in both tables, except for the following differences:
• The source table has a CUST_NAME column, whereas the target table has a NAME
column.
• A ten-digit PHONE_NO column in the source table corresponds to separate AREA_CODE,
PHONE_PREFIX, and PHONE_NUMBER columns in the target table.
• Separate YY, MM, and DD columns in the source table correspond to a single
TRANSACTION_DATE column in the target table.
See Understanding Default Column Mapping (page 11-12) for more information
about the rules followed by Oracle GoldenGate for default column mapping.
11-8
Chapter 11
Mapping Columns
11-9
Chapter 11
Mapping Columns
• For Oracle Database and DB2 databases, where names can be either case-
sensitive or case-insensitive in the same database and double quotes are required
to show case-sensitivity, COLMATCH requires an exact case and name match when a
name is in quotes in the database.
See Specifying Object Names in Oracle GoldenGate Input (page 4-17) for more
information about case-sensitivity support.
Syntax
COLMATCH
{NAMES target_column = source_column |
PREFIX prefix |
SUFFIX suffix |
RESET}
Argument Description
Maps based on column names.
NAMES target_column = source_column
Put double quotes around the column name if
it is case-sensitive and the database requires
quotes to enforce case-sensitivity. For these
database types, an unquoted column name is
treated as case-insensitive by Oracle
GoldenGate.
For databases that support case-sensitivity
without requiring quotes, specify the column
name as it is stored in the database.
If the COLMATCH is between columns in different
database types, make certain the names
reflect the appropriate case representation for
each one. For example, the following specifies
a case-sensitive target column name "aBc" in
an Oracle Database and a case-sensitive
source column name aBc in a case-sensitive
SQL Server database.
COLMATCH NAMES "aBc" = aBc
11-10
Chapter 11
Mapping Columns
Argument Description
Ignores the specified name prefix or suffix.
PREFIX prefix | SUFFIX suffix
Put double quotes around the prefix or suffix if
the database requires quotes to enforce case-
sensitivity, for example "P_". For those
database types, an unquoted prefix or suffix is
treated as case-insensitive.
For databases that support case-sensitivity
without requiring quotes, specify the prefix or
suffix as it is stored in the database. For
example, P_ specifies a capital P prefix.
The following example specifies a case-
insensitive prefix to ignore. The target column
name P_ABC is mapped to source column
name ABC, and target column name P_abc is
mapped to source column name abc.
COLMATCH PREFIX p_
The following example illustrates when to use COLMATCH. The source and target tables
are identical except for slightly different table and column names.The database is
case-insensitive.
CUST_CODE CUST_CODE
CUST_NAME CUST_NAME
CUST_ADDR ORDER_ID
PHONE ORDER_AMT
S_REP S_REP
S_REPCODE S_REPCODE
11-11
Chapter 11
Mapping Columns
CUSTOMER_CODE CUSTOMER_CODE
CUSTOMER_NAME CUSTOMER_NAME
CUSTOMER_ADDRESS ORDER_ID
PHONE ORDER_AMT
REP REP
REPCODE REPCODE
To map the source columns to the target columns in this example, as well as to handle
subsequent maps for other tables, the syntax is:
COLMATCH NAMES CUSTOMER_CODE = CUST_CODE
COLMATCH NAMES CUSTOMER_NAME = CUST_NAME
COLMATCH NAMES CUSTOMER_ADDRESS = CUST_ADDR
COLMATCH PREFIX S_
MAP SALES.ACCT, TARGET SALES.ACCOUNT, COLMAP (USEDEFAULTS);
MAP SALE.ORD, TARGET SALES.ORDER, COLMAP (USEDEFAULTS);
COLMATCH RESET
MAP SALES.REG, TARGET SALE.REG;
MAP SALES.PRICE, TARGET SALES.PRICE;
11-12
Chapter 11
Mapping Columns
• Target columns that do not correspond to any source column take default values
determined by the database.
If the default mapping cannot be performed, the target column defaults to one of the
values shown in Table 11-5 (page 11-13).
11-13
Chapter 11
Selecting and Filtering Rows
the resulting timestamp. Depending on the timezone, conversion may add or subtract
hours, which can cause the timestamp to exceed the lower or upper supported limit.
Required precision varies according to the data type and target platform. If the scale of
the target column is smaller than that of the source, data is truncated on the right. If
the scale of the target column is larger than that of the source, the column is extended
on the right with the values for the current date and time.
The FILTER clause offers you more functionality than the WHERE clause because you can
employ any of the Oracle GoldenGate column conversion functions, whereas the WHERE
clause accepts basic WHERE operators.
Note:
To filter a column based on a string, use one of the Oracle GoldenGate string
functions or use a WHERE clause.
The sytax for FILTER in a MAP statement is as follows and includes an error-handling
option.
MAP source_table, TARGET target_table,
, FILTER (
[, ON INSERT | ON UPDATE| ON DELETE]
[, IGNORE INSERT | IGNORE UPDATE | IGNORE DELETE]
[, RAISEERROR error_number]
, filter_clause);
11-14
Chapter 11
Selecting and Filtering Rows
– - (minus)
– * (multiply)
– / (divide)
– \ (remainder)
• Comparison operators:
– > (greater than)
– = (equal)
Use the RAISEERROR option of FILTER in the MAP parameter to generate a user-defined
error when the filter fails. This option is useful when you need to trigger an event in
response to the failure.
You can use the @RANGE function to divide the processing workload among multiple
FILTER clauses, using separate TABLE or MAP statements. For example, the following
splits the replication workload into two ranges (between two Replicat processes or two
threads of a coordinated Replicat) based on the ID column of the source acct table.
11-15
Chapter 11
Selecting and Filtering Rows
You can combine several FILTER clauses in one MAP or TABLE statement, as shown in
Table 11-6 (page 11-15), which shows part of a Replicat parameter file. Oracle
GoldenGate executes the filters in the order listed, until one fails or until all are
passed. If one filter fails, they all fail.
11-16
Chapter 11
Selecting and Filtering Rows
Element Examples
Column names
PRODUCT_AMT
Numeric values
-123, 5500.123
Literal strings
'AUTO', 'Ca'
Built-in column tests @NULL, @PRESENT, @ABSENT (column is null, present or absent in the
row). These tests are built into Oracle GoldenGate. See
Considerations for Selecting Rows with FILTER and WHERE
(page 11-17).
Comparison operators =, <>, >, <, >=, <=
Conjunctive operators
AND, OR
Grouping parentheses Use open and close parentheses ( ) for logical grouping of multiple
elements.
Oracle GoldenGate does not support FILTER for columns that have a multi-byte
character set or a character set that is incompatible with the character set of the local
operating system.
Arithmetic operators and floating-point data types are not supported by WHERE. To use
more complex selection conditions, use a FILTER clause or a user exit routine. See
Using User Exits to Extend Oracle GoldenGate Capabilities (page 16-16) for more
information.
The syntax for WHERE is identical in the TABLE and MAP statements:
TABLE table, WHERE (clause);
Note:
The examples in this section assume a case-insensitive database.
11-17
Chapter 11
Selecting and Filtering Rows
The following example returns all records when the amount column is over 10,000
and does not cause a record to be discarded when amount is absent.
WHERE (amount = @PRESENT AND amount > 10000)
The following returns TRUE if the column is NULL, and FALSE for all other cases (including
a column missing from the record).
WHERE (amount = @NULL)
The following returns TRUE only if the column is present in the record and not NULL.
WHERE (amount = @PRESENT AND amount <> @NULL)
11-18
Chapter 11
Retrieving Before and After Values
Note:
The previous example indicates a case-sensitive database such as
Oracle. The table names are in quote marks to reflect case-sensitivity.
Restricting the columns that are extracted can be useful when a target table does not
contain the same columns as the source table, or when the columns contain sensitive
information, such as a personal identification number or other proprietary business
information.
11-19
Chapter 11
Selecting and Converting SQL Operations
GETUPDATES | IGNOREUPDATES
GETDELETES | IGNOREDELETES
You can convert one type of SQL operation to another by using the following
parameters in the Replicat parameter file:
• Use INSERTUPDATES to convert source update operations to inserts into the target
table. This is useful for maintaining a transaction history on that table. The
transaction log record must contain all of the column values of the table, not just
changed values. Some databases do not log full row values to their transaction
log, but only values that changed.
• Use INSERTDELETES to convert all source delete operations to inserts into the target
table. This is useful for retaining a history of all records that were ever in the
source database.
• Use UPDATEDELETES to convert source deletes to updates on the target.
Retaining this history as a series of records can be useful in many ways. For example,
you can generate the net effect of transactions.
11-20
Chapter 11
Testing and Transforming Data
Replicat
INSERTALLRECORDS
MAP SALES.CUSTOMER, TARGET SALES.CUSTHIST,
COLMAP (TS = @GETENV ('GGHEADER', 'COMMITTIMESTAMP'),
BEFORE_AFTER = @GETENV ('GGHEADER', 'BEFOREAFTERINDICATOR'),
OP_TYPE = @GETENV ('GGHEADER', 'OPTYPE'),
ID = ID,
BALANCE = BALANCE);
Note:
This is not representative of a complete parameter file for an Oracle
GoldenGate process. Also note that these examples represent a case-
insensitive database.
This configuration makes possible queries such as the following, which returns the net
sum of each transaction along with the time of the transaction and the customer ID.
SELECT AFTER.ID, AFTER.TS, AFTER.BALANCE - BEFORE.BALANCE
FROM CUSTHIST AFTER, CUSTHIST BEFORE
WHERE AFTER.ID = BEFORE.ID AND AFTER.TS = BEFORE.TS AND
AFTER.BEFORE_AFTER = 'A' AND BEFORE.BEFORE_AFTER = 'B';
11-21
Chapter 11
Testing and Transforming Data
• Transform dates.
• Test for the presence of column values.
• Perform arithmetic operations.
• Manipulate numbers and character strings.
• Handle null, invalid, and missing data.
• Perform tests.
This chapter provides an overview of some of the Oracle GoldenGate functions related
to data manipulation. For the complete reference, see Reference for Oracle
GoldenGate for Windows and UNIX.
If you need to use logic beyond that which is supplied by the Oracle GoldenGate
functions, you can call your own functions by implementing Oracle GoldenGate user
exits. See Using User Exits to Extend Oracle GoldenGate Capabilities (page 16-16)
for more information about user exits.
Oracle GoldenGate conversion functions take the following general syntax:
Syntax
@function (argument)
11-22
Chapter 11
Testing and Transforming Data
See Manipulating Numbers and Character Strings (page 11-25) for more information.
Note:
Errors in argument parsing sometimes are not detected until records are
processed. Verify syntax before starting processes.
11-23
Chapter 11
Testing and Transforming Data
@DATE ('JTS',
'YYMMDDHHMISS',
ORDER_TAKEN_TIME) +
ORDER_MINUTES * 60 * 1000000)
– - (minus)
– * (multiply)
– / (divide)
– \ (remainder)
• Comparison operators:
– > (greater than)
– = (equal)
Results that are derived from comparisons can be zero (indicating FALSE) or non-
zero (indicating TRUE).
• Parentheses (for grouping results in the expression)
• The conjunction operators AND, OR. Oracle GoldenGate only evaluates the
necessary part of a conjunction expression. Once a statement is FALSE, the rest of
the expression is ignored. This can be valuable when evaluating fields that may be
missing or null. For example, if the value of COL1 is 25 and the value of COL2 is 10,
then the following are possible:
@COMPUTE ( (COL1 > 0) AND (COL2 < 3) ) returns 0.
@COMPUTE ( (COL1 < 0) AND (COL2 < 3) ) returns 0. COL2 < 3 is never evaluated.
@COMPUTE ((COL1 + COL2)/5) returns 7.
11-24
Chapter 11
Testing and Transforming Data
The following expression returns the same result as the previous one:
@STRNUM ((@COMPUTE (AMOUNT1 + AMOUNT2), LEFT)
11-25
Chapter 11
Testing and Transforming Data
The following @IF calculation uses @COLSTAT to return NULL to the target column if PRICE
and QUANTITY are less than zero.
ORDER_TOTAL = PRICE * QUANTITY, @IF ((PRICE < 0) AND (QUANTITY < 0), @COLSTAT (NULL))
The following example checks whether the AMOUNT column is present and NULL and
whether it is present but invalid.
@COLTEST (AMOUNT, NULL, INVALID)
11-26
Chapter 11
Using Tokens
In this example, if STATE is CA or NY, the expression returns COAST, which is the response
returned by @IF when the value is non-zero (meaning TRUE).
• somewhat high if AMOUNT is greater than 5000, and less than or equal to 10000, (unless
the prior condition was satisfied)
• A FIELD_MISSING indication if neither condition is satisfied.
Syntax
TABLE table_spec, TOKENS (token_name = token_data [, ...]);
Where:
• table_spec is the name of the source table. A container or catalog name, if
applicable, and an owner name must precede the table name.
11-27
Chapter 11
Using Tokens
• token_name is a name of your choice for the token. It can be any number of
alphanumeric characters and is not case-sensitive.
• token_data is a character string of up to 2000 bytes. The data can be either a string
that is enclosed within single quotes or the result of an Oracle GoldenGate
column-conversion function. The character set of token data is not converted. The
token must be in the character set of the source database for Extract and in the
character set of the target database for Replicat. In the trail file, user tokens are
stored in UTF-8.
TABLE ora.oratest, TOKENS (
TK-OSUSER = @GETENV ('GGENVIRONMENT' , 'OSUSERNAME'),
TK-GROUP = @GETENV ('GGENVIRONMENT' , 'GROUPNAME')
TK-HOST = @GETENV('GGENVIRONMENT' , 'HOSTNAME'));
As shown in this example, the Oracle GoldenGate @GETENV function is an effective way
to populate token data. This function provides several options for capturing
environment information that can be mapped to tokens and then used on the target
system for column mapping.
Syntax
COLMAP (target_column = @TOKEN ('token_name'))
The following MAP statement maps target columns host, gg_group, and so forth to tokens
tk-host, tk-group, and so forth. Note that the arguments must be enclosed within
single quotes.
tk-host :sysA
tk-group :extora
tk-osuser :jad
tk-domain :admin
tk-ba_ind :B
tk-pos :3604496
tk-rba :4058
11-28
Chapter 11
Using Tokens
tk-table :oratest
tk-optype :insert
The tokens in this example will look similar to the following within the record header in
the trail:
11-29
12
Associating Replicated Data with Metadata
This chapter describes the uses of metadata and how to associate replicated data with
metadata.
Topics:
12.1 Overview
When replicating data from one table to another, an important consideration is whether
the column structures (metadata) of the source and target tables are identical. Oracle
GoldenGate looks up metadata for the following purposes:
• On the source, to supply complete information about captured operations to the
Replicat process.
• On the target, to determine the structures of the target tables, so that the
replicated data is correctly mapped and converted (if needed) by Replicat.
In each of the following scenarios, you must use a different parameter or set of
parameters to describe the metadata properly to the Oracle GoldenGate process that
is processing it:
• You are replicating a source table to a target table that has identical metadata
definitions (homogeneous replication).
• You are replicating a source table to a target table that has different metadata
definitions.
• You are replicating a source table to two target tables, one with identical
definitions and one that has different definitions.
12-1
Chapter 12
Understanding Data Definition Files
metadata that indicates other data properties. Following the header are the table-
definition sections. Each table-definition section contains a table name, record length,
number of columns, and one or more column definitions.
12-2
Chapter 12
Understanding Data Definition Files
12-3
Chapter 12
Understanding Data Definition Files
configuration, then copy their contents to the existing master definitions file, and then
restart the process.
Note:
Do not create a data-definitions file for Oracle sequences. It is not
needed and DEFGEN does not support it.
Parameter Description
12-4
Chapter 12
Understanding Data Definition Files
Parameter Description
DEFSFILE file_name [APPEND | PURGE] [CHARSET Specifies the relative or fully qualified name of the data-
character_set] [FORMAT RELEASE major.minor] definitions file that is to be the output of DEFGEN.
See Reference for Oracle GoldenGate for important
• APPEND directs DEFGEN to write new content (from information about these parameter options and their
the current run) at the end of any existing content, if effect on character sets.
the specified file already exists.
See Understanding the Effect of Character Sets on
• PURGE directs DEFGEN to purge the specified file Definitions Files (page 12-2) for more information.
before writing new content from the current run.
This is the default.
• CHARSET generates the definitions file in the
specified character set instead of the default
character set of the operating system.
• FORMAT RELEASE specifies the Oracle GoldenGate
release version of the definitions file. Use when the
definitions file will be read by a process that is in an
earlier version of Oracle GoldenGate than the
DEFGEN process.
TABLE container. owner.table Specifies the fully qualified name of a table or tables for
[, {DEF | TARGETDEF} template]; which definitions will be defined and optionally uses the
metadata of the table as a basis for a definitions
Where: template. Case-sensitivity of both table name and
• container is a container in an Oracle container template name is preserved for case-sensitive
database. databases. See Specifying Object Names in Oracle
GoldenGate Input (page 4-17) for instructions on
• owner is the name of the schema that contains the
wildcarding and case-sensitivity.
table to be defined.
• table is the table that is to be defined. Specify a source table(s) if generating a source-
definitions file or a target table(s) if generating a target-
• [, {DEF | TARGETDEF} template] additionally
definitions file.
creates a definitions template based on the
metadata of this table. This option is not supported To exclude tables from a wildcard specification, use the
for initial loads. See Reference for Oracle TABLEEXCLUDE parameter.
GoldenGate for information about this option. Note that DEFGEN does not support UDTs.
12-5
Chapter 12
Understanding Data Definition Files
Where:
• defgen is the name of the program.
• Associate a source-definitions file with the Replicat group by using the SOURCEDEFS
parameter in the Replicat parameter file.
• If Oracle GoldenGate is to perform mapping or conversion on an intermediary
system that contains neither the source nor target database, associate a source-
definitions file and a target-definitions file with the data pump Extract by using
12-6
Chapter 12
Understanding Data Definition Files
SOURCEDEFS and TARGETDEFS in the parameter file. For Oracle databases, the Oracle
libraries also must be present on the intermediary system.
See Examples of Using a Definitions File (page 12-7) for the correct way to specify
multiple definitions files.
Do not use SOURCEDEFS and ASSUMETARGETDEFS in the same parameter file. See
Configuring Oracle GoldenGate to Assume Identical Metadata (page 12-14) for more
information about ASSUMETARGETDEFS.
Because these options direct the Extract or Replicat process to use the same
definitions as the specified template, you need not create a new definitions file for the
new table, nor restart the process.
12-7
Chapter 12
Understanding Data Definition Files
Note that definitions for tables that satisfy the wildcard specification acct.cust* are
obtained from the custdef template, as directed by the DEF option of the first MAP
statement.
Note:
See the previous example for the DEFGEN configuration that makes the
source-definitions file.
In this example, the source template named custdef (from the record.def file) and a
target template named tcustdef (from the trecord.def file) are used for the acc.cust*
tables. Definitions for the tables from the ord and hr schemas are obtained from
explicit definitions based on the table names (but a wildcard specification could have
been used here, instead)
12-8
Chapter 12
Using Automatic Trail File Recovery
12.2.7.3 Creating Multiple Source Definition Files for Use on a Target System
This is a simple example of how to use multiple definitions files. Your parameter
requirements may vary, based on the Oracle GoldenGate topology and database type.
The following is the DEFGEN parameter file that creates the first data-definitions file.
DEFSFILE C:\ggs\dirdef\sales.def
USERIDALIAS ogg
TABLE ord.*;
The following is the DEFGEN parameter file that creates the second data-definitions
file. Note the file name and table specification are different from the first one.
DEFSFILE C:\ggs\dirdef\admin.def
USERIDALIAS ogg
TABLE hr.*;
The tables for the first and second definitions file are mapped in the same Replicat
parameter file as follows:
REPLICAT acctrep
USERIDALIAS ogg
SOURCEDEFS c:\ggs\dirdef\sales.def
MAP ord.*, TARGET ord.*;
SOURCEDEFS c:\ggs\dirdef\admin.def
MAP hr.*, TARGET hr.*;
12-9
Chapter 12
Configuring Oracle GoldenGate to Use Self-Describing Trail Files
12-10
Chapter 12
Configuring Oracle GoldenGate to Use Self-Describing Trail Files
Using self-described trail files eliminates the need for SOURCEDEFS and ASSUMETARGETDEFS
so parameter files are simpler and it is simpler to configure heterogeneous replication
and provides:
• A reduction in trail file size due to object name compression.
• The ability to extract data from multiple catalogs with different character sets and
time zones into one trail.
• The ability to configure DDL replication among more than two Oracle databases.
There is no need to use the GETREPLICATS, UPDATEMETADATA, and NOTAG parameters.
You can replicate DDLs when source and target tables are not alike and without
having to synchronize Oracle GoldenGate processes .
• No necessity to create and maintain source definitions files.
12-11
Chapter 12
Configuring Oracle GoldenGate to Use Self-Describing Trail Files
completely output all the relevant database changes to the trail and is stopped. After
DDL is executed, you must restart the Extract. Unlike previous releases, there is no
need to stop Replicat and regenerate SOURCEDEFS using DEFGEN.
• For existing trail file configurations, you can easily switch between the previous
and self-describing extract trail methods of resolving the table metadata by:
– Use the USE_TRAILDEFS GLOBALS parameter to control all pumps and
Replicats.
– Use the OVERRIDE option of SOURCEDEFS and ASSUMETARGETDEFS to control an
individual pump or Replicat. Oracle does not recommend this.
• Logdump displays the metadata records similar to DEFGEN output.
• Reverse is not supported in the 12c (12.2.0.1) trail format.
• If a table is mapped, the generated TDR is based on the definition of the mapped
table not the source table.
• Metadata in the trail is supported for all databases except HP NonStop (Guardian).
12-12
Chapter 12
Configuring Oracle GoldenGate to Use Self-Describing Trail Files
You must use the OVERRIDE option with the ASSUMETARGETDEFS and SOURCEDEFS
parameters when using self-describing trail files.
12-13
Chapter 12
Configuring Oracle GoldenGate to Assume Identical Metadata
Note:
This section does not apply to self-describing trail files.
When source and target tables have identical metadata definitions, use the
ASSUMETARGETDEFS parameter in the Replicat parameter file. This parameter directs
Replicat to assume that the target definitions are the same as those of the source, and
to apply those definitions to the replicated data when constructing SQL statements.
The source and target tables must be identical in every way, thus needing no
conversion processing, although their catalogs or containers, owners and/or names
can be different.
12-14
Chapter 12
Configuring Oracle GoldenGate to Assume Dissimilar Metadata
When source and target structures are different, use the SOURCEDEFS parameter. See
Configuring Oracle GoldenGate to Assume Dissimilar Metadata (page 12-15).
ASSUMETARGETDEFS and SOURCEDEFS cannot be used in the same parameter file.
Note:
This section does not apply to self-describing trail files.
ASSUMETARGETDEFS and SOURCEDEFS can be used in the same parameter file. This can be
done when column mapping or conversion must be performed between some of the
source-target table pairs, but not for other table pairs that are identical.
The following is an example of how to use SOURCEDEFS and ASSUMETARGETDEFS in the
same parameter file. This example builds on the previous examples where tables in
the acct, ord, and hr schemas require SOURCEDEFS, but it adds a rpt schema with tables
that are dynamically created with the name stock appended with a random numerical
value. For Oracle GoldenGate to replicate the DDL as well as the DML, the target
tables must be identical. In that case, ASSUMETARGETDEFS is required.
REPLICAT acctrep
USERIDALIAS ogg
SOURCEDEFS c:\ggs\dirdef\record.def
MAP acct.cust*, TARGET acct.cust*, DEF custdef;
MAP ord.prod, TARGET ord.prod;
MAP ord.parts, TARGET ord.parts;
MAP hr.emp, TARGET hr.emp;
MAP hr.salary, TARGET hr.salary;
ASSUMETARGETDEFS
MAP rpt.stock, TARGET rpt.stock;
12-15
13
Configuring Online Change
Synchronization
This chapter describes how to configure online change synchronization.
Topics:
13-1
Chapter 13
Choosing Names for Processes and Files
13-2
Chapter 13
Creating a Checkpoint Table
13-3
Chapter 13
Creating a Checkpoint Table
More than one instance of Oracle GoldenGate (multiple installations) can use the
same checkpoint table. Oracle GoldenGate keeps track of the checkpoints, even if
Replicat group names are the same in different instances.
More than one checkpoint table can be used as needed. For example, you can use
different ones for different Replicat groups.
You can install your checkpoint tables in these ways:
• You can specify a default checkpoint table in the GLOBALS file. New Replicat groups
created with the ADD REPLICAT command will use this table automatically, without
requiring any special instructions. See "To Specify a Default Checkpoint Table in
the GLOBALS File (page 13-5)" for instructions.
• You can provide specific checkpoint table instructions when you create any given
Replicat group with the ADD REPLICAT command:
– To use a specific checkpoint table for a group, use the CHECKPOINTTABLE
argument of ADD REPLICAT. This checkpoint table overrides any default
specification in the GLOBALS file. If using only one Replicat group, you can use
this command and skip creating the GLOBALS file altogether.
– To omit using a checkpoint table for a group, use the NODBCHECKPOINT argument
of ADD REPLICAT. Without a checkpoint table, Replicat still maintains
checkpoints in a checkpoint file on disk, but you introduce the risk of data
inconsistency.
However you implement the checkpoint table, you must create it in the target database
prior to using the ADD REPLICAT command.
Where:
owner.table is the owner and name of the table, container is the name of a PDB if
installing into an Oracle multitenant container database. The owner and name can
be omitted if you are using this table as the default checkpoint table and this table
is specified with CHECKPOINTTABLE in the GLOBALS file. The name of this table must
not exceed the maximum length permitted by the database for object names. The
checkpoint table name cannot contain any special characters, such as quotes,
backslash, pound sign, and so forth.
13-4
Chapter 13
Creating an Online Extract Group
Where:
catalog.owner.table is the fully qualified name of the default checkpoint table,
including the name of the container if the database is an Oracle multitenant
container database (CDB).
3. Note the name of the table, then save and close the GLOBALS file. Make certain the
file was created in the root Oracle GoldenGate directory. If there is a file extension,
remove it.
13-5
Chapter 13
Creating an Online Extract Group
[, REPORT pathname]
[, DESC 'description']
Where:
• group is the name of the Extract group. A group name is required.
• datasource is required to specify the source of the data to be extracted. Use one of
the following:
– TRANLOG specifies the transaction log as the data source. Use for all databases
except Teradata. When using this option for Oracle Enterprise Edition, you
must issue the DBLOGIN command as the Extract database user (or a user with
the same privileges) before using ADD EXTRACT (and also before issuing DELETE
EXTRACT to remove an Extract group).
Use the bsds option for DB2 running on z/OS to specify the Bootstrap Data Set
file name of the transaction log.
– INTEGRATED TRANLOG specifies that this Extract will operate in integrated capture
mode to receive logical change records (LCR) from an Oracle Database
logmining server. This parameter applies only to Oracle Databases..
– VAM specifies that the Extract API known as the Vendor Access Module (VAM).
For Teradata it will interface with the Teradata Access Module (TAM).
– VAMTRAILSOURCE VAM trail name to specify a VAM trail.
Note:
Do not use the BEGIN parameter for a Teradata source.
• PASSIVE indicates that the group is a passive Extract. When using PASSIVE, you
must also use an alias Extract. This option can appear in any order among other
ADD EXTRACT options.
13-6
Chapter 13
Creating an Online Extract Group
Where:
• RMTHOST identifies this group as an alias Extract and specifies either the DNS name
of the remote host or its IP address.
• MGRPORT specifies the port on the remote system where Manager is running. Use
this option when using a dynamic Collector.
• PORT specifies a static Collector port. Use instead of MGRPORT only if running a static
Collector.
• RMTNAME specifies the passive Extract name, if different from that of the alias
Extract.
• DESC 'description' specifies a description of the group.
13-7
Chapter 13
Creating a Trail
13-8
Chapter 13
Creating a Trail
• The primary Extract process captures transactional data from the source database
and writes it to the local trail. A data-pump Extract reads that trail and then
transfers the data over the network to a remote trail on the target. If the network
fails, the data pump fails but the primary Extract continues to process data to the
local trail. There must be enough disk space to contain the data accumulation, or
the primary Extract will abend.
• For a trail at the target location, provide enough disk space to handle data
accumulation according to the purge rules set with the PURGEOLDEXTRACTS
parameter. Even with PURGEOLDEXTRACTS in use, data will always accumulate on the
target because it is transferred across the network faster than it can be applied to
the target database.
To prevent trail activity from interfering with business applications, assign a separate
disk or file system to contain the trail files. Trail files can reside on drives that are local
to the Oracle GoldenGate installation, or they can reside on NAS or SAN devices. In
an Oracle cluster, they can reside on ASM or DBFS storage.
Note:
This formula is a conservative estimate, and you should run tests once
you have configured Oracle GoldenGate to determine exactly how much
space you need.
13-9
Chapter 13
Creating a Parameter File for Online Extraction
Where:
• RMTTRAIL specifies a trail on a remote system.
– EXTTRAIL must be used to specify a local trail that is read by a data pump or to
specify a VAM trail that is linked to a primary Extract which interacts with a
Teradata Access Module (TAM). For more information about the Teradata
configuration, see Using Oracle GoldenGate for Heterogeneous Databases
• pathname is the relative or fully qualified name of the trail, including a two-character
name that can be any two alphanumeric characters, for example c:\ggs\dirdat\rt.
Oracle GoldenGate appends a serial number to each trail file as it is created
during processing. Typically, trails are stored in the dirdat sub-directory of the
Oracle GoldenGate directory.
• EXTRACT group specifies the name of the Extract group that writes to this trail. Only
one Extract group can write to a trail.
• MEGABYTES n is an optional argument with which you can set the size, in megabytes,
of each trail file (default is 100).
Example 13-8 VAM Trail
This example creates a VAM trail named /ggs/dirdat/vt for Extract group extvam.
ADD EXTTRAIL /ggs/dirdat/vt, EXTRACT extvam
Where:
name is either the name of the Extract group that you created with the ADD EXTRACT
command or the fully qualified name of the parameter file if you defined an
alternate location when you created the group.
2. Enter the parameters in Creating a Parameter File for Online Extraction
(page 13-10) in the order shown, starting a new line for each parameter statement.
Some parameters apply only for certain configurations.
13-10
Chapter 13
Creating a Parameter File for Online Extraction
Parameter Description
RMTHOSTOPTIONS host, Specifies the target system, the port where Manager is running, and
MGRPORT port, optional encryption of data across TCP/IP. Only required when sending
[, ENCRYPT algorithm KEYNAME data over IP to a remote system (if ADD RMTTRAIL was used to create
key_name] the trail). Not required if the trail is on the local system (if ADD EXTTRAIL
was used).
Not valid for a primary Extract group that interfaces with a Teradata
Access Module and writes to a VAM trail..
Not valid for a passive Extract group.
ENCRYPTTRAIL algorithm Encrypts all trails that are specified after this entry.
DECRYPTTRAIL (For a data pump) Decrypts the data in the input trail. Use only if the
data pump must process the data before writing it to the output trail.
RMTTRAIL pathname | Specifies a trail. If specifying multiple trails, follow each designation
EXTTRAIL pathname with the appropriate TABLE statements.
EXTTRAIL is not valid for a passive Extract group.
• Use RMTTRAIL to specify the relative
If trails or files will be of different versions, use the FORMAT option of
or fully qualified name of a remote
trail created with the ADD RMTTRAIL RMTTRAIL or EXTTRAIL. See EXTTRAILin Reference for Oracle
command. GoldenGate
• Use EXTTRAIL to specify the relative
or fully qualified name of a local trail
created with the ADD EXTTRAIL
command (to be read by a data
pump or VAM-sort Extract).
13-11
Chapter 13
Creating an Online Replicat Group
Parameter Description
LOGALLSUPCOLS Use when using integrated Replicat for an Oracle target, or when using
Conflict Detection and Resolution (CDR) support. Writes the before
images of scheduling columns to the trail. (Scheduling columns are
primary key, unique index, and foreign key columns.) See
LOGALLSUPCOLS in Reference for Oracle GoldenGate.
VAM library, Valid only for an Extract group that interfaces with a Teradata Access
PARAMS ('param' Module. Supplies the name of the library and parameters that must be
[, 'param'] [, ...]) passed to the Oracle GoldenGate API, such as the name of the TAM
initialization file and the program that interacts with the library as the
callback library.
Example:
VAM vam.dll, PARAMS ('inifile', 'vamerge1.ini', 'callbacklib',
'extract.exe')
SEQUENCE [container.]owner.sequence; Specifies the fully qualified name of an Oracle sequence to capture.
Include the container name if the database is a multitenant container
database (CDB).
TABLE [container. | Specifies the fully qualified name of an object or a fully qualified
catalog.]owner.object; wildcarded specification for multiple objects. If the database is an
Oracle multitenant container database, the object name must include
the name of the container or catalog unless SOURCECATALOG is used.
See Specifying Object Names in Oracle GoldenGate Input (page 4-17)
for guidelines for specifying object names in parameter files.
CATALOGEXCLUDE Parameters that can be used in conjunction with one another to
SCHEMAEXCLUDE exclude specific objects from a wildcard specification in the associated
TABLE statement.
TABLEEXCLUDE
EXCLUDEWILDCARDOBJECTSONLY
3. Enter any appropriate optional Extract parameters listed in the Oracle GoldenGate
Parameters in Reference for Oracle GoldenGate.
4. Save and close the parameter file.
13-12
Chapter 13
Creating an Online Replicat Group
As shown in Figure 13-1 (page 13-13), you can apply transactions in parallel with a
classic Replicat, but only by partitioning the workload across multiple Replicat
processes. A parameter file must be created for each Replicat.
To determine whether to use classic mode for any objects, you must determine
whether the objects in one Replicat group will ever have dependencies on objects in
any other Replicat group, transactional or otherwise. Not all workloads can be
partitioned across multiple Replicat groups and still preserve the original transaction
atomicity. For example, tables for which the workload routinely updates the primary
key cannot easily be partitioned in this manner. DDL replication (if supported for the
database) is not viable in this mode, nor is the use of some SQLEXEC or EVENTACTIONS
features that base their actions on a specific record.
13-13
Chapter 13
Creating an Online Replicat Group
If your tables do not have any foreign- key dependencies or updates to primary keys,
classic mode may be suitable. Classic mode requires less overhead than coordinated
mode.
For more information about using parallel Replicat groups, see Tuning the
Performance of Oracle GoldenGate (page 18-1).
13-14
Chapter 13
Creating an Online Replicat Group
Optionally, you can force other transactions to be treated like a barrier transaction
through the use of the COORDINATED keyword in a MAP statement. One use case for this
would be force a SQLEXEC to be executed in a manner similar to a serial execution. This
could be beneficial if the results can become ambiguous unless the state of the target
is consistent across all transactions.
13-15
Chapter 13
Creating an Online Replicat Group
Note:
Coordinated Replicat doesn't do dependency calculations for non-barrier
transactions when a mapped table is partitioned based on THNREADRANGE. It
relies on specified THREADRANGE columns to compute a hash value. It partitions
the incoming data based on the hash value and sends all the records that
match this hash value to same thread.
Note:
Coordinated Replicat is an online process only. Do not use it to perform initial
loads.
13-16
Chapter 13
Creating an Online Replicat Group
records that represent source DML or DDL transactions, and transmits these records
to an inbound server in the Oracle target database. The inbound server applies the
data to the target database.
For more information about using integrated Replicat, see About Integrated Mode in
Using Oracle GoldenGate for Oracle Database.
Note:
Integrated Replicat is an online process only. Do not use it to perform initial
loads.
Changes to the Replicat configuration should not be made after Replicat abends or is
forcibly terminated. Replicat should be allowed to recover to its last checkpoint after
startup. For coordinated Replicat, you can follow the administrative procedures in
Administering a Coordinated Replicat Configuration (page 19-26).. Once the recovery
is complete, Replicat can be shut down gracefully with the STOP REPLICAT command,
and then you can make the changes to the object specifications.
13-17
Chapter 13
Creating an Online Replicat Group
Where:
• group is the name of the Replicat group. A group name is required. See Naming
Conventions for Processes (page 13-2) for Oracle GoldenGate naming
conventions.
• EXTTRAIL path is the relative or fully qualified name of the trail that you defined with
the ADD RMTTRAIL command.
• INTEGRATED specified that this Replicat group will operate in integrated mode. This
mode is available for Oracle databases..
• COORDINATED specifies that this Replicat group will operate in coordinated mode.
MAXTHREADS specifies the maximum number of threads allowed for this group. Valid
values are from 1 through 500. MAXTHREADS is optional. The default number of
threads without MAXTHREADS is 25.
Note:
Each Replicat thread is considered a Replicat group in the context of the
MAXGROUPS parameter. MAXGROUPS controls the maximum number of
process groups allowed in the Oracle GoldenGate instance. MAXTHREADS
plus the number of other process groups in the Oracle GoldenGate
instance must not exceed the value set with MAXGROUPS (default is 1000).
• PARAMS path is required if the parameter file for this group will be stored in a
location other than the dirprm sub-directory of the Oracle GoldenGate directory.
Specify the fully qualified name. The default location is recommended.
• REPORT path is required if the process report for this group will be stored in a
location other than the dirrpt sub-directory of the Oracle GoldenGate directory.
Specify the fully qualified name. The default location is recommended.
13-18
Chapter 13
Creating a Parameter File for Online Replication
Where:
name is either the name of the Replicat group that you created with the ADD
REPLICAT command or the fully qualified name of the parameter file if you defined
an alternate location when you created the group.
2. Enter the parameters listed in Table 13-2 (page 13-19) in the order shown,
starting a new line for each parameter statement.
Parameter Description
Configures Replicat as an online process with
REPLICAT group
checkpoints.
• group is the name of the Replicat group that you
created with the ADD REPLICAT command.
Specifies how to interpret data definitions.
{SOURCEDEFS path} |
ASSUMETARGETDEFS For Oracle databases that use multi-byte character sets,
you must use SOURCEDEFS (with a DEFGEN-generated
• Use SOURCEDEFS if the source and target tables have definitions file) if the source semantics setting is in bytes
different definitions. Specify the source data- and the target is in characters. This is required even
definitions file generated by DEFGEN. See when the source and target data definitions are
Associating Replicated Data with Metadata identical. See Associating Replicated Data with
(page 12-1), for more information. Metadata (page 12-1), for more information.
• Use ASSUMETARGETDEFS if the source and target
tables have the same definitions.
Optional. Specifies an amount of time for Replicat to
[DEFERAPPLYINTERVAL n unit]
wait before applying its transactions to the target
system.
• n is a numeric value for the amount of time to delay
before applying transactions. Minimum is set by the
EOFDELAY parameter. Maximum is seven days.
• unit can be:
S | SEC | SECS | SECOND | SECONDS | MIN |
MINS | MINUTE | MINUTES | HOUR | HOURS | DAY
| DAYS
13-19
Chapter 13
Creating a Parameter File for Online Replication
Parameter Description
Specifies database connection information.
[TARGETDB dsn | container | catalog]
[, USERIDALIAS alias options | TARGETDB specifies the target datasource name (DSN).
, USERID user, options] See TARGETDB in Reference for Oracle GoldenGatefor
more information .
USERID and USERIDALIAS specify database credentials if
required.
HANDLECOLLISIONS Specifies collision handling. Use only if you are
performing an initial load concurrently with starting
online processing and the source database will remain
active during the load. HANDLECOLLISIONS resolves the
results of the copy with the ongoing replicated
transactional changes. It resolves insert operations for
which the row already exists and update and delete
operations for which the row does not exist. It can be
used globally for all MAP statements in a parameter file
or within a MAP statement, or both.
SOURCECATALOG Specifies a default container in a source Oracle
multitenant container database. Enables the use of two-
part names (schema.object) where three-part names
otherwise would be required for those databases. You
can use multiple instances of this parameter to specify
different default containers or catalogs for different sets
of MAP parameters.
Specifies a relationship between a source object or
MAP [container. | catalog.]owner.object,
objects and a target object or objects. MAP specifies the
TARGET owner.object[, DEF template]
source object, and TARGET specifies the target object.
[THREAD (thread_ID)]
[THREADRANGE (thread_range[, column_list])] For the source object, specify the fully qualified name of
[COORDINATED] the object or a fully qualified wildcarded specification for
; multiple objects. For an Oracle multitenant container
database the source object name must include the
name of the container or catalog unless SOURCECATALOG
is used.
For the target object, specify only the owner.object
components of the name, regardless of the type of
database. Replicat can only connect to one Oracle
container. Use a separate Replicat process for each
container or catalog to which you want to apply data.
SeeSpecifying Object Names in Oracle GoldenGate
Input (page 4-17) for guidelines for specifying object
names in parameter files.
The THREAD, THREADRANGE, and COORDINATED options are
valid for Replicat when in coordinated mode. They
enable you to partition the workload to one or more
specific Replicat threads. See in Reference for Oracle
GoldenGatefor syntax and usage.
The DEF option specifies a definitions template. See
Associating Replicated Data with Metadata (page 12-1)
for more information about data definitions.
13-20
Chapter 13
Creating a Parameter File for Online Replication
Parameter Description
CATALOGEXCLUDE Parameters that can be used in conjunction with one
SCHEMAEXCLUDE another to exclude specific source objects from a
wildcard specification in the associated MAP statement.
MAPEXCLUDE
EXCLUDEWILDCARDOBJECTSONLY
Note:
If using integrated Replicat for Oracle, see Understanding Replicat
Processing in Relation to Parameter Changes (page 13-17) for important
information about making configuration changes to Replicat once processing
is started.
13-21
14
Handling Processing Errors
This chapter describes how to configure the Oracle GoldenGate processes to handle
errors.
Oracle GoldenGate reports processing errors in several ways by means of its
monitoring and reporting tools. For more information about these tools, see Monitoring
Oracle GoldenGate Processing (page 17-1).
Topics:
• WARNLONGTRANS
• DBOPTIONS
• TRANLOGOPTIONS
To handle extraction errors that relate to DDL operations, use the DDLERROR parameter.
14-1
Chapter 14
Handling Replicat Errors during DML Operations
• DISCARD: log the error to the discard file and continue processing.
• EXCEPTION: send the error for exceptions processing. See Handling Errors as
Exceptions (page 14-2) for more information.
• IGNORE: ignore the error and continue processing.
• The first, a standard MAP statement, maps the source table to the actual target
table.
• The second, an exceptions MAP statement, maps the source table to the
exceptions table (instead of to the target table). An exceptions MAP statement
14-2
Chapter 14
Handling Replicat Errors during DML Operations
executes immediately after an error on the source table to send the row values to
the exceptions table.
To identify a MAP statement as an exceptions MAP statement, use the
INSERTALLRECORDS and EXCEPTIONSONLY options. The exceptions MAP statement must
immediately follow the regular MAP statement that contains the same source table.
Use a COLMAP clause in the exceptions MAP statement if the source and exceptions-
table columns are not identical, or if you want to map additional information to
extra columns in the exceptions table, such as information that is captured by
means of column-conversion functions or SQLEXEC.
For more information about these parameters, see Reference for Oracle GoldenGate.
• A regular MAP statement that maps the source table ggs.equip_account to its target
table equip_account2.
• An exceptions MAP statement that maps the same source table to the exceptions
table ggs.equip_account_exception.
In this case, four extra columns were created, in addition to the same columns that the
table itself contains:
DML_DATE
OPTYPE
DBERRNUM
DBERRMSG
Note:
There can be no primary key or unique index restrictions on the exception
table. Uniqueness violations are possible in this scenario and would generate
errors.
This example shows how to use REPERROR with EXCEPTIONSONLY and an exceptions MAP
statement. This example only shows the parameters that relate to REPERROR; other
parameters not related to error handling are also required for Replicat.
REPERROR (DEFAULT, EXCEPTION)
MAP ggs.equip_account, TARGET ggs.equip_account2,
COLMAP (USEDEFAULTS);
MAP ggs.equip_account, TARGET ggs.equip_account_exception,
EXCEPTIONSONLY,
14-3
Chapter 14
Handling Replicat Errors during DML Operations
INSERTALLRECORDS
COLMAP (USEDEFAULTS,
DML_DATE = @DATENOW (),
OPTYPE = @GETENV ('LASTERR', 'OPTYPE'),
DBERRNUM = @GETENV ('LASTERR', 'DBERRNUM'),
DBERRMSG = @GETENV ('LASTERR', 'DBERRMSG'));
In this example, the REPERROR parameter is set for DEFAULT error handling, and the
EXCEPTION option causes the Replicat process to treat failed operations as exceptions
and continue processing.
14-4
Chapter 14
Handling Replicat errors during DDL Operations
Column Description
Specifies a TCP/IP error for which you are defining a response.
Error
Controls whether or not Oracle GoldenGate tries to connect again after the
Response defined error. Valid values are either RETRY or ABEND.
Controls how long Oracle GoldenGate waits before attempting to connect again.
Delay
Controls the number of times that Oracle GoldenGate attempts to connect again
Max Retries before aborting.
If a response is not explicitly defined in the TCPERRS file, Oracle GoldenGate responds
to TCP/IP errors by abending.
Example 14-3 TCPERRS File
# TCP/IP error handling parameters
# Default error response is abend
#
# Error Response Delay(csecs) Max Retries
14-5
Chapter 14
Maintaining Updated Error Messages
The TCPERRS file contains default responses to basic errors. To alter the instructions or
add instructions for new errors, open the file in a text editor and change any of the
values in the columns shown in Table 14-1 (page 14-5):
14-6
15
Instantiating Oracle GoldenGate with an
Initial Load
This chapter describes running an initial data load to instantiate the replication
environment.
The initial load can be done in Classic Architecture and in Microservices Architecture.
15-1
Chapter 15
Overview of the Initial-Load Procedure
Note:
A primary index is required for all applications that access DB2 for z/OS
target tables. You can delete all other indexes from the target tables,
except for the primary index.
15-2
Chapter 15
Initial Load in Classic Architecture
Note:
If the load is performed from a quiet source database and will not be followed
by continuous change synchronization, you can omit these groups.
Do not start the Extract or Replicat groups until instructed to do so in the initial-load
instructions. Change synchronization keeps track of transactional changes while the
load is being applied, and then the target tables are reconciled with those changes.
Note:
The first time that Extract starts in a new Oracle GoldenGate configuration,
any open transactions will be skipped. Only transactions that begin after
Extract starts are captured.
See Getting Started with the Oracle GoldenGate Process Interfaces (page 4-1) for
more information about using OBEY and using macros.
15-3
Chapter 15
Initial Load in Classic Architecture
Note:
The objects and data types being loaded in this method must be supported
by Oracle GoldenGate for your database and also by the database utility that
is being used. For items that are supported for your database, see the Oracle
GoldenGate installation and configuration documentation for that database.
For items that are supported by the database utility, see the database
vendor's documentation.
1. Make certain that you have addressed the requirements in Prerequisites for Initial
Load (page 15-1).
2. On the source and target systems, run GGSCI and start the Manager process.
START MANAGER
Note:
In a Windows cluster, start the Manager resource from the Cluster
Administrator.
Where:
group is the name of the Extract group.
4. (Oracle, if replicating sequences) Issue the DBLOGIN command as the user who has
EXECUTE privilege on update.Sequence.
15-4
Chapter 15
Initial Load in Classic Architecture
Caution:
Do not use the VIEW PARAMS or EDIT PARAMS command to view or edit an
existing parameter file that is in a character set other than that of the
local operating system (such as one where the CHARSET option was used
to specify a different character set). View the parameter file from outside
GGSCI if this is the case; otherwise, the contents may become
corrupted..
Where:
group is the name of the Replicat group.
10. On the target system, issue the following command to verify the status of change
replication.
INFO REPLICAT group
11. Continue to issue the INFO REPLICAT command until you have verified that change
replication has posted all of the change data that was generated during the initial
load. Reference the time of completion that you recorded. For example, if the copy
stopped at 12:05, make sure change replication has posted data up to that point.
12. On the target system, issue the following command to turn off the HANDLECOLLISIONS
parameter and disable the initial-load error handling.
SEND REPLICAT group, NOHANDLECOLLISIONS
13. On the target system, edit the Replicat parameter file to remove the
HANDLECOLLISIONS parameter. This prevents HANDLECOLLISIONS from being enabled
again the next time Replicat starts.
15-5
Chapter 15
Initial Load in Classic Architecture
Caution:
Do not use the VIEW PARAMS or EDIT PARAMS command to view or edit an
existing parameter file that is in a character set other than that of the
local operating system (such as one where the CHARSET option was used
to specify a different character set). View the parameter file from outside
GGSCI if this is the case; otherwise, the contents may become
corrupted.
15-6
Chapter 15
Initial Load in Classic Architecture
Note:
When the source has no DOMAIN, do not specify a DOMAIN for the
downstream database.
Replicat queries the instantiation SCN on any new mapping and filter records
accordingly
For more information, see the Reference for Oracle GoldenGate for Windows and
UNIX.
15-7
Chapter 15
Initial Load in Classic Architecture
Note:
In a Windows cluster, start the Manager resource from the Cluster
Administrator.
15-8
Chapter 15
Initial Load in Classic Architecture
Parameter Description
RMTHOSTOPTIONS hostname, Specifies the target system, the port where Manager
MGRPORT portnumber is running, and optional encryption of data across
[, ENCRYPT algorithm KEYNAME keyname] TCP/IP.
RMTFILE path, Specifies the extract file to which the load data will be
[MEGABYTES n] written. Oracle GoldenGate creates this file during the
load. Checkpoints are not maintained with RMTFILE.
• path is the relative or fully qualified name of the file. Note that the size of an extract file cannot exceed
• MEGABYTES designates the size of each file. 2GB.
5. Enter any appropriate optional Extract parameters listed in the Reference for
Oracle GoldenGate.
6. Save and close the parameter file.
7. On the target system, issue the following command to create an initial-load
Replicat parameter file.
EDIT PARAMS initial-load_Replicat
8. Enter the parameters listed in Table 15-2 (page 15-10) in the order shown,
starting a new line for each parameter statement. The following is a sample initial-
load Replicat parameter file for loading data from file to Replicat.
SPECIALRUN
END RUNTIME
TARGETDB mydb, USERIDALIAS ogg
EXTFILE /ggs/dirdat/initld
SOURCEDEFS /ggs/dirdef/source_defs
15-9
Chapter 15
Initial Load in Classic Architecture
Parameter Description
EXTFILE path Specifies the input extract file specified with the Extract
parameter RMTFILE.
• path is the relative or fully qualified name of the file.
15-10
Chapter 15
Initial Load in Classic Architecture
Parameter Description
9. Enter any appropriate optional Replicat parameters listed in the Reference for
Oracle GoldenGate.
10. Save and close the file.
11. View the Replicat parameter file to make certain that the HANDLECOLLISIONS
parameter is listed. If not, add the parameter to the file.
12. On the source system, start change extraction.
15-11
Chapter 15
Initial Load in Classic Architecture
Windows:
C:\> GGS directory\extract paramfile dirprm\initial-load_Extract.prm reportfile
path
Where:
initial-load_Extract is the name of the initial-load Extract that you used when
creating the parameter file, and path is the relative or fully qualified name of the
Extract report file.
16. Verify the progress and results of the initial extraction by viewing the Extract report
file using the operating system's standard method for viewing files.
17. Wait until the initial extraction is finished.
Windows:
C:\> GGS directory\replicat paramfile dirprm\initial-load_Replicat.prm
reportfile path
Where:
initial-load_Replicat is the name of the initial-load Replicat that you used when
creating the parameter file, and path is the relative or fully qualified name of the
Replicat report file.
19. When the initial-load Replicat is finished running, verify the results by viewing the
Replicat report file using the operating system's standard method for viewing files.
20. On the target system, start change replication.
15-12
Chapter 15
Initial Load in Classic Architecture
Caution:
Do not use the VIEW PARAMS or EDIT PARAMS command to view or edit an
existing parameter file that is in a character set other than that of the
local operating system (such as one where the CHARSET option was used
to specify a different character set). View the parameter file from outside
GGSCI if this is the case; otherwise, the contents may become
corrupted.
To control which port is used by Replicat, and to speed up the search and bind
process, use the DYNAMICPORTLIST parameter in the Manager parameter file. Manager
passes the list of port numbers that are specified with this parameter to the Replicat
task process. Replicat first searches for a port from this list, and only if no ports are
available from the list does Replicat begin scanning in ascending order from the
default Manager port number until it finds an available port.
This method supports standard character, numeric, and datetime data types, as well
as CLOB, NCLOB, BLOB, LONG, XML, and user-defined datatypes (UDT) embedded with the
following attributes: CHAR, NCHAR, VARCHAR, NVARCHAR, RAW, NUMBER, DATE, FLOAT, TIMESTAMP,
CLOB, BLOB, XML, and UDT. Character sets are converted between source and target
where applicable.
15-13
Chapter 15
Initial Load in Classic Architecture
This method supports Oracle internal tables, but does not convert between the source
and target character sets during the load.
Note:
In a Windows cluster, start the Manager resource from the Cluster
Administrator.
3. On the source, issue the following command to create the initial-load Extract.
ADD EXTRACT initial-load_Extract, SOURCEISTABLE
Where:
• initial-load_Extract is the name of the initial-load Extract, up to eight
characters.
• SOURCEISTABLE designates Extract as an initial-load process that reads
complete records directly from the source tables. Do not use any of the other
ADD EXTRACT service options or datasource arguments.
Table 15-3 Initial-load Extract Parameters for Oracle GoldenGate Direct Load
Parameter Description
15-14
Chapter 15
Initial Load in Classic Architecture
Table 15-3 (Cont.) Initial-load Extract Parameters for Oracle GoldenGate Direct Load
Parameter Description
RMTHOSTOPTIONS hostname, Specifies the target system, the port where Manager is
MGRPORT portnumber running, and optional encryption of data across TCP/IP.
[, ENCRYPT algorithm KEYNAME keyname]
6. Enter any appropriate optional Extract parameters listed in Reference for Oracle
GoldenGate.
7. Save and close the file.
8. On the target system, issue the following command to create the initial-load
Replicat task.
ADD REPLICAT initial-load_Replicat, SPECIALRUN
Where:
• initial-load_Replicat is the name of the initial-load Replicat task.
15-15
Chapter 15
Initial Load in Classic Architecture
Table 15-4 Initial-load Replicat parameters for Oracle GoldenGate Direct Load
Parameter Description
15-16
Chapter 15
Initial Load in Classic Architecture
11. Enter any appropriate optional Replicat parameters listed in Reference for Oracle
GoldenGate.
12. Save and close the parameter file.
Note:
Do not start the initial-load Replicat. The Manager process starts it
automatically and terminates it when the load is finished.
18. On the target system, issue the following command to find out if the load is
finished. Wait until the load is finished before going to the next step.
VIEW REPORT initial-load_Replicat
19. On the target system, start change replication.
15-17
Chapter 15
Initial Load in Classic Architecture
Caution:
Do not use the VIEW PARAMS or EDIT PARAMS command to view or edit an
existing parameter file that is in a character set other than that of the
local operating system (such as one where the CHARSET option was used
to specify a different character set). View the parameter file from outside
GGSCI if this is the case; otherwise, the contents may become
corrupted.
24. Save and close the parameter file. From this point forward, Oracle GoldenGate
continues to synchronize data changes.
15-18
Chapter 15
Initial Load in Classic Architecture
Where:
• initial-load_Extract is the name of the initial-load Extract, up to eight
characters.
• SOURCEISTABLE designates Extract as an initial-load process that reads
complete records directly from the source tables. Do not use any of the other
ADD EXTRACT service options or datasource arguments.
Table 15-5 Initial-load Extract Parameters for a Direct Bulk Load to SQL*Loader
Parameter Description
RMTHOSTOPTIONS hostname, Specifies the target system, the port where Manager is
MGRPORT portnumber running, and optional encryption of data across TCP/IP.
[, ENCRYPT algorithm KEYNAME keyname]
15-19
Chapter 15
Initial Load in Classic Architecture
Table 15-5 (Cont.) Initial-load Extract Parameters for a Direct Bulk Load to SQL*Loader
Parameter Description
Where:
• initial-load_Replicat is the name of the initial-load Replicat task.
Parameter Description
15-20
Chapter 15
Initial Load in Classic Architecture
Table 15-6 (Cont.) Initial-load Replicat Parameters for Direct Load to SQL*Loader
Parameter Description
{SOURCEDEFS full_pathname} | Specifies how to interpret data definitions. For more information
ASSUMETARGETDEFS about data definitions files, see Associating Replicated Data with
Metadata (page 12-1).
• Use SOURCEDEFS if the source and target
tables have different definitions. Specify the
source-definitions file generated by
DEFGEN.
• Use ASSUMETARGETDEFS if the source and
target tables have the same definitions.
SOURCECATALOG Specifies a default source Oracle container for subsequent MAP
statements. Enables the use of two-part names (schema.object)
where three-part names otherwise would be required. You can
use multiple instances of this parameter to specify different
default containers for different sets of MAP parameters.
12. Enter any appropriate optional Replicat parameters listed in Reference for Oracle
GoldenGate.
15-21
Chapter 15
Initial Load in Classic Architecture
Caution:
Do not start the initial-load Replicat. The Manager process starts it
automatically and terminates it when the load is finished.
19. On the target system, issue the following command to determine when the load is
finished. Wait until the load is finished before proceeding to the next step.
VIEW REPORT initial-load_Extract
20. On the target system, start change replication.
15-22
Chapter 15
Initial Load in Classic Architecture
Caution:
Do not use the VIEW PARAMS or EDIT PARAMS command to view or edit an
existing parameter file that is in a character set other than that of the
local operating system (such as one where the CHARSET option was used
to specify a different character set). View the parameter file from outside
GGSCI if this is the case; otherwise, the contents may become
corrupted..
Caution:
Do not use the VIEW PARAMS or EDIT PARAMS command to view or edit an
existing parameter file that is in a character set other than that of the
local operating system (such as one where the CHARSET option was used
to specify a different character set). View the parameter file from outside
GGSCI if this is the case; otherwise, the contents may become
corrupted..
15-23
Chapter 15
Initial Load in Classic Architecture
15-24
16
Customizing Oracle GoldenGate
Processing
This chapter describes how to customize Oracle GoldenGate processing.
Topics:
• Retrieve output parameters from a procedure for input to a FILTER or COLMAP clause.
Note:
SQLEXEC provides minimal globalization support. To use SQLEXEC in the capture
parameter file of the source capture, make sure that the client character set
in the source .prm file is either the same or a superset of the source
database character set.
16-1
Chapter 16
Executing Commands, Stored Procedures, and Queries with SQLEXEC
Syntax
This syntax executes a procedure within a TABLE or MAP statement.
SQLEXEC (SPNAME sp_name,
[ID logical_name,]
{PARAMS param_spec | NOPARAMS})
Argument Description
Required keyword that begins a clause to execute a stored
SPNAME
procedure.
Syntax
This syntax executes a query within a TABLE or MAP statement.
SQLEXEC (ID logical_name, QUERY ' query ',
{PARAMS param_spec | NOPARAMS})
Argument Description
Defines a logical name for the query. A logical name is required
ID logical_name
in order to extract values from the query results. ID
logical_name references the column values returned by the
query.
Specifies the SQL query syntax to execute against the database.
QUERY ' sql_query '
It can either return results with a SELECT statement or change the
database with an INSERT, UPDATE, or DELETE statement. The
query must be within single quotes and must be contained all on
one line. Specify case-sensitive object names the way they are
stored in the database, such as within quotes for Oracle case-
sensitive names.
SQLEXEC 'SELECT "col1" from "schema"."table"'
16-2
Chapter 16
Executing Commands, Stored Procedures, and Queries with SQLEXEC
If you want to execute a query on a table residing on a different database than the
current database, then the different database name has to be specified with the table.
The delimiter between the database name and the tablename should be a colon (:).
The following are some example use cases:
select col1 from db1:tab1
select col2 from db2:schema2.tab2
select col3 from tab3
select col3 from schema4.tab4
Execute a query
SQLEXEC 'sql_query'
Argument Description
Specifies the name of a stored procedure to execute. The statement
'call
must be enclosed within single quotes.
procedure_name ()'
Example:
SQLEXEC 'call prc_job_count ()'
SQLEXEC provides options to control processing behavior, memory usage, and error
handling. For more information, see Reference for Oracle GoldenGate.
16-3
Chapter 16
Executing Commands, Stored Procedures, and Queries with SQLEXEC
Syntax
PARAMS ([OPTIONAL | REQUIRED] param = {source_column | function}
[, ...] )
Where:
• OPTIONAL indicates that a parameter value is not required for the SQL to execute. If
a required source column is missing from the database operation, or if a column-
conversion function cannot complete successfully because a source column is
missing, the SQL executes anyway.
• REQUIRED indicates that a parameter value must be present. If the parameter value
is not present, the SQL will not be executed.
• param is one of the following:
– For a stored procedure, it is the name of any parameter in the procedure that
can accept input, such as a column in a lookup table.
– For an Oracle query, it is the name of any input parameter in the query
excluding the leading colon. For example, :param1 would be specified as
param1 in the PARAMS clause.
– For a non-Oracle query, it is pn, where n is the number of the parameter within
the statement, starting from 1. For example, in a query with two parameters,
the param entries are p1 and p2.
• {source_column | function} is the column or Oracle GoldenGate conversion function
that provides input to the procedure.
Syntax
{procedure_name | logical_name}.parameter
Where:
• procedure_name is the actual name of the stored procedure. Use this argument only
if executing a procedure one time during the life of the current Oracle GoldenGate
process.
16-4
Chapter 16
Executing Commands, Stored Procedures, and Queries with SQLEXEC
• logical_name is the logical name specified with the ID option of SQLEXEC. Use this
argument if executing a query or a stored procedure that will be executed multiple
times.
• parameter is either the name of the parameter or RETURN_VALUE, if extracting
returned values.
Note:
Additional SQLEXEC options are available for use when a procedure or query
includes parametes. See the full SQLEXEC documentation in Reference for
Oracle GoldenGate.
SQLEXEC executes the LOOKUP stored procedure. Within the SQLEXEC clause, the PARAMS
(code_param = account_code) statement identifies code_param as the procedure
parameter to accept input from the account_code column in the account table.
Replicat executes the LOOKUP stored procedure prior to executing the column map, so
that the COLMAP clause can extract and map the results to the newacct_val column.
16-5
Chapter 16
Executing Commands, Stored Procedures, and Queries with SQLEXEC
• The column map requires a column that is missing from the source database
operation. This can occur for an update operation if the database only logs the
values of columns that changed, rather than all of the column values. By default,
when a required column is missing, or when an Oracle GoldenGate column-
conversion function results in a "column missing" condition, the stored procedure
does not execute. Subsequent attempts to extract an output parameter from the
stored procedure results in a "column missing condition" in the COLMAP or FILTER
clause.
• The database generates an error.
Action Description
Causes Oracle GoldenGate to ignore all errors associated with the stored
IGNORE
procedure or query and continue processing. Any resulting parameter
extraction results in a "column missing" condition. This is the default.
16-6
Chapter 16
Executing Commands, Stored Procedures, and Queries with SQLEXEC
Action Description
Ensures that all errors associated with the stored procedure or query are
REPORT
reported to the discard file. The report is useful for tracing the cause of the
error. It includes both an error description and the value of the parameters
passed to and from the procedure or query. Oracle GoldenGate continues
processing after reporting the error.
Handles errors according to rules set by a REPERROR parameter specified in the
RAISE
Replicat parameter file. Oracle GoldenGate continues processing other stored
procedures or queries associated with the current TABLE or MAP statement
before processing the error.
Performs in a similar way to RAISE except that when an error associated with a
FINAL
procedure or query is encountered, any remaining stored procedures and
queries are bypassed. Error processing is called immediately after the error.
Causes Oracle GoldenGate to abend immediately upon encountering an error
FATAL
associated with a procedure or query.
16-7
Chapter 16
Using Oracle GoldenGate Macros to Simplify and Automate Work
• All objects that are affected by a SQLEXEC stored procedure or query must exist with
the correct structures prior to the execution of the SQL. Consequently, DDL on
these objects that affects structure (such as CREATE or ALTER) must happen before
the SQLEXEC executes.
• All objects affected by a standalone SQLEXEC statement must exist before the
Oracle GoldenGate processes start. Because of this, DDL support must be
disabled for those objects; otherwise, DDL operations could change the structure
or delete the object before the SQLEXEC procedure or query executes on it.
Syntax
MACRO #macro_name
PARAMS (#p1, #p2 [, ...])
BEGIN
macro_body
END;
16-8
Chapter 16
Using Oracle GoldenGate Macros to Simplify and Automate Work
Argument Description
Required. Indicates the start of an Oracle GoldenGate macro
MACRO
definition.
The macro body. The body is a syntax statement that defines the
macro_body
function that is to be performed by the macro. A macro body can
include any of the following types of statements.
• Simple parameter statements, as in:
COL1 = COL2
• Complex parameter statements with parameter substitution
as in:
MAP #o.#t, TARGET #o.#t, KEYCOLS (#k), COLMAP
(USEDEFAULTS);
• Invocations of other macros, as in:
#colmap (COL1, #sourcecol)
Ends the macro definition. The semicolon is required to complete
END;
the definition.
16-9
Chapter 16
Using Oracle GoldenGate Macros to Simplify and Automate Work
MACRO #macro1
PARAMS ( #o, #t, #k )
BEGIN
MAP #o.#t, TARGET #o.#t, KEYCOLS (#k), COLMAP (USEDEFAULTS);
END;
The following is an example of a macro that does not define parameters. It executes a
frequently used set of parameters.
MACRO #option_defaults
BEGIN
GETINSERTS
GETUPDATES
GETDELETES
INSERTDELETES
END;
Syntax
[target =] macro_name (val[, ...])
Argument Description
target = Optional. Specifies the target to which the results of the macro
are assigned or mapped. For example, target can be used to
specify a target column in a COLMAP statement. In the following
call to the #make_date macro, the column DATECOL1 is the target
and will be mapped to the macro results.
DATECOL1 = #make_date (YR1, MO1, DAY1)
macro_name The name of the macro that is being called, for example:
#make_date.
16-10
Chapter 16
Using Oracle GoldenGate Macros to Simplify and Automate Work
Argument Description
( val[, ...]) The parameter input values. This component is required whether
or not the macro defines parameters. If the macro defines
parameters, specify a comma-separated list of input values, in
the order that corresponds to the parameter definitions in the
MACRO parameter, and enclose the list within parentheses. If the
macro does not define parameters, specify the open and close
parentheses with nothing between them (). For more information
about this syntax, see the following:
Calling a Macro that Contains Parameters (page 16-11).
Calling a Macro without Input Parameters (page 16-13).
( val | {val, The parameter input values. This component is required whether
val, ...} )[, ...] or not the macro defines parameters. If the macro defines
parameters, specify a comma-separated list of input values, in
the order that corresponds to the parameter definitions in the
MACRO parameter, and enclose the list within parentheses. To
pass multiple values to one parameter, separate them with
commas and enclose the list within curly brackets. If the macro
does not define parameters, specify the open and close
parentheses with nothing between them (). For more information
about this syntax, see the following:
Calling a Macro that Contains Parameters (page 16-11).
Calling a Macro without Input Parameters (page 16-13).
16-11
Chapter 16
Using Oracle GoldenGate Macros to Simplify and Automate Work
3. If a parameter name does not appear in the PARAMS statement, the macro
processor evaluates whether or not the item is, instead, a call to another macro.
(See Calling Other Macros from a Macro (page 16-13).) If the call succeeds, the
nested macro is executed. If it fails, the whole macro fails.
Example 16-3 Using Parameters to Populate a MAP Statement
The following macro definition specifies three parameter that must be resolved. The
parameters substitute for the names of the table owner (parameter #o), the table
(parameter #t), and the KEYCOLS columns (parameter #k) in a MAP statement.
MACRO #macro1 PARAMS ( #o, #t, #k ) BEGIN MAP #o.#t, TARGET #o.#t, KEYCOLS (#k),
COLMAP (USEDEFAULTS); END;
Assuming a table in the MAP statement requires only one KEYCOLS column, the following
syntax can be used to call #macro1. In this syntax, the #k parameter can be resolved
with only one value.
#macro1 (SCOTT, DEPT, DEPTNO1)
To call the macro for a table that requires two KEYCOLS columns, the curly brackets are
used as follows to enclose both of the required values for the column names:
#macro1 (SCOTT, DEPT, {DEPTNO1, DEPTNO2})
The DEPTNO1 and DEPTNO2 values are passed as one argument to resolve the #t
parameter. Tables with three or more KEYCOLS can also be handled in this manner,
using additional values inside the curly brackets.
Example 16-4 Using a Macro to Perform Conversion
In this example, a macro defines the parameters #year, #month, and #day to convert a
proprietary date format.
MACRO #make_date
PARAMS (#year, #month, #day)
BEGIN
@DATE ('YYYY-MM-DD', 'CC', @IF (#year < 50, 20, 19), 'YY', #year, 'MM', #month,
'DD', #day)
END;
16-12
Chapter 16
Using Oracle GoldenGate Macros to Simplify and Automate Work
'DD', DAY2)
);
The following macro is defined without input parameters. The body contains frequently
used parameters.
MACRO #option_defaults
BEGIN
GETINSERTS
GETUPDATES
GETDELETES
INSERTDELETES
END;
#option_defaults ()
MAP owner.srctab2, TARGET owner.targtab2;
GETINSERTS
GETUPDATES
GETDELETES
INSERTDELETES
MAP owner.srctab2, TARGET owner.targtab2;
The nested macro must define all, or a subset of, the same parameters that are
defined in the base macro. In other words, the input values when the base macro is
called must resolve to the parameters in both macros.
The following defines #assign_date:
MACRO #assign_date
PARAMS (#target_col, #year, #month, #day)
BEGIN
#target_col = #make_date (#year, #month, #day)
END;
16-13
Chapter 16
Using Oracle GoldenGate Macros to Simplify and Automate Work
The following defines #make_date. This macro creates a date format that includes a
four-digit year, after first determining whether the two-digit input date should be
prefixed with a century value of 19 or 20. Notice that the PARAMS statement of
#make_date contains a subset of the parameters in the #assign_date macro.
MACRO #make_date
PARAMS (#year, #month, #day)
BEGIN
@DATE ('YYYY-MM-DD', 'CC', @IF (#year < 50, 20, 19), 'YY', #year, 'MM', #month,
'DD', #day)
END;
The macro expands to the following given the preceding input values and the
embedded #make_date macro:
COL1 = @DATE ('YYYY-MM-DD', 'CC', @IF (YEAR < 50, 20, 19),'YY', YEAR, 'MM', MONTH,
'DD', DAY)
Where:
filename is the name of the file. The .mac extension defines the file as a macro
library.
The following sample library named datelib contains two macros, #make_date and
#assign_date.
MACRO #assign_date
PARAMS (#target_col, #year, #month, #day)
BEGIN
#target_col = #make_date (#year, #month, #day)
END;
16-14
Chapter 16
Using Oracle GoldenGate Macros to Simplify and Automate Work
To use a macro library, use the INCLUDE parameter at the beginning of a parameter file,
as shown in the following sample Replicat parameter file.
INCLUDE /ggs/dirprm/datelib.mac
REPLICAT rep
ASSUMETARGETDEFS
USERIDALIAS ogg
MAP fin.acct_tab, TARGET fin.account;
When including a long macro library in a parameter file, you can use the NOLIST
parameter to suppress the listing of each macro in the Extract or Replicat report file.
Listing can be turned on and off by placing the LIST and NOLIST parameters anywhere
within the parameter file or within the macro library file. In the following example,
NOLIST suppresses the listing of each macro in the hugelib macro library. Specifying
LIST after the INCLUDE statement restores normal listing to the report file.
NOLIST
INCLUDE /ggs/dirprm/hugelib.mac
LIST
INCLUDE /ggs/dirprm/mdatelib.mac
REPLICAT REP
Syntax
CMDTRACE [ON | OFF | DETAIL]
Where:
• ON enables tracing.
In the following example, tracing is enabled before #testmac is called, then disabled
after the macro's execution.
REPLICAT REP
MACRO #testmac
BEGIN
COL1 = COL2,
COL3 = COL4,
END;
...
CMDTRACE ON
MAP test.table1, TARGET test.table2,
COLMAP (#testmac);
CMDTRACE OFF
16-15
Chapter 16
Using User Exits to Extend Oracle GoldenGate Capabilities
16-16
Chapter 16
Using User Exits to Extend Oracle GoldenGate Capabilities
Note:
User exits are case-sensitive for database object names. Names are
returned exactly as they are defined in the hosting database. Object names
must be fully qualified.
Where:
• function_code is the function to be executed by the callback routine.
16-17
Chapter 16
Using User Exits to Extend Oracle GoldenGate Capabilities
• On Windows systems, Extract and Replicat export the ERCALLBACK function that
is to be called from the user exit routine. The user exit must explicitly load the
callback function at run-time using the appropriate Windows API calls.
4. Include the CUSEREXIT parameter in your Extract or Replicat parameter file. This
parameter accepts the name of the shared object or DLL and the name of the
exported routine that is to be called from Extract or Replicat. You can specify the
full path of the shared object or DLL or let the operating system's standard search
strategy locate the shared object.
CUSEREXIT {DLL | shared_object} routine
[, INCLUDEUPDATEBEFORES]
[, PARAMS 'startup_string']
Where:
• DLL is a Windows DLL and shared_object is a UNIX shared object that contains
the user exit function.
• INCLUDEUPDATEBEFORES gets before images for UPDATE operations.
16-18
Chapter 16
Using User Exits to Extend Oracle GoldenGate Capabilities
For more information about these components, see Reference for Oracle GoldenGate
for Windows and UNIX.
16-19
Chapter 16
Using User Exits to Extend Oracle GoldenGate Capabilities
16-20
Chapter 16
Using User Exits to Extend Oracle GoldenGate Capabilities
• exitdemo.c shows how to initialize the user exit, issue callbacks at given exit
points, and modify data. It also demonstrates how to retrieve the fully qualified
table name or a specific metadata part, such as the name of the catalog or
container, or the schema, or just the unqualified table name. In addition, this demo
shows how to process DDL data. The demo is not specific to any database type.
• exitdemo_utf16.c shows how to use UTF16-encoded data (both metadata and
column data) in the callback structures for information exchanged between the
user exit and the caller process.
• exitdemo_more_recs.c shows an example of how to use the same input record
multiple times to generate several target records.
• exitdemo_lob.c shows an example of how to get read access to LOB data.
• exitdemo_pk_befores.c shows how to access the before and after image portions of
a primary key update record, as well as the before images of regular updates
(non-key updates). It also shows how to get target row values with SQLEXEC in the
Replicat parameter file as a means for conflict detection. The resulting fetched
values from the target are mapped as the target record when it enters the user
exit.
Each directory contains the *.c files as well as makefiles and a readme.txt file.
16-21
Chapter 16
Using the Oracle GoldenGate Event Marker System to Raise Database Events
The following causes the process to issue a checkpoint, log an informational message,
and ignore the entire transaction (without processing any of it), plus generate a report.
EVENTACTIONS (CP BEFORE, REPORT, LOG, IGNORE TRANSACTION)
The following writes the event record to the discard file and ignores the entire
transaction.
EVENTACTIONS (DISCARD, IGNORE TRANS)
The following logs an informational message and gracefully stop the process.
EVENTACTIONS (LOG INFO, STOP)
The following rolls over the trail file and does not write the event record to the new file.
16-22
Chapter 16
Using the Oracle GoldenGate Event Marker System to Raise Database Events
For syntax details and additional usage instructions, see Reference for Oracle
GoldenGate.
To allow users to continue working with the new source table, it is added to the Extract
parameter file, but not to the Replicat parameter file. Extract begins capturing data
from this table to the trail, where it is stored.
At the point where the source and target are read-consistent after the export, an event
record is inserted into the event table on the source, which propagates to the target.
When Replicat receives the event record (marking the read-consistent point), the
process stops as directed by EVENTACTIONS STOP. This allows the new table to be added
to the Replicat MAP statement. Replicat can be positioned to start replication from the
timestamp of the event record, eliminating the need to use the HANDLECOLLISIONS
parameter. Operations in the trail from before the event record can be ignored
because it is known that they were applied in the export.
The event record itself is ignored by Replicat, but an informational message is logged.
16-23
Chapter 16
Using the Oracle GoldenGate Event Marker System to Raise Database Events
used for the source table, so that the ABORT action stops Replicat before it applies the
anomaly to the target database. ABORT takes precedence over processing the record.
MAP source.account, TARGET target.account;
TABLE source.account, FILTER (withdrawal > balance), EVENTACTIONS (ABORT);
Note:
If a logical batch delete were to be composed of multiple smaller batches,
each smaller batch would require an insert into the job table as the first
record in the transaction.
Replicat:
16-24
Chapter 16
Using the Oracle GoldenGate Event Marker System to Raise Database Events
To use this configuration, a statement table is populated with the first operation in the
transaction, that being the INSERT INTO...SELECT, which becomes the event record.
Note:
For large SQL statements, the statement can be written to multiple columns
in the table. For example, eight VARCHAR (4000) columns could be used to
store SQL statements up to 32 KB in length.
Because of the IGNORE TRANS INCLUDEEVENT, Extract ignores all of the subsequent
inserts that are associated with the SELECT portion of the statement, but writes the
event record that contains the SQL text to the trail. Using a TABLE statement, Replicat
passes the event record to a SQLEXEC statement that concatenates the SQL text
columns, if necessary, and executes the INSERT INTO...SELECT statement using the
target tables as the input for the SELECT sub-query.
Extract:
TABLE src.*;
TABLE test.event;
Replicat:
MAP src.*, TARGET targ.*;
MAP test.event, TARGET test.event, FILTER (@streq (event_type, 'COMPARE')=1), &
EVENTACTIONS (SHELL 'compare_db.sh', FORCESTOP);
16-25
17
Monitoring Oracle GoldenGate Processing
This chapter describes the monitoring of Oracle GoldenGate processing.
Topics:
17-1
Chapter 17
Monitoring an Extract Recovery
17-2
Chapter 17
Using Automatic Heartbeat Tables to Monitor
Note:
The INFO command also returns a lag statistic, but this statistic is taken from
the last record that was checkpointed, not the current record that is being
processed. It is less accurate than LAG or INFO.
17-3
Chapter 17
Using Automatic Heartbeat Tables to Monitor
Note:
The Automatic Heartbeat functionality is not supported on MySQL version
5.5.
Ensure that Self-Describing Trail Files functionality is enabled, see Using Self-
Describing Trail Files (page 12-12).
Add a heartbeat table to each of your databases with the ADD HEARTBEATTABLE
command. Add the heartbeat table to all source and target instances and then restart
existing Oracle GoldenGate processes (not necessary for processes running against
HP-OSS for MX) to enable heartbeat functionality. Depending on your specific
database system, you may or may not be required to create or enable a job to
populate heartbeat table data.
(Optional) For Oracle Databases, you must ensure that the Oracle DBMS_SCHEDULER is
operating correctly as the heartbeat update relies on it. You can query the
DBMS_SCHEDULER by issuing:
17-4
Chapter 17
Using Automatic Heartbeat Tables to Monitor
Run an Extract, which on receiving the logical change records (LCR) checks the value
in the OUTGOING_EXTRACT column.
• If the Extract name matches this value, the OUTGOING_EXTRACT_TS column is
updated and the record is entered in the trail.
• If the Extract name does not match then the LCR is discarded.
• If the OUTGOING_EXTRACT value is NULL, it is populated along with
OUTGOING_EXTRACT_TS and the record is entered in the trail.
The Pump or Distribution server on reading the record, checks the value in the
OUTGOING_ROUTING_PATH column. This column has a list of distribution paths.
If the value is NULL, the column is updated with the current group name (and path if
this is a Distribution server),"*", update the OUTGOING_ROUTING_TS column, and the
record is written into its target trail file.
If the value has a "*" in the list, then replace it with group name[:pathname],"*"',
update the OUTGOING_ROUTING_TS column, and the record is written into its target trail
file. When the value does not have a asterisk (*) in the list and the pump name is in
the list, then the record is sent to the path specified in the relevant group
name[:pathname],"*"' pair in the list. If the pump name is not in the list, the record is
discarded.
Run a Replicat, which on receiving the record checks the value in the
OUTGOING_REPLICAT column.
• If the Replicat name matches the value, the row in the heartbeat table is updated
and the record is inserted into the history table.
• If the Replicat name does not match, the record is discarded.
• If the value is NULL, the row in the heartbeat and heartbeat history tables are
updated with an implicit invocation of the Replicat column mapping.
Automatic Replicat Column Mapping:
REMOTE_DATABASE = LOCAL_DATABASE
INCOMING_EXTRACT = OUTGOING_EXTRACT
INCOMING_ROUTING_PATH = OUTGOING_ROUTING_PATH with "*" removed
INCOMING_REPLICAT = @GETENV ("GGENVIRONMENT", "GROUPNAME")
INCOMING_HEARTBEAT_TS = HEARTBEAT_TIMESTAMP
INCOMING_EXTRACT_TS = OUTGOING_EXTRACT_TS
INCOMING_ROUTING_TS = OUTGOING_ROUTING_TS
INCOMING_REPLICAT_TS = @DATE ('UYYYY-MM-DD
HH:MI:SS.FFFFFF','JTSLCT',@GETENV ('JULIANTIMESTAMP'))
LOCAL_DATABASE = REMOTE_DATABASE
OUTGOING_EXTRACT = INCOMING_EXTRACT
17-5
Chapter 17
Using Automatic Heartbeat Tables to Monitor
OUTGOING_ROUTING_PATH = INCOMING_ROUTING_PATH
OUTGOING_HEARTBEAT_TS = INCOMING_HEARTBEAT_TS
OUTGOING_REPLICAT = INCOMING_REPLICAT
OUTGOING_HEARTBEAT_TS = INCOMING_HEARTBEAT_TS
There is just one column for OUTGOING_ROUTING_TS. If a record passes through multiple
pump before being applied by a Replicat, each pump will overwrite the
OUTGOING_ROUTING_TS column so that the pumps lag that is calculated is not specific to
a single pump and refers to the lag across all the pumps specified in PUMP_PATH.
Additional Considerations:
Computing lags as the heartbeat flows through the system relies on the clocks of the
source and target systems to be set up correctly. It is possible that the lag can be
negative if the target system is ahead of the source system. The lag is shown as a
negative number so that you are aware of their clock discrepancy and can take actions
to fix it.
The timestamp that flows through the system is in UTC. There is no time zone
associated with the timestamp so when viewing the heartbeat tables, the lag can be
viewed quickly even if different components are in different time zones. You can write
any view you want on top of the underlying tables; UTC is recommended.
All the heartbeat entries are written to the trail in UTF-8.
The outgoing and incoming paths together uniquely determine a row. Meaning that if
you have two rows with same outgoing path and a different incoming path, then it is
considered two unique entries.
Heartbeat Table Details
The GG_HEARTBEAT table displays timestamp information of the end-to-end replication
time and the timing information at the different components primary and secondary
Extract and Replicat.
In a unidirectional environment, only the target database contains information about
the replication lag. That is the time when a record is generated at the source database
and becomes visible to clients at the target database.
Note:
The automatic heartbeat tables don’t populate the OUTGOING_% columns
with data, when both the source and remote databases have the same
name. To change the database name, use the utility DBNEWID. For details, see
the DBNEWID Utility.
17-6
Chapter 17
Using Automatic Heartbeat Tables to Monitor
17-7
Chapter 17
Using Automatic Heartbeat Tables to Monitor
17-8
Chapter 17
Using Automatic Heartbeat Tables to Monitor
The GG_LAG view displays information about the replication lag between the local and
remote databases.
In a unidirectional environment, only the destination database contains information
about the replication lag. The lag is measured in seconds.
17-9
Chapter 17
Using Automatic Heartbeat Tables to Monitor
The GG_LAG_HISTORY view displays the history information about the replication lag
history between the local and remote databases.
In a unidirectional environment, only the destination database contains information
about the replication lag.
The unit of the lag units is in seconds.
17-10
Chapter 17
Using Automatic Heartbeat Tables to Monitor
17-11
Chapter 17
Monitoring Processing Volume
Command Description
ADD HEARTBEATTABLE Creates the objects required for automatic heartbeat
functionality.
ALTER HEARTBEATTABLE Alters existing heartbeat objects.
DELETE HEARTBEATTABLE Deletes existing heartbeat objects.
DELETE HEARTBEATENTRY Deletes entries in the heartbeat table.
INFO HEARTBEATTABLE Displays heartbeat table information.
For more information, see the Reference for Oracle GoldenGate for Windows and
UNIX.
You can send interim statistics to the report file at any time with the SEND EXTRACT or
SEND REPLICAT command with the REPORT option.
17-12
Chapter 17
Using the Error Log
17-13
Chapter 17
Using the Discard File
Where:
– The value for process is either extract or replicat.
– The value for path.prm is the fully qualified name of the parameter file, for
example:
replicat paramfile /ogg/dirdat/repora.prm
By default, reports have a file extension of .rpt, for example EXTORA.rpt. The default
location is the dirrpt sub-directory of the Oracle GoldenGate directory. However,
these properties can be changed when the group is created. Once created, a report
file must remain in its original location for Oracle GoldenGate to operate properly after
processing has started.
To determine the name and location of a process report, use the INFO EXTRACT, INFO
REPLICAT, or INFO MANAGER command in GGSCI.
To send runtime statistics to the report on demand, use the SEND EXTRACT or SEND
REPLICAT command with the REPORT option to view current runtime statistics when
needed.
17.7.3 Preventing SQL Errors from Filling the Replicat Report File
Use the WARNRATE parameter to set a threshold for the number of SQL errors that can
be tolerated on any target table before being reported to the process report and to the
error log. The errors are reported as a warning. If your environment can tolerate a
large number of these errors, increasing WARNRATE helps to minimize the size of those
files. For more information, see Reference for Oracle GoldenGate.
17-14
Chapter 17
Maintaining the Discard and Report Files
• REPORTROLLOVER
No process ever has more than ten aged reports or discard files and one active report
or discard file. After the tenth aged file, the oldest is deleted when a new report is
created. It is recommended that you establish an archiving schedule for aged reports
and discard files in case they are needed to resolve a service request.
17-15
Chapter 17
Reconciling Time Differences
Table 17-7 Current Extract and Manager Reports Plus Aged Reports
-rw-rw-rw-
1 ggs ggs 3996 Oct 5 14:02 MGR0.rpt
17-16
18
Tuning the Performance of Oracle
GoldenGate
This chapter contains suggestions for improving the performance of Oracle
GoldenGate components.
Topics:
18-1
Chapter 18
Using Multiple Process Groups
18-2
Chapter 18
Using Multiple Process Groups
If your tables do not have any foreign- key dependencies or updates to primary keys,
you may be able to use multiple processes. Keep related DML together in the same
process stream to ensure data integrity.
Note:
When creating the groups, keep tables that have relational constraints to
each other in the same group.
18.1.1.3 Memory
The system must have sufficient swap space for each Oracle GoldenGate Extract and
Replicat process that will be running. To determine the required swap space:
1. Start up one Extract or Replicat.
2. Run GGSCI.
3. View the report file and find the line PROCESS VM AVAIL FROM OS (min).
4. Round up the value to the next full gigabyte if needed. For example, round up
1.76GB to 2 GB.
5. Multiply that value by the number of Extract and Replicat processes that will be
running. The result is the maximum amount of swap space that could be required
See the CACHEMGR parameter in Reference for Oracle GoldenGate for more information
about how memory is managed.
18-3
Chapter 18
Using Multiple Process Groups
can then isolate those tables into their own Extract groups, assuming that
transactional integrity can be maintained.
• In its classic mode, Replicat process can be a source of performance bottlenecks
because it is a single-threaded process that applies operations one at a time by
using regular SQL. Even with BATCHSQL enabled (see Reference for Oracle
GoldenGate) Replicat may take longer to process tables that have large or long-
running transactions, heavy volume, a very large number of columns that change,
and LOB data. You can then isolate those tables into their own Replicat groups,
assuming that transactional integrity can be maintained.
Note:
This configuration includes Extract data-pumps.
1. On the source, use the ADD EXTRACT command to create a primary Extract group.
2. On the source, use the ADD EXTTRAIL command to specify as many local trails as
the number of Replicat groups that you will be creating. All trails must be
associated with the primary Extract group.
3. On the source create a data-pump Extract group.
4. On the source, use the ADD RMTTRAIL command to specify as many remote trails as
the number of Replicat groups that you will be creating. All trails must be
associated with the data-pump Extract group.
5. On the source, use the EDIT PARAMS command to create Extract parameter files,
one for the primary Extract and one for the data pump, that contain the parameters
required for your database environment. When configuring Extract, do the
following:
• Divide the source tables among different TABLE parameters.
• Link each TABLE statement to a different trail. This is done by placing the TABLE
statements after the EXTTRAIL or RMTTRAIL parameter that specifies the trail you
want those statements to be associated with.
18-4
Chapter 18
Using Multiple Process Groups
Note:
This configuration includes data pumps.
1. On the source, use the ADD EXTRACT command to create the primary Extract
groups.
2. On the source, use the ADD EXTTRAIL command to specify a local trail for each of
the Extract groups that you created.
3. On the source create a data-pump Extract group to read each local trail that you
created.
4. On the source, use the ADD RMTTRAIL command to specify a remote trail for each of
the data-pumps that you created.
5. On the source, use the EDIT PARAMS command to create an Extract parameter file
for each primary Extract group and each data-pump Extract group.
18-5
Chapter 18
Splitting Large Tables Into Row Ranges Across Process Groups
18-6
Chapter 18
Configuring Oracle GoldenGate to Use the Network Efficiently
18-7
Chapter 18
Configuring Oracle GoldenGate to Use the Network Efficiently
2. Look for the Write Checkpoint statistic. This is the place where Extract is writing to
the trail.
Write Checkpoint #1
4. Refer to the information that you noted, and make the following validation:
• Is the primary Extract generating a series of checkpoints, or just the initial
checkpoint?
• If a data pump is in use, is it generating a series of checkpoints, or just one?
5. Issue INFO EXTRACT for the primary and data pump Extract processes again.
• Has the most recent write checkpoint increased? Look at the most recent
Sequence, RBA, and Timestamp values to see if their values were incremented
forward since the previous INFO EXTRACT command.
6. Issue the following command to view the status of the Replicat process.
SEND REPLICAT group, STATUS
• The status indicates whether Replicat is delaying (waiting for data to process),
processing data, or at the end of the trail (EOF).
There is a network bottleneck if the status of Replicat is either in delay mode or at the
end of the trail file and either of the following is true:
• You are only using a primary Extract and its write checkpoint is not increasing or is
increasing too slowly. Because this Extract process is responsible for sending data
across the network, it will eventually run out of memory to contain the backlog of
extracted data and abend.
• You are using a data pump, and its write checkpoint is not increasing, but the write
checkpoint of the primary Extract is increasing. In this case, the primary Extract
can write to its local trail, but the data pump cannot write to the remote trail. The
data pump will abend when it runs out of memory to contain the backlog of
extracted data. The primary Extract will run until it reaches the last file in the trail
sequence and will abend because it cannot make a checkpoint.
18-8
Chapter 18
Configuring Oracle GoldenGate to Use the Network Efficiently
Note:
Even when there is a network outage, Replicat will process in a normal
manner until it applies all of the remaining data from the trail to the target.
Eventually, it will report that it reached the end of the trail file.
18-9
Chapter 18
Eliminating Disk I/O Bottlenecks
The required unit for TCPBUFSIZE is bytes, so you would set it to a value of
1000000.
The maximum socket buffer size for non-Windows systems is usually limited by
default. Ask your system administrator to increase the default value on the source and
target systems so that Oracle GoldenGate can increase the buffer size configured with
TCPBUFSIZE.
Note:
CHECKPOINTSECS is not valid for an integrated Replicat on an Oracle
database system.
• Use the GROUPTRANSOPS parameter to control the number of SQL operations that are
contained in a Replicat transaction when operating in its normal mode. Increasing
the number of operations in a Replicat transaction improves the performance of
Oracle GoldenGate by reducing the number of transactions executed by Replicat,
and by reducing I/O activity to the checkpoint file and the checkpoint table, if used.
Replicat issues a checkpoint whenever it applies a transaction to the target, in
addition to its scheduled checkpoints.
18-10
Chapter 18
Managing Virtual Memory and Paging
Note:
GROUPTRANSOPS is not valid for an integrated Replicat on an Oracle
database system, unless the inbound server parameter parallelism is
set to 1.
• Use the EOFDELAY or EOFDELAYCSECS parameter to control how often Extract, a data
pump, or Replicat checks for new data after it has reached the end of the current
data in its data source. You can reduce the system I/O overhead of these reads by
increasing the value of this parameter.
Note:
Increasing the values of these parameters improves performance, but it also
increases the amount of data that must be reprocessed if the process fails.
This has an effect on overall latency between source and target. Some
testing will help you determine the optimal balance between recovery and
performance.
If the current resources are not sufficient, a message like the following may be
returned:
2013-11-11 14:16:22 WARNING OGG-01842 CACHESIZE PER DYNAMIC DETERMINATION (32G) LESS
THAN RECOMMENDED: 64G (64bit system)vm found: 63.97GCheck swap space. Recommended
swap/extract: 128G (64bit system).
If the system exhibits excessive paging and the performance of critical processes is
affected, you can reduce the CACHESIZE option of the CACHEMGR. parameter. You can also
control the maximum amount of disk space that can be allocated to the swap directory
with the CACHEDIRECTORY option. For more information about CACHEMGR, see Reference for
Oracle GoldenGate.
18-11
Chapter 18
Optimizing Data Filtering and Conversion
18-12
Chapter 18
Tuning Replicat Transactions
The gathering of SQL statements into batches improves efficiency but also consumes
memory. To maintain optimum performance, use the following BATCHSQL options:
BATCHESPERQUEUE
BYTESPERQUEUE
OPSPERBATCH
OPSPERQUEUE
As a benchmark for setting values, assume that a batch of 1,000 SQL statements at
500 bytes each would require less than 10 megabytes of memory.
You can use BATCHSQL with the BATCHTRANSOPS option to tune array sizing. BATCHTRANSOPS
controls the maximum number of batch operations that can be grouped into a
transaction before requiring a commit. The default for non-integrated Replicat is 1000.
The default for integrated Replicat is 50. If there are many wait dependencies when
using integrated Replicat, try reducing the value of BATCHTRANSOPS. To determine the
number of wait dependencies, view the TOTAL_WAIT_DEPS column of the
V$GG_APPLY_COORDINATOR database view in the Oracle database.
See Reference for Oracle GoldenGate for additional usage considerations and syntax.
To make row selection more efficient, use a KEYCOLS clause in the TABLE and MAP
statements to identify one or more columns as unique. Replicat will use the specified
columns as a key. The following example shows a KEYCOLS clause in a TABLE statement:
TABLE hr.emp, KEYCOLS (FIRST_NAME, LAST_NAME, DOB, ID_NO);
For usage guidelines and syntax, see the TABLE and MAP parameters in Reference for
Oracle GoldenGate.
18-13
Chapter 18
Tuning Replicat Transactions
Note:
MAXTRANSOPS is not valid for an integrated Replicat on an Oracle database
system.
• Network problems prevent trail data from being delivered to the target system.
• Running out of disk space on any system, preventing trail data from being written.
• Collector abends (a rare event).
• Extract abends or is terminated in the middle of writing records for a transaction.
• An Extract data pump abends or is terminated.
• There is a source system failure, such as a power outage or system crash.
18-14
Chapter 18
Tuning Replicat Transactions
18-15
19
Performing Administrative Operations
This chapter contains instructions for making changes to applications, systems, and
Oracle GoldenGate while the replication environment is active and processing data
changes.
Topics:
19-1
Chapter 19
Initializing the Transaction Logs
5. Wait until the data pump (if used) and Replicat are finished processing the data in
their respective trails. To determine when they are finished, use the following
commands until they return At EOF.
SEND EXTRACT group GETLAG
SEND REPLICAT group GETLAG
6. Stop the data pump and Replicat.
STOP EXTRACT group
STOP REPLICAT group
At this point, the data in the source and target should be identical, because all of
the replicated transactional changes from the source have been applied to the
target.
7. Apply the patch on the target.
8. If the patches changed table definitions, run DEFGEN for the source tables to
generate updated source definitions, and then replace the old definitions with the
new ones in the existing source definitions file on the target system.
9. Start the Oracle GoldenGate processes whenever you are ready to begin
capturing user activity again.
19-2
Chapter 19
Shutting Down the System
19-3
Chapter 19
Changing Database Attributes
This procedure stops Extract, and then creates a new trail file. The new database
metadata is included in this new file with the transactions that started after the change.
1. Stop transaction activity on the source database. Do not make the metadata
change to the database yet.
2. In GGSCI on the source system, issue the SEND EXTRACT command with the LOGEND
option until it shows there is no more redo data to capture.
SEND EXTRACT group LOGEND
3. Stop Extract.
STOP EXTRACT group
4. On each target system, issue the SEND REPLICAT command with the STATUS option
until it shows a status of "At EOF" to indicate that it finished processing all of the
data in the trail. This must be done on all target systems until all Replicat
processes return "At EOF."
SEND REPLICAT group STATUS
5. Stop the data pumps and Replicat.
STOP EXTRACT group
STOP REPLICAT group
19-4
Chapter 19
Changing Database Attributes
Note:
For Oracle and Teradata databases, you can enable the DDL support feature
of Oracle GoldenGate to automatically capture and apply the DDL that adds
new tables, instead of using this procedure. See the appropriate instructions
for your database in this documentation.
Review these steps before starting. The process varies slightly, depending on whether
or not the new tables satisfy wildcards in the TABLE parameter, and whether or not
names or data definitions must be mapped on the target.
19-5
Chapter 19
Changing Database Attributes
2. (If new tables do not satisfy a wildcard) If you are adding numerous tables that do
not satisfy a wildcard, make a copy of the Extract and Replicat parameter files,
and then add the new tables with TABLE and MAP statements. If you do not want to
work with a copy, then edit the original parameter files after you are prompted to
stop each process.
3. (If new tables satisfy wildcards) In the Extract and Replicat parameter files, make
certain the WILDCARDRESOLVE parameter is not being used, unless it is set to the
default of DYNAMIC.
4. (If new tables do not satisfy a wildcard) If the new tables do not satisfy a wildcard
definition, stop Extract.
STOP EXTRACT group
5. Add the new tables to the source and target databases.
6. If required for the source database, issue the ADD TRANDATA command in GGSCI for
the new tables. Before using ADD TRANDATA, issue the DBLOGIN command.
7. Depending on whether the source and target definitins are identical or different,
use either ASSUMETARGETDEFS or SOURCEDEFS in the Replicat parameter file. If
SOURCEDEFS is needed, you can do either of the following:
• Run DEFGEN, then copy the new definitions to the source definitions file on
the target.
• If the new tables match a definitions template, specify the template with the
DEF option of the MAP parameter. (DEFGEN not needed.)
8. To register the new source definitions or new MAP statements, stop and then start
Replicat.
STOP REPLICAT group
START REPLICAT group
9. Start Extract, if applicable.
START EXTRACT group
10. Permit user access to the new tables.
Note:
See also Performing an ALTER TABLE to Add a Column on DB2 z/OS
Tables (page 19-8).
19-6
Chapter 19
Changing Database Attributes
Note:
This procedure assumes that the Oracle GoldenGate DDL support feature is
not in use, or is not supported for your database. For Oracle and Teradata
databases, you can enable the DDL support feature of Oracle GoldenGate to
propagate the DDL changes to the target, instead of using this procedure.
1. On the source and target systems, create a table, to be known as the marker
table, that can be used for the purpose of generating a marker that denotes a
stopping point in the transaction log. Just create two simple columns: one as a
primary key and the other as a regular column. For example:
CREATE TABLE marker
(
id int NOT NULL,
column varchar(25) NOT NULL,
PRIMARY KEY (id)
);
2. Insert a row into the marker table on both the source and target systems.
INSERT INTO marker VALUES (1, 1);
COMMIT;
3. On the source system, run GGSCI.
4. Open the Extract parameter file for editing.
Caution:
Do not use the VIEW PARAMS or EDIT PARAMS command to view or edit an
existing parameter file that is in a character set other than that of the
local operating system (such as one where the CHARSET option was used
to specify a different character set). View the parameter file from outside
GGSCI if this is the case; otherwise, the contents may become
corrupted..
5. Add the marker table to the Extract parameter file in a TABLE statement.
TABLE marker;
6. Save and close the parameter file.
7. Add the marker table to the TABLE statement of the data pump, if one is being used.
8. Stop the Extract and data pump processes, and then restart them immediately to
prevent capture lag.
STOP EXTRACT group
START EXTRACT group
STOP EXTRACT pump_group
START EXTRACT pump_group
9. On the target system, run GGSCI.
10. Open the Replicat parameter file for editing.
19-7
Chapter 19
Changing Database Attributes
Caution:
Do not use the VIEW PARAMS or EDIT PARAMS command to view or edit an
existing parameter file that is in a character set other than that of the
local operating system (such as one where the CHARSET option was used
to specify a different character set). View the parameter file from outside
GGSCI if this is the case; otherwise, the contents may become
corrupted.
11. Add the marker table to the Replicat parameter file in a MAP statement, and use the
EVENTACTIONS parameter as shown to stop Replicat and ignore operations on the
marker table.
MAP marker, TARGET marker, EVENTACTIONS (STOP, IGNORE);
12. Save and close the parameter file.
19-8
Chapter 19
Changing Database Attributes
4. Restart Extract.
5. Allow table activity to resume.
19-9
Chapter 19
Changing Database Attributes
19-10
Chapter 19
Adding Process Groups to an Active Configuration
been committed, so Extract must keep track of all open transactions. To do so, Extract
requires access to the archive log where each open transaction started and all archive
logs thereafter.
Extract reads the current archive log (the read checkpoint) for new transactions and
also has a checkpoint (the recovery checkpoint) in the oldest archive log for which
there is an uncommitted transaction.
Use the following command in GGSCI to determine Extract's checkpoint positions.
INFO EXTRACT group, SHOWCH
• The Input Checkpoint field shows where Extract began processing when it was
started.
• The Recovery Checkpoint field shows the location of the oldest uncommitted
transaction.
• The Next Checkpoint field shows the position in the redo log that Extract is reading.
• The Output Checkpoint field shows the position where Extract is writing.
You can write a shell script that purges all archive logs no longer needed by Extract by
capturing the sequence number listed under the Recovery Checkpoint field. All archive
logs prior to that one can be safely deleted.
19-11
Chapter 19
Adding Process Groups to an Active Configuration
Note:
You can copy the original parameter file to use for this group, but make
certain to change the Extract group name and any other relevant
parameters that apply to this new group.
• TABLE statement(s) (and TABLEEXCLUDE, if appropriate) for the tables that are to
be processed by the new group.
7. Save and close the file.
8. Edit the original Extract parameter file(s) to remove the TABLE statements for the
tables that are being moved to the new group or, if using wildcards, add the
TABLEEXCLUDE parameter to exclude them from the wildcard specification.
9. (Oracle) If you are using Extract in integrated mode, register the new Extract group
with the source database.
REGISTER EXTRACT group DATABASE [CONTAINER (container[, ...])]
10. Lock the tables that were moved to the new group, and record the timestamp for
the point when the locks were applied. For Oracle tables, you can run the following
script, which also releases the lock after it is finished.
-- temp_lock.sql
-- use this script to temporary lock a table in order to
19-12
Chapter 19
Adding Process Groups to an Active Configuration
-- get a timestamp
12. Stop the old Extract group(s) and any existing data pumps.
Where:
• EXTTRAIL creates a local trail. Use this option if you will be creating a data
pump for use with the new Extract group. Specify the trail that is specified with
EXTTRAIL in the parameter file. After creating the trail, go To Link a Local Data
Pump to the New Extract Group (page 19-13).
• RMTTRAIL creates a remote trail. Use this option if a data pump will not be used.
Specify the trail that is specified with RMTTRAIL in the parameter file. After
creating the trail, go To Link a Remote Replicat to the New Data Pump
(page 19-14)
You can specify a relative or full path name. Examples:
ADD RMTTRAIL dirdat/rt, EXTRACT primary
ADD EXTTRAIL c:\ogg\dirdat\lt, EXTRACT primary
For example:
ADD EXTRACT pump2, EXTTRAILSOURCE dirdat\lt
2. Create a parameter file for the data pump.
EDIT PARAMS pump
3. In the parameter file, include the appropriate Extract parameters for your
configuration, plus:
• RMTHOST parameter to point to the target system.
• TABLE parameter(s) for the tables that are to be processed by this data pump.
4. In GGSCI on the source system, add a remote trail for the data-pump. Use the trail
name that you specified with RMTTRAIL in the parameter file.
ADD RMTTRAIL trail, EXTRACT pump
19-13
Chapter 19
Adding Process Groups to an Active Configuration
For example:
ADD RMTTRAIL dirdat/rt, EXTRACT pump2
5. Follow the steps in To Link a Remote Replicat to the New Data Pump
(page 19-14).
For example:
ADD REPLICAT rep2, EXTTRAIL /home/ggs/dirdat/rt
2. Create a parameter file for this Replicat group. Use MAP statement(s) to specify the
same tables that you specified for the new primary Extract and the data pump (if
used).
3. On the source system, start the Extract groups and data pumps.
START EXTRACT group
START EXTRACT pump
4. On the target system, start the new Replicat group.
START REPLICAT group
For example:
ADD EXTTRAIL dirdat\lt, EXTRACT primary
3. Open the parameter file of the primary Extract group, and replace the RMTTRAIL
parameter with an EXTTRAIL parameter that points to the local trail that you created.
19-14
Chapter 19
Adding Process Groups to an Active Configuration
Caution:
Do not use the VIEW PARAMS or EDIT PARAMS command to view or edit an
existing parameter file that is in a character set other than that of the
local operating system (such as one where the CHARSET option was used
to specify a different character set). View the parameter file from outside
GGSCI if this is the case; otherwise, the contents may become
corrupted..
For example:
ADD EXTRACT pump, EXTTRAILSOURCE dirdat\lt
7. Create a parameter file for the new data pump.
EDIT PARAMS group
8. In the parameter file, include the appropriate Extract parameters for your
configuration, plus:
• TABLE parameter(s) for the tables that are to be processed by this data pump.
9. In GGSCI on the source system, add a remote trail for the data-pump. Use the trail
name that is specified with RMTTRAIL in the data pump's parameter file, and specify
the group name of the data pump for EXTRACT.
ADD RMTTRAIL trail, EXTRACT group
For example:
ADD RMTTRAIL dirdat/rt, EXTRACT pump
Note:
This command binds a trail name to an Extract group but does not
actually create the trail. A trail file is created when processing starts.
11. Add a new Replicat group and link it with the remote trail.
19-15
Chapter 19
Adding Process Groups to an Active Configuration
For example:
ADD REPLICAT rep, EXTTRAIL dirdat/rt
12. Create a parameter file for this Replicat group. You can copy the parameter file
from the original Replicat group, but make certain to change the REPLICAT
parameter to the new group name.
13. On the source system, stop the primary Extract group, then start it again so that
the parameter changes you made take effect.
STOP EXTRACT group
START EXTRACT group
14. On the source system, start the data pump.
Note:
Do not delete the old remote trail, just in case it is needed later on for a
support case or some other reason. You can move it to another location,
if desired.
19-16
Chapter 19
Adding Process Groups to an Active Configuration
Note:
You can copy the original parameter file to use for this group, but make
certain to change the Replicat group name and any other relevant
parameters that apply to this new group.
4. Add MAP statements (or edit copied ones) to specify the tables that you are adding
or moving to this group. If this group will be a coordinated Replicat group, include
the appropriate thread specifications.
5. Save and close the parameter file.
6. On the source system, run GGSCI.
7. Stop the Extract group.
STOP EXTRACT group
8. Issue the INFO REPLICAT command for the old Replicat group, and continue issuing
it until it reports At EOF, no more records to process.
INFO REPLICAT group
9. On the target system, edit the old Replicat parameter file to remove MAP statements
that specified the tables that you moved to the new Replicat group. Keep only the
MAP statements that this Replicat will continue to process.
11. Issue the INFO REPLICAT command for the old Replicat group, and continue issuing
it until it reports At EOF, no more records to process.
INFO REPLICAT group
12. Obtain the current Replicat checkpoint.
13. Stop the old Replicat group. If you are stopping a coordinated Replicat, make
certain the stop is clean so that all threads stop at the same trail record.
STOP REPLICAT group
14. Alter the new Replicat to position at the same trail sequence number and RBA as
the old replicat group
ALTER REPLICAT group, EXTSEQNO seqno, EXTRBA rba
The seqno is the trail sequence number from the old group checkpoint obtained in
step 11 (page 19-17) and the rba is the trail record RBA number from the old
group checkpoint.
15. Add the new Replicat group. For EXTTRAIL, specify the trail that this Replicat group
is to read.
ADD REPLICAT group, EXTTRAIL trail
For example:
19-17
Chapter 19
Changing the Size of Trail Files
(Local trail)
INFO EXTTRAIL *
2. Issue one of the following commands, depending on the location of the trail, to
change the file size.
(Remote trail)
ALTER RMTTRAIL trail, EXTRACT group, MEGABYTES n
(Local trail)
ALTER EXTTRAIL trail, EXTRACT group, MEGABYTES n
3. Issue the following command to cause Extract to switch to the next file in the trail.
SEND EXTRACT group, ROLLOVER
19-18
Chapter 19
Switching Extract from Classic Mode to Integrated Mode
To support the transition to integrated mode, the transaction log that contains the start
of the oldest open transaction must be available on the source or downstream mining
system, depending on where Extract will be running.
To determine the oldest open transaction, issue the SEND EXTRACT command with the
SHOWTRANS option. You can use the FORCETRANS or SKIPTRANS options of this command to
manage specific open transactions, with the understanding that skipping a transaction
may cause data loss and forcing a transaction to commit to the trail may add unwanted
data if the transaction is rolled back by the user applications. Review these options in
SEND EXTRACT Reference for Oracle GoldenGatebefore using them.
Where: alias specifies the alias of a user in the credential store who has the
privileges granted through the Oracle dbms_goldengate_auth.grant_admin_privilege
procedure.
5. Register the Extract group with the mining database. Among other things, this
creates the logmining server.
REGISTER EXTRACT group DATABASE
6. Issue the following command to determine whether the upgrade command can be
issued. Transactions that started before the registration command must be written
to the trail before you can proceed with the upgrade. You may have to issue this
command more than once until it returns a message stating that Extract can be
upgraded.
INFO EXTRACT group UPGRADE
7. Stop the Extract group.
STOP EXTRACT group
8. Switch the Extract group to integrated mode. See Oracle RAC options for this
command in STOP EXTRACTin Reference for Oracle GoldenGate, if applicable.
ALTER EXTRACT group UPGRADE INTEGRATED TRANLOG
9. Replace the old parameter file with the new one, keeping the same name.
19-19
Chapter 19
Switching Extract from Integrated Mode to Classic Mode
19-20
Chapter 19
Switching Replicat from Nonintegrated Mode to Integrated Mode
Where: alias is the alias of a user in the credential store who has the privileges
granted through the Oracle dbms_goldengate_auth.grant_admin_privilege
procedure.
7. Switch the Extract group to classic mode.
ALTER EXTRACT group DOWNGRADE INTEGRATED TRANLOG
If on a RAC system, then the THREADS option has to be used with the downgrade
command to specify the number of RAC threads.
8. Unregister the Extract group from the mining database. Among other things, this
removes the logmining server.
UNREGISTER EXTRACT group DATABASE
9. Replace the old parameter file with the new one, keeping the same name.
10. Start the Extract group.
Note:
Do not configure the switch between Replicat modes to occur immediately
after Extract recovers from a failure or is repositioned to a different location in
the transaction log.
19-21
Chapter 19
Switching Replicat from Integrated Mode to Nonintegrated Mode
Where: alias is the alias of a user in the credential store who has the privileges
granted through the Oracle dbms_goldengate_auth.grant_admin_privilege
procedure.
7. Alter Replicat to integrated mode.
ALTER REPLICAT group, INTEGRATED
8. Replace the old parameter file with the new one, keeping the same name.
9. Start Replicat.
START REPLICAT group
10. Verify that Replicat is in integrated mode.
Note:
Do not configure the switch between Replicat modes to occur immediately
after Extract recovers from a failure or is repositioned to a different location in
the transaction log.
Bug 17079228
19-22
Chapter 19
Switching Replicat to Coordinated Mode
Where: alias is the alias of a user in the credential store who has the privileges
granted through the Oracle dbms_goldengate_auth.grant_admin_privilege
procedure.
6. Create a checkpoint table in the target database for the nonintegrated Replicat to
use to store its recovery checkpoints. If a checkpoint table was previously
associated with this Replicat group and still exists, you can omit this step. See
Creating a Checkpoint Table (page 13-3) for more information about options for
using a checkpoint table.
ADD CHECKPOINTTABLE [container.]table
7. Stop Replicat.
STOP REPLICAT group
8. Alter Replicat to nonintegrated mode. For the CHECKPOINTTABLE argument, specify
the checkpoint table that you created for this Replicat group.
ALTER REPLICAT group, NONINTEGRATED, CHECKPOINTTABLE [container.]table
9. Replace the old parameter file with the new one, keeping the same name.
10. Start Replicat.
After issuing this command, wait until there is some activity on the source
database so that the switchover can be completed. (Replicat waits until its internal
high-water mark is exceeded before removing the status of "switching from
integrated mode.")
11. Verify that Replicat switched to nonintegrated mode.
19-23
Chapter 19
Switching Replicat to Coordinated Mode
Note:
Do not create the Replicat group until prompted by these instructions.
1. Back up the current parameter files of all of the Extract groups, data pumps, and
Replicat groups. You will be editing them.
2. Create a working directory outside the Oracle GoldenGate directory. You will use
this directory to create and stage new versions of the parameter files. If needed,
you can create a working directory on the source and target systems.
3. In the working directory, create a parameter file for a coordinated Replicat. Copy
the MAP parameters from the active parameter files of all of the Replicat groups to
this parameter file, and then add the thread specifications and any other
parameters that support your required coordinated Replicat configuration
4. If using multiple primary Extract groups, select one to keep, and then save a copy
of its current parameter file to the working directory.
5. Copy all of the TABLE statements from the other Extract groups to the new
parameter file of the primary Extract that you are keeping.
6. In the working directory, save a copy of the parameter file of the data pump that is
linked to the primary Extract that you are keeping.
7. Copy all of the TABLE statements from the other data pumps to the copied
parameter file of the kept data pump.
8. In the source database, create a simple dummy table on which a simple INSERT
statement can be performed. For this procedure, the name schema.event is used.
19-24
Chapter 19
Switching Replicat to Coordinated Mode
9. Create the same table on the target system, to avoid the need for additional
configuration parameters.
10. Edit the active parameter files (not the copies) of all primary and data-pump
Extract groups to add the following EVENTACTIONS parameter to each one.
TABLE schema.event, EVENTACTIONS(STOP);
11. Edit the active parameter files (not the copies) of all of the Replicat groups to add
the following EVENTACTIONS parameter to each one.
MAP schema.event, TARGET schema.event, EVENTACTIONS(IGNORE, STOP);
12. Stop the Oracle GoldenGate processes gracefully in the following order:
18. Log into the target database by using the DBLOGIN command.
19. Delete all of the Replicat groups and their active parameter files.
20. Copy or move the new coordinated Replicat parameter file from the working
directory to the Oracle GoldenGate directory.
21. In GGSCI, issue the INFO EXTRACT command for the data pump and make note of
its write checkpoint position in the output (remote) trail.
INFO EXTRACT pump, DETAIL
22. Add a new coordinated Replicat group with the following parameters.
Where:
• group is the name of the coordinated Replicat group. The name must match
that of the new parameter file created for this group.
19-25
Chapter 19
Administering a Coordinated Replicat Configuration
• EXTTRAIL trail is the name of the trail that the data pump writes to.
This section provides instructions for cleanly shutting down Replicat before performing
a re-partitioning, as well as instructions for attempting to recover Replicat continuity
when a re-partitioning is performed after an unclean shutdown.
The following tasks can be performed for a Replicat group in coordinated mode.
Performing a Planned Re-partitioning of the Workload (page 19-27)
Recovering Replicat After an Unplanned Re-partitioning (page 19-27)
Synchronizing Threads After an Unclean Stop (page 19-29)
19-26
Chapter 19
Administering a Coordinated Replicat Configuration
19-27
Chapter 19
Administering a Coordinated Replicat Configuration
watermark value. This output also shows you the active thread IDs under the Group
Name column. Make a record of these, as well.
info ra002
REPLICAT RA002 Last Started 2013-05-01 14:15 Status
ABENDEDCOORDINATED Replicat Thread Thread 2Checkpoint
Lag 00:00:00 (updated 00:00:06 ago)Process ID 11455
Log Read Checkpoint File ./dirdat/withMaxTransOp/
bg000000001 2013-05-02 07:49:15.837271 RBA 45603
4. Issue the ALTER REPLICAT command for the coordinator thread (Replicat as a whole,
without any thread ID) and position to the low watermark RBA that you recorded.
ALTER REPLICAT group EXTRBA low_watermark_rba
5. Start Replicat.
START REPLICAT group
6. Issue the basic INFO REPLICAT command until it shows an RBA that is higher than
the high watermark that you recorded. HANDLECOLLISIONS handles any collisions that
occur due to previously applied transactions.
INFO REPLICAT group
7. Stop Replicat.
STOP REPLICAT group
8. Remove or comment out the HANDLECOLLISIONS parameter.
9. Start Replicat.
START REPLICAT group
19-28
Chapter 19
Administering a Coordinated Replicat Configuration
1. Save the new parameter file to a different name, and then rename the saved
original parameter file to the correct name (same as the group name). The saved
parameter file has a .backup suffix and is stored in the dirprm subdirectory of the
Oracle GoldenGate installation directory.
2. Issue the following command to synchronize the Replicat threads to the maximum
checkpoint position. This command automatically starts Replicat and executes the
threads until they reach the maximum checkpoint position.
SYNCHRONIZE REPLICAT group
3. Issue the STATUS REPLICAT command until it shows that Replicat stopped cleanly.
STATUS REPLICAT group
4. Save the original parameter file to a different name, and then rename the new
parameter file to the group name.
5. Start Replicat.
START REPLICAT group
19-29
Chapter 19
Restarting a Primary Extract after System Failure or Corruption
Note:
In an Oracle RAC environment, the lowest SCN of all of the threads is
transmitted to Replicat. Transactions that may already have been committed
by Replicat are handled as duplicates at startup. However, any thread that
has been idle past a certain threshold will not be considered for the BSN
value, to avoid Extract having to read too far back in the log stream when
restarted.
The bounded recovery checkpoint is not taken into account when calculating the
LOGBSN. The failure that affected the Extract checkpoint file may also involve a loss of
the persisted bounded recovery data files and bounded recovery checkpoint
information.
19-30
Chapter 19
Restarting a Primary Extract after System Failure or Corruption
3. (Classic capture mode only. Skip if using integrated capture mode.) Query the
source database to find the sequence number of the transaction log file that
contains the value of the LOGBSN that you identified in the previous step. This
example assumes 1855798 is the LOGBSN value and shows that the sequence
number of the transaction log that contains that LOGBSN value is 163.
SQL> select name, thread#, sequence# from v$archived_log
where 1855798 between first_change# and next_change#;
Note:
There is a limit on how far back Extract can go in the transaction stream,
when in integrated mode. If the required SCN is no longer available, the
ALTER EXTRACT command fails.
5. Issue the following command in GGSCI to the primary Extract to view the new
sequence number of the Extract Write Checkpoint. This command shows the trail
and RBA where Extract will begin to write new data. Because a rollover was
issued, the start point is at the beginning (RBA 0) of the new trail file, in this
example file number 7.
INFO EXTRACT group SHOWCH
Sequence #: 7
RBA: 0
6. Issue the following command in GGSCI to reposition the downstream data pump
and start a new output trail file.
ALTER EXTRACT pump EXTSEQNO 7
ALTER EXTRACT pump EXTRBA 0
ALTER EXTRACT pump ETROLLOVER
7. Issue the following command in GGSCI to the data pump Extract to view the new
sequence number of the data pump Write Checkpoint, in this example trail number
9.
INFO EXTRACT pump SHOWCH
Sequence #: 9
RBA: 0
8. Reposition Replicat to start reading the trail at the new Write Checkpoint of the
data pump.
19-31
Chapter 19
Restarting a Primary Extract after System Failure or Corruption
Note:
The LOGBSN gives you the information needed to set Extract back in time to
reprocess transactions. Some filtering by Replicat is necessary because
Extract will likely re-generate a small amount of data that was already
applied by Replicat. FILTERDUPTRANSACTIONS directs Replicat to find and filter
duplicates at the beginning of the run.
19-32
Part II
Administering Oracle GoldenGate
Microservices Architecture
The Oracle GoldenGate MA provides all the tools you need to configure, monitor, and
administer deployments and security. It is designed with the industry-standard
HTTP(s) communication protocol and the JavaScript Object Notation (JSON) data
interchange format. In addition, the architecture provides you with the ability to verify
the identity of clients with basic authentication or Secure Sockets Layer client
certificates.
20
Loading Data from File to Replicat in
Microservices Architecture
By following the steps provided in this topic, data can be precisely replicated from a
source to a target database with zero data loss using a combination of file-based initial
load and change data capture (CDC) processes.
In Loading Data from File to Replicat (page 15-7), the initial load process is
implemented using files. However, Microservices Architecture uses a different
approach. The process of creating and running a replication solution constitutes:
• Initial Load: Used to copy the existing contents of one or more tables from the
source to the target database.
• Change Data Capture: Used to copy transactional changes from the source to the
target database.
Note:
MA doesn’t support loading data with an Oracle GoldenGate direct load.
File-based initial load process is the preferred method for performing data replication
in MA. It’s key components are:
• Initial Load Extract and Replicat: Replicates the existing content of the database
tables.
• Primary Extract and Replicat: Replicates change data from the database tables.
• Distribution Paths: Transfers trail files to the target system.
Before you begin, make sure that the database credential alias is created.
20-1
Chapter 20
Important:
This topic demonstrates the steps for initial load processing using the
AdminClient. However, you can also use curl to perform these steps.
Note:
For precise instantiation to work, the instantiation SCN must come after the
registration SCN.
• The primary Extract is started. It is responsible for change data capture and noting
it’s registration SCN.
• The database is monitored. The database waits for the oldest open transaction’s
SCN to come after the registration SCN. This is the instantiation SCN.
• The instantiation SCN is used when creating the initial load Extract and Replicat
processes.
• The instantiation SCN is used to create the primary Replicat, once the initial load
replication is complete.
To begin, create and start the primary Extract EXTPRIM from the AdminClient, as shown
in the following example:
OGG (not connected) 1> connect https://phoenix.oggdevops.us:9100 as oggadmin
password oggadmin !
Using default deployment 'Phoenix'
20-2
Chapter 20
ExtTrail AA
Table user01.*;
After creating the primary Extract, retrieve the SCN registration number. Run the
REGISTER EXTRACT command in the AdminClient. The following example retrieves an
SCN value of 1608891.
OGG (https://phoenix.oggdevops.us:9100 Phoenix as oggadmin) 4> register extract
EXTPRIM database
2018-03-16T13:37:30Z INFO OGG-02003 Extract EXTPRIM successfully registered
with database at SCN 1608891.
Union All
--
-- Query for current status
--
Select current_scn, 'CURRENT', CURRENT_DATE,
NULL, NULL, NULL, 'SYS', NULL, NULL, NULL
from v$database
Order by 1;
The results of this query can be used to determine the instantiation SCN. The results
for this specific query are:
20-3
Chapter 20
The SCN used to instantiate the initial load Extract is obtained using SQL*Plus. In the
following example, the SQL query uses the instantiation SCN value as 1624963, which
is the oldest SCN of all open transactions that are also past the registration SCN of
1608891.
MIN(START_SCN)
--------------
1624963
If there are no open transactions, then this SQL query returns an empty result. A
detailed query that takes into account the situation where there are no open
transactions is:
Select MIN(SCN) as INSTANTIATION_SCN
From (Select MIN(START_SCN) as SCN
From gv$transaction
Union All
Select current_scn
From gv$database);
20-4
Chapter 20
20-5
Chapter 20
Note:
The primary Replicat is started at the instantiation SCN.
20-6
A
Supported Character Sets
This appendix lists the character sets that Oracle GoldenGate supports when
converting data from source to target.
The identifiers that are shown should be used for Oracle GoldenGate parameters or
commands when a character set must be specified, instead of the actual character set
name. Currently Oracle GoldenGate does not provide a facility to specify the
database-specific character set.
Topics:
A-1
Appendix A
Supported Character Sets - Oracle
A-2
Appendix A
Supported Character Sets - Oracle
A-3
Appendix A
Supported Character Sets - Oracle
A-4
Appendix A
Supported Character Sets - Oracle
A-5
Appendix A
Supported Character Sets - Oracle
A-6
Appendix A
Supported Character Sets - Oracle
A-7
Appendix A
Supported Character Sets - Non-Oracle
ISO-10646 UTF-16
UTF-16
ISO-10646 UTF-32
UTF-32
Windows Cyrillic
windows-1251
Windows Latin-1
windows-1252
Windows Greek
windows-1253
Windows Turkish
windows-1254
Windows Hebrew
windows-1255
A-8
Appendix A
Supported Character Sets - Non-Oracle
Windows Baltic
windows-1257
Windows Vietnam
windows-1258
Windows Thai
windows-874
DOS Latin-1
cp437
DOS Arabic
ibm-720
DOS Greek
cp737
DOS Baltic
cp775
DOS multilingual
cp850
DOS Greek-1
cp851
DOS Latin-2
cp852
DOS Cyrillic
cp855
DOS Turkish
cp857
DOS Portuguese
cp860
DOS Icelandic
cp861
DOS Hebrew
cp862
DOS French
cp863
DOS Arabic
cp864
DOS Nordic
cp865
A-9
Appendix A
Supported Character Sets - Non-Oracle
DOS Urdu
cp868
DOS Greek-2
cp869
ISO-8859-5 Latin/Cyrillic
ISO-8859-5
ISO-8859-6 Latin/Arabic
ISO-8859-6
ISO-8859-7 Latin/Greek
ISO-8859-7
ISO-8859-8 Latin/Hebrew
ISO-8859-8
ISO-8859-9 Latin-5/Turkish
ISO-8859-9
ISO-8859-10 Latin-6/Nordic
ISO-8859-10
ISO-8859-11 Latin/Thai
ISO-8859-11
ISO-8859-14 Latin-8/Celtic
ISO-8859-14
A-10
Appendix A
Supported Character Sets - Non-Oracle
A-11
Appendix A
Supported Character Sets - Non-Oracle
A-12
Appendix A
Supported Character Sets - Non-Oracle
Ukranian (KOI8-U)
KOI8-U
EUC Thai
eucTH
DEC Multilingual
DEC-MCS
HP Latin-1 Roman8
hp-roman8
A-13
Appendix A
Supported Character Sets - Non-Oracle
Windows Japanese
windows-932
Windows Korean
windows-949
EUC Japanese
eucjis
EUC Korean
EUC-KR
A-14
Appendix A
Supported Character Sets - Non-Oracle
GB-18030
gb18030
GB-2312-1980
GB2312
GBK
GBK
HZ GB2312
HZ
A-15
Appendix A
Supported Character Sets - Non-Oracle
A-16
B
Supported Locales
This appendix lists the locales that are supported by Oracle GoldenGate. The locale is
used when comparing case-insensitive object names.
af
af_NA
af_ZA
am
am_ET
ar
ar_AE
ar_BH
ar_DZ
ar_EG
ar_IQ
ar_JO
ar_KW
ar_LB
ar_LY
ar_MA
ar_OM
ar_QA
ar_SA
ar_SD
ar_SY
ar_TN
ar_YE
as
as_IN
az
az_Cyrl
az_Cyrl_AZ
az_Latn
az_Latn_AZ
be
be_BY
bg
bg_BG
bn
bn_BD
bn_IN
ca
ca_ES
cs
B-1
Appendix B
cs_CZ
cy
cy_GB
da
da_DK
de
de_AT
de_BE
de_CH
de_DE
de_LI
de_LU
el
el_CY
el_GR
en
en_AU
en_BE
en_BW
en_BZ
en_CA
en_GB
en_HK
en_IE
en_IN
en_JM
en_MH
en_MT
en_NA
en_NZ
en_PH
en_PK
en_SG
en_TT
en_US
en_US_POSIX
en_VI
en_ZA
en_ZW
eo
es
es_AR
es_BO
es_CL
es_CO
es_CR
es_DO
es_EC
B-2
Appendix B
es_ES
es_GT
es_HN
es_MX
es_NI
es_PA
es_PE
es_PR
es_PY
es_SV
es_US
es_UY
es_VE
et
et_EE
eu
eu_ES
fa
fa_AF
fa_IR
fi
fi_FI
fo
fo_FO
fr
fr_BE
fr_CA
fr_CH
fr_FR
fr_LU
fr_MC
ga
ga_IE
gl
gl_ES
gu
gu_IN
gv
gv_GB
haw
haw_US
he
he_IL
hi
hi_IN
hr
hr_HR
hu
B-3
Appendix B
hu_HU
hy
hy_AM
hy_AM_REVISED
id
id_ID
is
is_IS
it
it_CH
it_IT
ja
ja_JP
ka
ka_GE
kk
kk_KZ
kl
kl_GL
km
km_KH
kn
kn_IN
ko
ko_KR
kok
kok_IN
kw
kw_GB
lt
lt_LT
lv
lv_LV
mk
mk_MK
ml
ml_IN
mr
mr_IN
ms
ms_BN
ms_MY
mt
mt_MT
nb
nb_NO
nl
nl_BE
B-4
Appendix B
nl_NL
nn
nn_NO
om
om_ET
om_KE
or
or_IN
pa
pa_Guru
pa_Guru_IN
pl
pl_PL
ps
ps_AF
pt
pt_BR
pt_PT
ro
ro_RO
ru
ru_RU
ru_UA
sk
sk_SK
sl
sl_SI
so
so_DJ
so_ET
so_KE
so_SO
sq
sq_AL
sr
sr_Cyrl
sr_Cyrl_BA
sr_Cyrl_ME
sr_Cyrl_RS
sr_Latn
sr_Latn_BA
sr_Latn_ME
sr_Latn_RS
sv
sv_FI
sv_SE
sw
sw_KE
B-5
Appendix B
sw_TZ
ta
ta_IN
te
te_IN
th
th_TH
ti
ti_ER
ti_ET
tr
tr_TR
uk
uk_UA
ur
ur_IN
ur_PK
uz
uz_Arab
uz_Arab_AF
uz_Cyrl
uz_Cyrl_UZ
uz_Latn
uz_Latn_UZ
vi
vi_VN
zh
zh_Hans
zh_Hans_CN
zh_Hans_SG
zh_Hant
zh_Hant_HK
zh_Hant_MO
zh_Hant_TW
B-6
C
About the Oracle GoldenGate Trail
This appendix contains information about the Oracle GoldenGate trail that you may
need to know for troubleshooting, for a support case, or for other purposes. To view
the Oracle GoldenGate trail records, use the Logdump utility.
Topics:
C-1
Appendix C
Trail Record Format
can be viewed in the Logdump utility and also by retrieving it with the GGFILEHEADER
option of the @GETENV function.
A trail or extract file must have a version that is equal to, or lower than, that of the
process that reads it. Otherwise the process will abend. Additionally, Oracle
GoldenGate forces the output trail or file of a data pump to be the same version as that
of its input trail or file. Upon restart, Extract rolls a trail to a new file to ensure that each
file is of only one version (unless the file is empty).
Note:
As enhancements are made to the Oracle GoldenGate software, the trail
record format is subject to changes that may not be reflected in this
documentation. To view the current structure, use the Logdump utility.
Figure C-1 Sample Trail Record as Viewed with the Logdump Utility
C-2
Appendix C
Record Header Area
Field Description
Hdr-Ind Should always be a value of E, indicating that the record was created
by the Extract process. Any other value indicates invalid data.
UndoFlag (NonStop) Conditionally set if Oracle GoldenGate is extracting
aborted transactions from the TMF audit trail. Normally, UndoFlag is
set to zero, but if the record is the backout of a previously successful
operation, then UndoFlag will be set to 1. An undo that is performed
by the disc process because of a constraint violation is not marked as
an undo.
RecLength The length, in bytes, of the record buffer.
IOType The type of operation represented by the record. See Table C-2
(page C-7) for a list of operation types.
TransInD The place of the record within the current transaction. Values are:
0 — first record in transaction
1 — neither first nor last record in transaction
2 — last record in the transaction
3 — only record in the transaction
SyskeyLen (NonStop) The length of the system key (4 or 8 bytes) if the source is
a NonStop file and has a system key. If a system key exists, the first
Syskeylen bytes of the record are the system key. Otherwise,
SyskeyLen is 0.
AuditRBA Identifies the transaction log identifier, such as the Oracle redo log
sequence number.
Continued (Windows and UNIX) Identifies whether or not the record is a segment
of a larger piece of data that is too large to fit within one record. LOBs,
CLOBS, and some VARCHARs are stored in segments. Unified records
that contain both before and after images in a single record (due to
the UPDATERECORDFORMAT parameter) may exceed the maximum
length of a record and may also generate segments.
Y — the record is a segment; indicates to Oracle GoldenGate that this
data continues to another record.
N — there is no continuation of data to another segment; could be the
last in a series or a record that is not a segment of larger data.
C-3
Appendix C
Record Header Area
Field Description
Partition For Windows and UNIX records, this field will always be a value of 4
(FieldComp compressed record in internal format). For these
platforms, the term Partition does not indicate that the data
represents any particular logical or physical partition within the
database structure.
For NonStop records, the value of this field depends on the record
type:
• In the case of BulkIO operations, Partition indicates the
number of the source partition on which the bulk operation was
performed. It tells Oracle GoldenGate which source partition the
data was originally written to. Replicat uses the Partition field to
determine the name of the target partition. The file name in the
record header will always be the name of the primary partition.
Valid values for BulkIO records are 0 through 15.
• For other non-bulk NonStop operations, the value can be either 0
or 4. A value of 4 indicates that the data is in FieldComp record
format.
BeforeAfter Identifies whether the record is a before (B) or after (A) image of an
update operation. Records that combine both before and after images
as the result of the UPDATERECORDFORMAT parameter are marked as
after images. Inserts are always after images, deletes are always
before images.
IO Time The time when the operation occurred, in local time of the source
system, in GMT format. This time may be the same or different for
every operation in a transaction depending on when the operation
occurred.
OrigNode (NonStop) The node number of the system where the data was
extracted. Each system in a NonStop cluster has a unique node
number. Node numbers can range from 0 through 255.
For records other than NonStop in origin, OrigNode is 0.
FormatType Identifies whether the data was read from the transaction log or
fetched from the database.
F — fetched from database
R — readable in transaction log
Incomplete This field is obsolete.
AuditPos Identifies the position in the transaction log of the data.
RecCount (Windows and UNIX) Used for LOB data when it must be split into
chunks to be written to the Oracle GoldenGate file. RecCount is used
to reassemble the chunks.
• GGS_TRANS_TIMESTAMP
C-4
Appendix C
Record Data Area
• GGS_TRANS_RBA
• GGS_OP_TYPE
• GGS_BEFORE_AFTER_IND
Each full record image has the same format as if retrieved from a program reading the
original file or table directly. For SQL tables, datetime fields, nulls, and other data is
written exactly as a program would select it into an application buffer. Although
datetime fields are represented internally as an eight-byte timestamp, their external
form can be up to 26 bytes expressed as a string. Enscribe records are retrieved as
they exist in the original file.
When the operation type is Insert or Update, the image contains the contents of the
record after the operation (the after image). When the operation type is Delete, the
image contains the contents of the record before the operation (the before image).
For records generated from an Enscribe database, full record images are output
unless the original file has the AUDITCOMPRESS attribute set to ON. When AUDITCOMPRESS is
ON, compressed update records are generated whenever the original file receives an
update operation. (A full image can be retrieved by the Extract process by using the
FETCHCOMPS parameter.)
C-5
Appendix C
Tokens Area
processes on Windows and UNIX systems are always compressed. The format of a
compressed record is as follows:
column_index column_length column_data[...]
Where:
• column_index is the ordinal index of the column within the source table (2 bytes).
Enscribe records written from the NonStop platform may be compressed. The format
of a compressed Enscribe record is as follows:
field_offset field_length field_value[...]
Where:
• field_offset is the offset within the original record of the changed value (2 bytes).
The first field in a compressed Enscribe record is the primary or system key.
Parameter Value
TKN-HOST : syshq
TKN-GROUP : EXTORA
TKN-BA_IND : AFTER
TKN-COMMIT_TS : 2011-01-24 17:08:59.000000
TKN-POS : 3604496
TKN-RBA : 4058
TKN-TABLE : SOURCE.CUSTOMER
TKN-OPTYPE : INSERT
TKN-LENGTH : 57
TKN-TRAN_IND : BEGIN
C-6
Appendix C
Oracle GoldenGate Operation Types
C-7
Appendix C
Oracle GoldenGate Operation Types
C-8
Appendix C
Oracle GoldenGate Trail Header Record
C-9
Appendix C
Oracle GoldenGate Trail Header Record
support any given token, that token is ignored. Depracated tokens are assigned a
default value to preserve compatibility with previous versions of Oracle GoldenGate.
You can view the trail header with the FILEHEADER command in the Logdump utility. For
more information about the tokens in the file header, see Logdump Reference for
Oracle GoldenGate.
C-10
D
Using the Commit Sequence Number
This appendix contains information about using the Oracle GoldenGate Commit
Sequence Number (CSN) with Oracle and non-Oracle databases.
All database platforms except Oracle, DB2 LUW, and DB2 z/OS have fixed-length
CSNs, which are padded with leading zeroes as required to fill the fixed length. CSNs
that contain multiple fields can be padded within each field. For more information on
CSN, see Overview of CSN in Understanding Oracle GoldenGate
MySQL does not create a transaction ID as part of its event data, so Oracle
GoldenGate considers a unique transaction identifier to be a combination of the
following:
• the log file number of the log file that contains the START TRANSACTION record for the
transaction that is being identified
• the record offset of that record
DB2 z/OS
RBA
where:
• RBA is the 6-byte relative byte address of the commit record within the
transaction log.
Example:
1274565892
D-1
Appendix D
Where:
• LogNum is the the name of the log file that contains the START
TRANSACTION record for the transaction that is being identified.
• LogPosition is the event offset value of that record. Event offset
values are stored in the record header section of a log record.
For example, if the log number is 12 and the log position is 121, the CSN
is:
000012:000000000000121
Oracle
system_change_number
Where:
• system_change number is the Oracle SCN value.
Example:
6488359
SQL Server Can be any of these, depending on how the database returns it:
• Colon separated hex string (8:8:4) padded with leading zeroes and
0X prefix
• Colon separated decimal string (10:10:5) padded with leading zeroes
• Colon separated hex string with 0X prefix and without leading zeroes
• Colon separated decimal string without leading zeroes
• Decimal string
Where:
• The first value is the virtual log file number, the second is the
segment number within the virtual log, and the third is the entry
number.
Examples:
0X00000d7e:0000036b:01bd
0000003454:0000000875:00445
0Xd7e:36b:1bd
3454:875:445
3454000000087500445
Teradata
sequence_ID
Where:
• sequence_ID is a generic fixed-length printable sequence ID.
Example:
0x0800000000000000D700000021
D-2
E
About Checkpoints
This appendix provides information about checkpoints. When working with Oracle
GoldenGate, you might need to refer to the checkpoints that are made by a
process. Checkpoints save the state of the process for recovery purposes. Extract and
Replicat use checkpoints.
Topics:
Read Checkpoint #1
E-1
Appendix E
Extract Checkpoints
SCN: 0.8440969
Redo File: /orarac/oradata/racq/redo01.log
Read Checkpoint #2
Write Checkpoint #1
Sequence #: 2
RBA: 2142224
Timestamp: 2011-01-01 14:16:50.567638
Extract Trail: ./dirdat/eh
Header:
Version = 2
Record Source = A
Type = 6
# Input Checkpoints = 2
# Output Checkpoints = 1
File Information:
Block Size = 2048
Max Blocks = 100
Record Length = 2048
Current Offset = 0
Configuration:
Data Source = 3
Transaction Integrity = 1
Task Type = 0
Status:
Start Time = 2011-01-01 14:15:14
E-2
Appendix E
Extract Checkpoints
See Internal Checkpoint Information (page E-5)for information about the internal
information that starts with the Header entry in the SHOWCH output.
• Timestamp: The timestamp of the record at which the checkpoint was made.
• SCN: The system change number of the record at which the checkpoint was made.
• Redo File: The path name of the transaction log containing the record where the
checkpoint was made.
E-3
Appendix E
Replicat Checkpoints
• Trail Type: Identifies the trail type. EXTTRAIL identifies the trail as a local trail, which
means that it is directly accessible by Oracle GoldenGate processes through the
host filesystem. RMTTRAIL identifies the trail as a remote trail, which means it is not
directly accessible by Oracle GoldenGate processes through the host filesystem.
A trail stored on a shared network device and accessible through NFS-like
services are considered local because they are accessible transparently through
the host filesystem.
See Internal Checkpoint Information (page E-5) for information about the internal
information that starts with the Header entry in the SHOWCH output.
E-4
Appendix E
Internal Checkpoint Information
• Timestamp: The timestamp of the record at which the checkpoint was made.
E-5
Appendix E
Oracle GoldenGate Checkpoint Tables
Column Description
GROUP_NAME (primary key) The name of a Replicat group using this table for checkpoints.
There can be multiple Replicat groups using the same table.
This column is part of the primary key.
GROUP_KEY (primary key) A unique identifier that, together with GROUPNAME, uniquely
identifies a checkpoint regardless of how many Replicat groups
are writing to the same table. This column is part of the primary
key.
SEQNO The sequence number of the input trail that Replicat was reading
at the time of the checkpoint.
RBA The relative byte address that Replicat reached in the trail
identified by SEQNO. RBA + SEQNO provide an absolute position in
the trail that identifies the progress of Replicat at the time of
checkpoint.
AUDIT_TS The timestamp of the commit of the source transaction.
CREATE_TS The date and time when the checkpoint table was created.
LAST_UPDATE_TS The date and time when the checkpoint table was last updated.
E-6
Appendix E
Oracle GoldenGate Checkpoint Tables
Column Description
CURRENT_DIR The current Oracle GoldenGate home directory or folder.
LOG_CSN Stores the high watermark, or the upper boundary, of the CSNs.
Any transaction with a CSN higher than this value has not been
processed.
LOG_XID Not used. Retained for backward compatibility.
LOG_CMPLT_CSN Stores the low watermark, or the lower boundary, of the CSNs.
Any transaction with a lower CSN than this value has already
been processed.
LOG_CMPLT_XIDS Stores the transactions between the high and low watermarks
that are already applied.
VERSION The version of the checkpoint table format. Enables future
enhancements to be identified as version numbers of the table.
Column Description
GROUP_NAME The name of a Replicat group using this table for checkpoints.
There can be multiple Replicat groups using the same table.
This column is part of the primary key of the transaction table.
GROUP_KEY A unique identifier that, together with GROUPNAME, uniquely
identifies a checkpoint regardless of how many Replicat groups
are writing to the same table. This column is part of the primary
key of the transaction table.
LOG_CMPLT_CSN The foreign key that references the checkpoint table. This
column is part of the primary key of the transaction table.
LOG_CMPLT_XIDS_SEQ Creates unique rows in the event there are so many overflow
transactions that multiple rows are required to store them all.
This column is part of the primary key of the transaction table.
LOG_CMPLT_XIDS Stores the overflow of transactions between the high and low
watermarks that are already applied.
E-7