Real-time Data Replication

tcVISION is a cross-system solution for the timely, bidirectional data synchronization and replication based on changed data. It turns data exchange into a single-step operation. No middleware or message queueing is required. The data is exchanged in raw format, compressed and reduced to the processing of changed data. Unidirectional or bidirectional data transfers in real-time, time-controlled, or event-based are supported.

Areas of Use

Architecture

The tcVISION replication solution has a modular design. It supports mass data load from one source to one or more targets as well as continuous data exchange process in realtime with change data capture technology.

Features

FLEXIBILITY AND ACTUALITY

High integration potential of the tcVISION solution: Multiple Change Data Capture technologies can be used depending on change frequencies and latency times
Intuitive data mapping offers comprehensive functions for data type conversion and data transformation up to a complete change of the data model
Comprehensive conversion of historically developed mainframe data structures
Highest actuality through continuous real-time processing
Automatic or user-controlled data transformation (ASCII - EBCDIC) for the target (conversion, reformatting, interpretation, etc.)
Support of relational and non-relational databases

USER-FRIENDLINESS

Intuitive dashboard for administration and controlling
Comprehensive monitoring and logging of all data movements ensure transparency across all data exchange processes
Integrated database-specific „Apply“ function to efficiently merge data into the target systems, e.g. direct Insert, Update, Delete, or via JSON through Kafka, or DBMS loader
Integrated data repository with history management to maintain all data structures and data exchange rules
Key management for non-indexed data
Elimination of programming efforts for data transfers
Integrated pooling/streaming processes avoid programming efforts
Message queueing prevents data loss because of unavailability of the target system or delays

DATA INTEGRITY

Practice-proven processes are available to restart a replication after system failures (database errors, transmission errors, etc.)
Master Data Management to ensure data consistency
Ensuring referential integrity through transaction-bound data transfer

tcVISION Components

1. TRANSFORMATION PLATFORM WITH REPOSITORY

This contains all utilities of automatic data mapping to generate metadata for sources and targets, and the rule set for extracting the data from the source, the transformation/processing of the data for the target systems as well as the implementation into the targets. A cost-effective system platform such as UNIX or Linux is recommended for operating the tcVISION transformation platform.

2. DASHBOARD / ADMINISTRATION GUI – COMMAND LINE EDITOR

The tcVISION dashboard is provided for administration, review, operation, controlling, and monitoring of all data exchange processes. The tcVISION Command Line Utilities can be used to automate data exchange processes as well as for the "unattended" operation of data synchronization processes.

3. DATA SOURCES

tcVISION Bulk Reader for the implementation of mass data (initial load or periodical mass data transfers)

Log-based Change Data Capture Agents to capture the change data on record level

4. DATA TARGETS

tcVISION Bulk Loader for the efficient load of mass data into the targets

tcVISION APPLY to use DBMS-specific APIs for the efficient implementation of data changes in realtime in combination with CDC technology at the source

5. EFFICIENT DATA EXCHANGE

The data is exchanged between source and target compressed and in "raw format" via TCP/IP. The data exchange is limited to a minimum.

Change Data Capture Mechanism

DBMS-Extension

Real-time

Timely capturing of all change data
Obtains the change data information directly from DBMS
Secure data management – even across a DBMS restart
Minimum latency

File Processing

Event-based or time-controlled

Processing of DBMS log files
Transfer of the change data within predefined time intervals
Ideal for nightly batch processing
Processing occurs right after log commit

Bulk Transfer

Mass data transfer

Efficient transfer of entire databases and files
Periodic transfer of mass data with low frequency of changes
Ideal as „initial load“ prior to real-time synchronization
For periodic mass data transfers

Batch Compare

Snapshot processing

Comparison with data snapshots
Efficient transfer of change data since the last batch compare run
Automatic determination, creation, and transfer of deltas by tcVISION
Secure restart/recovery after error incidents

Supported Sources and Targets

Sources

Mainframe

IBM Db2 (zOS
IBM Db2 (zVSE)
IBM Db2 (zVM)
IBM IMS/DB
IBM DL1 (zVSE)
VSAM (Batch & CICS - zOS)
VSAM (CICS - zOS & zVSE)
Software AG ADABAS
CA IDMS/DB
CA DATACOM/DB
Flat File Integration

Mainframe (Massload)

IBM Db2 (zOS) Imagecopy/Flashcopy
IBM IMS/DB Unload
Software AG ADABAS Unlaod and ADASAV
CA IDMS/DB Backup
CA DATACOM/DB Backup
Flat File / VSAM

Non-Mainframe

IBM Db2 LUW
IBM Informix
Software AG ADABAS LUW
Microsoft SQL Server
MySQL / MariaDB
ODBC
JDBC
Apache Kafka
Oracle
PostgreSQL
SAP Hana
Solid DB
Teradata
Exasol
Teradata
Kafka

Targets