DOCSEARCHbyAlgolia
Submit Docs

WanDisco Live Data Migrator

website·Indexed 35 days ago

https://docs.wandisco.com/live-data-migrator/docs

A data migration tool for moving data and metadata between various sources and targets, including HDFS, S3, Azure ADLS Gen2, Ceph, IBM COS, and Oracle Object Storage. The product supports Hive Migrator for connecting to Apache Hive and AWS Glue Data Catalog metastores, provides Prometheus endpoints for monitoring, and includes a CLI and UI for managing migrations. Version 4.0 features include rate limiting, target matching, and remote agent secure connections.

Tokens
538.4K
Snippets
2K
Records
3.6K
Agent score
59%

What's inside Live Data Migrator

  1. Overview of Hive Migrator metadata transfer

    Hive Migrator (bundled with Data Migrator) allows the transfer of metadata from a source metastore to one or more target metastores. Connection is established via local or remote metadata agents. Note that remote metadata agents must be created using the Data Migrator CLI. A source or target filesystem must be configured before connecting to a metastore.

  2. Overview of Migration Management in Data Migrator

    Migration Management allows administrators to delegate the management of specific data migrations to non-admin users (Migration Managers) organized into Management Groups. This is particularly useful for large-scale deployments across multiple teams.

    Key Permissions and Constraints:

    • Admin Users: Retain full access to all features, can assign/change/remove Management Groups for any migration, and are the only users who can adjust migration priority.
    • Non-Admin Migration Managers: Can manage migrations assigned to their group. They can assign migrations to any group they belong to, but cannot leave the assignment blank.
    • Access Restrictions: Non-admin users in a Management Group cannot access migration metric data from the dashboard or metric submenu. Non-admin users not assigned to any group lose access to view or manage all migrations once Migration Management is enabled.

  3. Overview of Cirata Data Migrator capabilities

    Cirata Data Migrator is a software solution designed to automate the migration of Hadoop data and Hive metadata at petabyte scale. It supports migrations while data is under active change, ensuring no business disruption, downtime, or risk of data loss.

  4. Overview of Cirata Data Migrator capabilities

    Cirata Data Migrator is a software solution designed to automate the migration of Hadoop data and Hive metadata at petabyte scale. It supports migrations while data is under active change, ensuring no business disruption, downtime, or risk of data loss.

  6. Overview of Kerberos support in Data Migrator

    Data Migrator supports Kerberos-enabled Hadoop Distributed File System (HDFS) environments as source filesystems. The service authenticates using a Kerberos keytab and does not use passwords or credential caches for the principal. Supported scenarios include:

    • Kerberos active only on the source filesystem.
    • Kerberos active only on the target filesystem.
    • Kerberos active on both source and target with cross-realm trust enabled.
    • Kerberos active on both source and target without cross-realm trust.

  7. Overview of Hive filesystem target agents

    A Hive filesystem target agent allows you to migrate metadata to a specified target filesystem location. This is typically used to move metadata to a loading location for manipulation or reading before it reaches its final destination. Note that Hive filesystem target agents are currently only available via the CLI.

  8. Overview of Cirata Data Migrator

    Cirata Data Migrator is a software solution designed to automate the migration of Hadoop data and Hive metadata at petabyte scale. It supports migrations while data is under active change, ensuring no business disruption, downtime, or risk of data loss.

  9. Overview of Kerberos support in Live Data Migrator

    Live Data Migrator supports Kerberos-enabled Hadoop Distributed File System (HDFS) environments as source filesystems. The service authenticates using a Kerberos keytab and does not use passwords or credential caches for the principal. Supported scenarios include:

    • Kerberos active only on the source filesystem.
    • Kerberos active on both source and target with cross-realm trust enabled.
    • Kerberos active only on the target filesystem.
    • Kerberos active on both source and target without cross-realm trust.

  10. Overview of Migration Management delegation

    Migration Management allows administrators to delegate the control of specific data migrations to non-admin users (Migration Managers) by organizing them into Management Groups.

    Key Permissions & Constraints:

    • Admin Users: Retain full access to all features, including the ability to assign, change, or remove Management Groups and adjust migration priority.
    • Non-Admin Migration Managers: Can manage migrations assigned to their group. They can assign a migration to any group they belong to, but cannot leave a migration unassigned.
    • Restrictions for Non-Admins: Migration metric data (from the dashboard or metric submenu) and migration priority adjustments are not available to non-admin users.
    • Access Control: Non-admin users not assigned to any Migration Management Group will have no access to view or manage any migrations once Migration Management is enabled.

  11. Overview of Hive filesystem target agents

    Hive filesystem target agents allow you to migrate metadata to a specified target filesystem location. This is typically used to move metadata to a loading location for manipulation or reading before it reaches its final destination. In version 4.0, these agents are only configurable via the CLI. A target filesystem must be configured in Data Migrator before a Hive filesystem target agent can be created.

  12. Overview of Cirata Data Migrator

    Cirata Data Migrator is a software solution designed to automate the migration of Hadoop data and Hive metadata at petabyte scale. It allows for data transfer without business disruption or downtime, even while the data is under active change, ensuring no risk of data loss.