DataWave Documentation

The config-compare tool is used to compare the configurations between two DataWave config.xml files.

It operates under the assumption that each file has a configured data.name field, which is also used as a prefix for other fields. The tool identifies and reports:

• Identical fields: Fields that are configured the same (the tool ignores the differing prefixes).
• Different fields: Fields that exist in both configuration files but contain different values.
• Unique fields: Fields that exist in only one of the two files.

The DataWave Quickstart is designed to automate the configuration, building, deployment, and testing of a standalone DataWave environment. It provides automation for setting up and tearing down core cluster services including DataWave, Hadoop, Accumulo, and ZooKeeper.

Key features include:

• Automatic downloading, extraction, and installation of cluster services under the quickstart home directory.
• Services are owned and executed by the current user for simplicity.
• Convenience methods for testing DataWave's ingest and query components.

Docker Alternative: If you prefer not to install services locally, you can run the entire quickstart environment as a Docker container (hosted on a CentOS 7 base image) by following the instructions in the docker/ directory.

DataWave is a Java-based ingest and query framework that leverages Apache Accumulo to provide fast, secure access to data. It is designed for use cases such as:

• Data fusion across structured and unstructured datasets.
• Construction and analysis of distributed graphs.
• Multi-tenant data architectures with distinct security requirements.
• Fine-grained data access control integrated with user-authorization services and PKI.

For a quick start, refer to the DataWave Quickstart.

The common-utils library provides low-level utility classes used across the DataWave ecosystem. It is a foundational component utilized by tablet server iterators, microservices, and the legacy Wildfly web service.

Important Usage Note: Because this library is used system-wide at all layers, it is intended to be highly stable. When contributing or extending it, avoid adding new dependencies. Only low-level additions with minimal or no external dependencies should be included to prevent cascading upgrade requirements across the entire DataWave system.

In-Memory Accumulo is a lightweight, in-memory implementation of Apache Accumulo. It is designed for two primary use cases:

1. Unit Testing: Use it to execute unit tests that require Accumulo interaction without the overhead and startup time of a full Mini Accumulo Cluster.
2. Local Caching: Use it as an in-memory local cache that sits in front of a remote, production Accumulo server.

Note: This is a simplified implementation and does not attempt to replicate all features of the full Apache Accumulo distribution.