repository·main·Indexed 22 days ago
https://github.com/apache/accumuloA sorted, distributed key/value store designed for robust and scalable data storage and retrieval, leveraging Apache Hadoop's HDFS for storage and Apache Zookeeper for consensus. This documentation covers building from source using Maven, using the accumulo CLI for environment management and JShell sessions, and managing cluster services via accumulo-cluster, accumulo-service, and accumulo-util tools.
Integration tests check for regressions and require significant resources (recommended: 3-4GB free memory and 10GB free disk space).
To run a specific integration test (e.g., WriteAheadLogIT), use the -Dit.test flag. Note that you may need to skip SpotBugs to run these efficiently.
mvn clean verify -Dit.test=WriteAheadLogIT -Dtest=foo -Dspotbugs.skipApache Accumulo uses Maven to compile, test, and package its source. To build the binary tarball from the source code, run the mvn package command. The resulting build artifact will be located at assemble/target/accumulo-<version>-bin.tar.gz. If you wish to speed up the build process by skipping the test suite, append the -DskipTests flag.
# Build the binary tar.gz from source
mvn package
# Build without running tests
mvn package -DskipTestsSpotBugs runs by default during the build process to perform static code analysis for potential bugs. To skip SpotBugs, use the -Dspotbugs.skip flag. To run the security-specific findbugs plugin, use the sec-bugs profile with the following command:
mvn clean verify -Psec-bugs -DskipTestsSunnyDay tests are a minimal set of tests designed to verify basic functionality quickly. They are useful before submitting patches or fixing bugs to ensure no core functions are broken. These run by default during the integration-test phase, but you can trigger them specifically using the sunny profile.
mvn clean verify -PsunnyTo begin using Apache Accumulo, follow these recommended steps:
You can run integration tests against an existing standalone Accumulo cluster by providing a properties file via -Daccumulo.it.properties.
Prerequisite: Copy the accumulo-test jar from $ACCUMULO_HOME/test/target into the lib folder of your standalone Accumulo instance before running tests.
To run all tests compatible with a standalone cluster, use the StandaloneCapableCluster group:
mvn clean verify -Dtest=foo -Daccumulo.it.properties=/home/user/my_cluster.properties -Dfailsafe.groups=StandaloneCapableCluster -Dspotbugs.skipThese tests use MiniAccumuloCluster, a multi-process implementation of Accumulo managed via Java APIs. MAC can use the local filesystem or Apache Hadoop's MiniDFSCluster. While these tests run by default during the integration-test phase, they can be invoked manually. Note that MAC has a higher startup/shutdown overhead compared to other methods. Tests typically run in $ACCUMULO_HOME/test/target/mini-tests.
mvn clean verify -Dspotbugs.skipmvn clean package command from the root of the source tree. You must use the package phase rather than the test phase because some modules are not Java artifacts and require the package phase to resolve sibling dependencies correctly. This ensures you are not building against stale artifacts in your local Maven repository.mvn clean packageWhen using accumulo-service, you can manage groups of services or multiple instances per host using specific configuration patterns.
You can specify a group using the .group= syntax in your arguments. This value is exported as ACCUMULO_RESOURCE_GROUP. For example, if you provide mygroup.group=test, the service name will be constructed using test.
The number of service instances to start or stop is controlled by the ACCUMULO_CLUSTER_ARG environment variable. If this variable is set, the script will attempt to manage multiple processes (e.g., tserver_group_1, tserver_group_2, etc.) based on that value.
To stop or kill every running instance of a specific service type (e.g., all tservers), use the --all flag:
accumulo-service tserver stop --all
# Example: Starting a specific service instance
./accumulo-service tserver start tserver_mygroup_1
# Example: Stopping all tserver instances
./accumulo-service tserver stop --all
# Example: Using group parameter
./accumulo-service tserver start mygroup.group=productionWhen running tests against a standalone cluster, use the following properties to define the environment. These can be passed as command-line arguments (e.g., -Daccumulo.it.cluster.type=STANDALONE) or via a properties file.
| Property | Description |
|---|---|
accumulo.it.cluster.type | The cluster type (MINI or STANDALONE) |
accumulo.it.cluster.clientconf | Path to accumulo-client.properties |
accumulo.it.cluster.standalone.admin.principal | Standalone cluster admin principal (user) |
accumulo.it.cluster.standalone.admin.password | Admin password (if not using Kerberos) |
accumulo.it.cluster.standalone.admin.keytab | Admin keytab (if using Kerberos) |
accumulo.it.cluster.standalone.zookeepers | ZooKeeper quorum |
accumulo.it.cluster.standalone.instance.name | Accumulo instance name |
accumulo.it.cluster.standalone.hadoop.conf | Hadoop configuration directory |
accumulo.it.cluster.standalone.home | Accumulo installation directory |
accumulo.it.cluster.standalone.client.conf | Accumulo conf directory on client |
accumulo.it.cluster.standalone.server.conf | Accumulo conf directory on server |
| Property | Description |
|---|---|
accumulo.it.cluster.standalone.client.cmd.prefix | Prefix for Accumulo client commands |
accumulo.it.cluster.standalone.server.cmd.prefix | Prefix for Accumulo service commands |
To test with multiple users, use indexed properties. The admin user will automatically create these accounts in Accumulo if they do not exist.
For Unsecure (Password-based):
accumulo.it.cluster.standalone.users.$x: The username/principal.accumulo.it.cluster.standalone.passwords.$x: The password for the user.For Kerberos:
accumulo.it.cluster.standalone.users.$x: The principal name.accumulo.it.cluster.standalone.keytabs.$x: The path to the keytab file.For complex compaction requirements, you can use plugins or predefined profiles.
Use --profile <name> to apply a set of iterators defined in the shell's iterator profiles to the compaction.
You can provide custom Java classes to control which files are selected or how the compaction is configured:
--selector <ClassName>: A class implementing a compaction selector.--selectorConfig <key=value,...>: Key-value options passed to the selector.--configurer <ClassName>: A class implementing a compaction configurer.--configurerConfig <key=value,...>: Key-value options passed to the configurer.Note: You cannot specify a custom selector/configurer if you are also using the file selection (sf-*) or file output options.
When using table-based commands in the Accumulo Shell, you can specify which tables the command should operate on using several different methods. The shell resolves these targets in the following order of precedence:
-p, --pattern): Uses a regex pattern to match table names from the list of existing tables.--table): Operates on a specific, single table name.--namespace): Operates on all tables within a specified namespace.If no tables match the provided criteria, the shell will log a warning: No tables found that match your criteria.
# Examples of how table targeting might look in the shell (conceptual):
# Using a pattern
command -p "^test_.*"
# Using a specific table
command --table my_table
# Using a namespace
command --namespace my_namespace
# Using positional arguments
command table1 table2 table3