OpenSearch Helm Charts

Data Prepper pipelines can be configured in several ways via the Helm chart:

1. Demo Pipeline: Set pipelineConfig.demoPipeline: true to provision a default pipeline with a random source and stdout sink.
2. Inline Configuration: Enable pipelineConfig.enabled: true and provide the configuration object in pipelineConfig.config.
3. Existing Secret: Provide the name of a Kubernetes secret containing your pipeline configuration in pipelineConfig.existingSecret. The secret must contain a key named pipelines.yaml.

pipelineConfig:
  enabled: true
  config: {}
  # OR
  existingSecret: "my-pipeline-secret"

You can manage configuration files in /usr/share/opensearch-dashboards/ using the config parameter.

• String format: The entire content of the configuration file will be replaced by the provided string.
• Map format: The content of the configuration file will be a result of a merge.

Note that content passed through config is processed via tpl (Helm template engine).

To remove the OpenSearch Dashboards deployment and its associated components, use the helm uninstall command with the name of your release.

helm uninstall my-release

To install Data Prepper on Kubernetes, first add the OpenSearch Helm repository and update your local cache. Then, use helm install to deploy the chart. If you do not define an explicit pipeline, the chart will deploy a demo pipeline using a random source and an stdout sink.

# Add the OpenSearch Helm repository
helm repo add opensearch https://opensearch-project.github.io/helm-charts/
helm repo update

# Install the Data Prepper chart
helm install my-data-prepper-release opensearch/data-prepper

To install OpenSearch on a Kubernetes cluster, first ensure you have added the Helm repository following the repository-level instructions. You can then deploy OpenSearch with its associated components (data statefulsets, masters, and clients) using the helm install command. This will deploy the cluster using the default configuration.

Note for Helm 2 users: You must explicitly provide the --name flag. If you do not specify it, Helm 2 will attempt to autogenerate a name.

helm install my-release opensearch/opensearch

Install the OpenSearch Dashboards Helm chart to deploy OpenSearch Dashboards with configurable TLS, RBAC, and other settings. Ensure you have added the Helm repository following the project's main installation instructions before running the install command.

Requirements:

• Kubernetes >= 1.14
• Helm >= 2.17.0
• Memory: A minimum of 4 GiB is required, though 8 GiB is recommended to prevent deployment failure.

helm install my-release opensearch/opensearch-dashboards

To remove the OpenSearch deployment and all its associated components, use the helm uninstall command with the name of your release.

helm uninstall my-release

To contribute code to the OpenSearch Helm Charts repository, follow these steps:

1. Fork the repository: Create a fork of the repository on GitHub.
2. Clone locally: Clone your fork to your local machine using git clone.
3. Submit changes: Follow the specific contribution guidelines outlined in the CONTRIBUTING.md file for pull request processes and coding standards.

To use the OpenSearch Helm charts, add the official repository to your Helm client, update your local cache, and then search or install the desired charts.

Prerequisites:

• Helm installed on your local machine.
• Kubernetes cluster (tested with version 1.19 and above).

# Add the OpenSearch Helm repository
helm repo add opensearch https://opensearch-project.github.io/helm-charts/

# Update your local Helm repository cache
helm repo update

# Search for available charts
helm search repo opensearch

# Install a chart (replace <chart name> with the actual chart name, e.g., opensearch)
helm install my-deployment opensearch/<chart name>

OpenSearch requires vm.max_map_count to be set to a specific value (default 262144) to function correctly. You can enable an initContainer to set this automatically using the sysctlInit option. This container requires privileged permissions.

Use sysctlInit: { enabled: true } to automate the configuration, or manually manage the setting on your nodes.

sysctlInit:
  enabled: true
sysctlVmMaxMapCount: 262144

You can configure lifecycle hooks for the OpenSearch container within the StatefulSet using either opensearchLifecycle or lifecycle. These allow you to define custom actions during container lifecycle events.

# Using opensearchLifecycle
opensearchLifecycle:
  # lifecycle hook configuration

# Or using lifecycle
lifecycle:
  # lifecycle hook configuration