repository·master·Indexed 42 days ago
https://github.com/konpyutaika/nifikopA Kubernetes operator for automating the lifecycle of Apache NiFi clusters, including provisioning, scaling, and secure operations. It provides Custom Resource Definitions (CRDs) for managing NiFi clusters, users, user groups, registry clients, parameter contexts, and dataflows. The operator supports OIDC integration with Keycloak, path normalization via NGINX proxies for custom context paths, and native NiFi TLS auto-reload.
NiFiKop is a Helm chart used to install the Konpyūtāika Nifi Kubernetes operator. The operator is designed to create, configure, and manage Apache NiFi clusters within a Kubernetes Namespace.
Upon installation, the chart manages several Custom Resource Definitions (CRDs) that allow you to define NiFi-specific resources as Kubernetes objects.
NiFiKop is a Kubernetes operator designed to automate the provisioning, management, autoscaling, and operations of Apache NiFi clusters on Kubernetes.
Key features include:
When updating a dataflow, you must specify an updateStrategy to define how the operator handles existing data in the flow.
drain (DrainStrategy): Shuts down only input components (e.g., Input processors, remote input process groups) and waits for the dataflow to be fully drained before proceeding.drop (DropStrategy): Shuts down all components and immediately drops all flowfiles currently in the flow.spec:
updateStrategy: drain # or 'drop'NiFiKop implements auto-scaling by mimicking the Kubernetes Horizontal Pod Autoscaler (HPA) pattern. Instead of scaling pods directly, NiFiKop introduces a separation of concerns between the cluster definition and the scaling management.
NiFiCluster resource manages the pods, while a new resource, NifiNodeGroupAutoScaler, manages the scaling of the NiFiCluster.scale sub-resource in the Custom Resource Definition (CRD). This allows standard Kubernetes tools (like HPA) to manage the scale.spec.replicas field of the resource.When defining a NifiClusterSpec, you must specify templates for naming node identities and controllers.
Warning: Once these values are defined, do not change them. Changing these values will prevent the operator from being able to manage the cluster.
nodeUserIdentityTemplate: A template for naming the node user identity (e.g., node-%d-mysuffix). Defaults to node-%d-<cluster-name>.nifiControllerTemplate: A template for naming the node controller (e.g., %s-mysuffix). Defaults to %s-controller.controllerUserIdentity: Specifies the identity for the static admin user.The website uses versioning to allow users to access API references for specific React Native releases.
docs and website/sidebars.json are copied to website/versioned_docs and website/versioned_sidebars.versioned_docs or versioned_sidebars unless absolutely necessary, as edits to older versions do not propagate to newer ones.website/versions.json file tracks all available versions in reverse chronological order.website directory.cd react-native-website
cd website
yarn run docusaurus docs:version <newVersion>To avoid creating numerous .yaml files for users and groups across multiple NiFi clusters, NiFiKop provides two built-in managed groups:
You can define the users belonging to these groups directly within the NifiCluster.Spec field using managedAdminUsers and managedReaderUsers. The operator will automatically create the corresponding NifiUser resources and manage the group memberships.
In addition to these, the operator automatically manages a managed-nodes group, which contains a NifiUser for each node in the cluster to ensure correct access permissions.
apiVersion: nifi.konpyutaika.com/v1
kind: NifiCluster
metadata:
name: mynifi
spec:
...
managedAdminUsers:
- identity: "alexandre.guitton@konpyutaika.com"
name: "aguitton"
- identity: "nifiuser@konpyutaika.com"
name: "nifiuser"
managedReaderUsers:
- identity: "toto@konpyutaika.com"
name: "toto"NiFiKop provides automated user and group management to support fully automated cluster setups, including node scaling and operator administrative rights.
NifiUser: Creates or binds a user in the NiFi cluster and applies managed access policies that are continuously synced by the operator.NifiUserGroup: Creates groups in the NiFi cluster, applies access policies, and manages a list of NifiUser resources.For simpler use cases where creating individual Kubernetes resources is unnecessary, you can use two built-in default groups:
NiFiKop allows you to manage the entire lifecycle of NiFi applications using a cloud-native approach. Instead of manual configuration, you describe your desired state using Custom Resources (CRDs), and the operator uses the NiFi REST API to ensure the cluster stays in sync with your definitions.
Key capabilities include:
NiFiDataflow resources based on a NiFi Registry.registry clients, parameter contexts, and dataflows as YAML files to redeploy the same application across different namespaces or clusters.Parameter Contexts to associate different configurations with a Dataflow depending on the target environment.NiFiKop provides several key capabilities for managing Apache NiFi on Kubernetes:
The updateStrategy field in NifiConnectionSpec determines how the operator handles existing data when a connection is deleted:
drain: Leads to blocking the stopping of input/output components until they are empty.drop: Leads to dropping all flowfiles from the connection immediately.spec:
updateStrategy: drain # or 'drop'