Data Punctum Documentation

website·Indexed 31 days ago

https://docs.datapunctum.com

Documentation for Data Punctum's SPL-based integration tools, including ElasticSPL (v1.3.0) for querying Elasticsearch via DSL or ES|QL and S3SPL (v1.0.0) for querying S3 files using S3Select. Covers configuration dashboards, command references (such as elasticquery, elasticparse, and s3splquery), role-based access control, and installation guides for Splunk environments.

Tokens
13.9K
Snippets
29
Records
103
Agent score
54%

What's inside Data Punctum

  1. Overview of S3SPL Add-On for Splunk

    S3SPL is an add-on for Splunk that enables immediate insight into data stored in S3 using custom Splunk commands. It works with any data source stored in S3 that can be queried using S3 Select, including JSON, CSV, Parquet, and files generated by Splunk Ingest Actions. Key capabilities include:

    • Ad-hoc querying of S3 using WHERE statements via S3 Select.
    • Saving and sharing queries with other users.
    • Automatic timestamp management based on defined field names.
    • Dynamic query adaptation using replacements.
    • An interactive workbench for creating and previewing queries.
    • Administrative management of multiple buckets and saved queries.
    • Granular access control to buckets and prefixes based on Splunk capabilities and roles.
  2. Overview of Splunk Supporting Add-on for Elasticsearch (ElasticSPL)

    ElasticSPL allows Splunk users to query data stored in Elasticsearch using custom Splunk commands. Key capabilities include:

    • Ad-hoc querying of time-series data using DSL, Lucene, or ES|QL via elasticadhoc and elasticquery commands.
    • Ad-hoc querying of aggregated data via elasticadhocstats and elasticquerystats commands.
    • Saving and sharing DSL, Lucene, or ES|QL queries.
    • Automatic timestamp management based on defined field names.
    • Dynamic query adaptation using replacements in DSL or Lucene queries.
    • An interactive explorer dashboard for creating and previewing DSL or Lucene queries.
    • Administrative management of multiple Elasticsearch instances and saved queries with granular access control based on Splunk roles and capabilities.
  3. Execute ad-hoc Elasticsearch queries with elasticadhoc

    The elasticadhoc command is used to query Elasticsearch using DSL, Lucene, or ES|QL syntax. It is recommended for testing queries or performing quick checks. For repeated queries, elasticquery is preferred due to the complexity and readability of the elasticadhoc syntax.

    Supported query syntaxes include:

    • Elasticsearch Query DSL
    • Apache Lucene
    • Elasticsearch ES|QL
  4. Manage s3spl application settings via the Configuration Dashboard

    All s3spl configuration is managed through the app's configuration dashboard, accessible via the main application menu. The dashboard is organized into five functional sections: Bucket, Query, License, Proxy, and Logging.
  5. Query S3 files using the s3spladhoc command

    The s3spladhoc command allows for data exploration and quick checks of files in S3 by executing S3 Select queries. The query syntax follows the S3 Select SQL reference, specifically the WHERE clause.
  6. Understand ElasticSPL Community Edition limitations and support

    The ElasticSPL Community Edition (version 1.3.0) is free to use but is limited to one (1) configured node connection. Support for the Community Edition is provided on a best-practices basis with an expected response time of 2-3 business days via the Splunk Usergroups Slack Channel (#elasticspl) or email at support@datapunctum.com.
  7. Manage ElasticSPL configuration via the dashboard

    ElasticSPL configuration is managed through a centralized configuration dashboard accessible via the application menu. The dashboard is divided into four primary sections: Instance, Query, License, and Logging. Access to these sections is restricted based on the user's assigned role.
  8. Run ad-hoc Elasticsearch queries with elasticadhocstats

    The elasticadhocstats command (v1.3.0) is used to query Elasticsearch using DSL or ES|QL provided directly within the search command. It is primarily intended for testing queries and quick checks. Results are returned as aggregated data formatted as a table. While timestamp fields and dynamic timestamp parsing are supported, the returned data does not necessarily have to be time-series data. For repeatable queries, elasticquerystats is recommended over elasticadhocstats due to the potential length and complexity of the command.
  9. Update a saved query in S3SPL Workbench

    To update a saved query, you must have the s3spl_power role and be a member of one of the query's writing groups.

    1. Select the query to update from the Query dropdown.
    2. Select the target bucket from the Bucket dropdown.
    3. Modify the input fields in the sidebar as needed.
    4. Click the Update button to open the Update Query modal.
    5. Click Update to save the changes.
  10. Manage permissions for s3splquery

    To execute a query via s3splquery, a user must meet two permission requirements:

    1. Query Permissions: The user must have 'read' permissions assigned to the specific saved query.
    2. Command Permissions: The user must have execution permissions for the command, which is provided by the s3spl_user role.
  11. Upload and apply a license in ElasticSPL

    To upload a license in ElasticSPL (S3SPL), navigate to the License tab and click the Add License button in the top right corner. Paste the license JSON into the provided text area and click Add. The system will validate the license for validity and compatibility with the installed S3SPL version before applying it immediately. License status can be monitored within the License tab.
  12. Control access to S3SPL buckets and queries using Knowledge Object Permissions

    S3SPL utilizes Splunk's access control mechanisms to manage permissions at the knowledge object level (buckets and queries) rather than the user level.

    Each knowledge object maintains two lists:

    1. Roles allowed to access (read) the object.
    2. Roles allowed to edit (write) the object.

    Note: The owner of a knowledge object always retains full access and editing privileges, regardless of whether they are members of a role that provides write access.