DOCSEARCHbyAlgolia
Submit Docs

ElastiFlow Documentation

website·Indexed 34 days ago

https://docs.elastiflow.com/

Comprehensive network observability and security analytics documentation. Includes guides for NetObserv Flow, Docker Compose installation with Elasticsearch, and detailed mappings of supported Information Elements (IEs) for NetFlow (v1, v5, v6, v7, v9), IPFIX, AWS VPC Flow Logs, and Azure VNet Flow Logs across various vendor Private Enterprise Numbers (PEN).

Tokens
113.5K
Snippets
451
Records
694
Agent score
60%

What's inside ElastiFlow

  1. Overview of ElastiFlow Network Observability

    ElastiFlow is a high-performance flow collector designed for network observability and security analytics. It collects and analyzes network flow and telemetry data from multiple sources, including NetFlow, IPFIX, and sFlow, to provide a unified, contextualized view of network performance, reliability, and security across infrastructure.

  2. Overview of Azure VNet Flow Log collection

    NetObserv Flow collects 'Virtual network flow logs' from Microsoft Azure Network Watcher to provide visibility into IP traffic flowing through a virtual network. The retrieval process is event-based: NetObserv Flow listens for events from an Azure Event Hub and subsequently retrieves and ingests all new flow logs since the last event.

  3. Overview of Generic HTTP Output

    The Generic HTTP output (currently a technology preview) allows ElastiFlow to send records to an HTTP endpoint. Common use cases include sending data to the http_endpoint input of Elastic Filebeat or the http input of Elastic Logstash.

  4. Overview of NetObserv SNMP capabilities

    NetObserv SNMP is a production-grade SNMP poller designed for high-demand environments. Key features include:

    • Turnkey Setup: Uses pre-defined MIB Object definitions, Object Groups, and Device Groups to avoid manual polling configurations. Adding a device requires only an IP address, credentials, and a Device Group (e.g., juniper_ex).
    • Extensibility: Supports popular network equipment out-of-the-box and allows users to add or modify configurations for custom or rare devices.
    • Efficient Polling: Minimizes bandwidth and device load by performing a low-touch discovery to create an 'SNMP object inventory,' allowing the scheduler to collect multiple OIDs (often 50+) in a single poll.
    • Adaptive Scheduling: Dynamically responds to device reconfigurations or restarts by removing unavailable objects or triggering rediscovery.
    • Scalability: Scales via a configurable pool of concurrent poller workers and utilizes the same processing techniques as NetObserv Flow.

  6. Overview of NetObserv configuration file encryption

    Introduced in NetObserv 7.21.0, this feature allows the encryption of configuration files, which are decrypted on startup. Supported files include:

    • NetObserv Flow: flowcoll.yml and Azure vnet config files
    • NetObserv SNMP: snmpcoll.yml
    • NetObserv SNMP Trap: trapcoll.yml

  7. Overview of SNMP Device Auto-Discovery

    ElastiFlow provides a technology preview feature to automatically discover SNMP-enabled devices using the netobserv CLI tool. The process involves ping-sweeping specified IP ranges, retrieving the sysOID via SNMP to identify the device type, matching it to a prebuilt NetObserv device group, and generating YAML configuration files for NetObserv SNMP to poll.

    Key behaviors include:

    • Routers are identified by their administrative loopback IP address (the first one found).
    • Devices found in earlier network groups are excluded from subsequent groups to prevent duplicates.
    • Local address ranges (127.0.0.0/8, 169.254.0.0/16, fe80::1/10, ::1/128) are ignored.
    • Output YAML files are created by default in /etc/elastiflow/snmp/devices.

  8. Overview of Splunk HEC output

    The Splunk HEC output allows sending records to Splunk Enterprise or Splunk Cloud Platform via the HTTP Event Collector. Note that this output is currently a technology preview and may be subject to change.

  9. Overview of NetObserv Flow capabilities

    NetObserv Flow is a network telemetry processor that receives, decodes, transforms, normalizes, translates, and enriches network flow records. It supports data ingestion from network devices and applications using the following protocols: SNMP, IPFIX, Netflow, sFlow, and Cloud Flow Logs. Additionally, it can enrich flow records with threat information and IP reputation tags via the ElastiFlow NetIntel service to identify suspicious network activity.

  10. Overview of SNMP-Based Flow Enrichment

    NetObserv Flow can enrich flow records by polling exporting devices via SNMP. This process allows the system to determine the names and additional attributes of the ingress and egress interfaces that network traffic traversed, using the interface indices provided in the flow records.

  11. Overview of ElastiFlow NetObserv supported Information Elements (IEs)

    ElastiFlow NetObserv processes network flow records and telemetry from network devices, cloud services, and applications using protocols such as SNMP, IPFIX, Netflow, sFlow, and Cloud Flow Logs. It supports over 7,400 Information Elements (IEs)—attributes related to observed network traffic—across various vendors and networking technologies. Support for specific IEs varies by license tier (Community, Basic, Standard, and Premium).

  12. Netflow v9 Information Element (IE) support in NetObserv Flow

    NetObserv Flow supports over 1500 unique Netflow v9 information elements (IE). These elements are categorized by Private Enterprise Numbers (PEN), covering both IANA standards (PEN 0) and various vendor-specific implementations.
    PENVendorIEs
    0IANA462
    9Cisco581
    193Ericsson8
    4056StreamCORE28
    4321Riverbed12
    8712LANcope (Cisco)43
    25461Palo Alto Networks2
    33610FastIP / Boundary23
    35632ntop370
    40842A10 Networks15