Malcolm Network Traffic Analysis Suite

Malcolm is a network traffic analysis platform that processes packet capture (PCAP) files and Zeek logs. It functions by parsing network session data and enriching it with metadata such as:

• GeoIP mapping: Geographic location of IP addresses.
• Hardware manufacturer lookups: Using MAC address OUIs.
• Asset identification: Assigning names to network segments and hosts based on a user-defined asset inventory.
• JA4 fingerprinting: Network fingerprinting for identification.

Enriched data is stored in OpenSearch and is accessible via two primary interfaces:

1. OpenSearch Dashboards: For flexible data visualization and protocol overviews using prebuilt dashboards.
2. Arkime: A powerful tool for searching and identifying specific network sessions related to security incidents.

Malcolm can be deployed as a dedicated sensor appliance or locally on a workstation for processing offline artifacts (Zeek logs, PCAPs) during incident response.

Malcolm is a comprehensive network analysis platform built upon a collection of open-source tools. Its architecture integrates traffic capture, protocol analysis, metadata indexing, threat detection, and visualization.

Key functional areas include:

• Traffic Capture & Analysis: Uses Arkime (for PCAP processing and browsing), Zeek (network analysis framework), and Suricata (IDS/threat detection). For simpler deployments, netsniff-ng or tcpdump can be used.
• Data Indexing & Search: Primarily uses OpenSearch (or Elasticsearch) for indexing session metadata, with OpenSearch Dashboards (or Kibana) for visualization. Logstash and Filebeat handle Zeek log ingestion.
• Threat Hunting & File Scanning: Strelka provides container-based file scanning, utilizing tools like Yara, Capa, and ClamAV to identify malware and capabilities.
• Intelligence & Enrichment: Integrates threat intelligence feeds (MISP, TAXII, Google, Mandiant) via the Zeek intelligence framework and uses NetBox for asset inventory enrichment.
• Infrastructure & Orchestration: Deployed via Docker or Podman for local use, and Kubernetes (including the Malcolm Helm chart) for distributed or cloud environments.

Malcolm is a network traffic analysis tool suite designed for network security monitoring. It automates the normalization, enrichment, and correlation of network data to provide visibility into communications.

Key Capabilities:

• Data Ingestion: Accepts full packet capture (PCAP) files, Zeek logs, and Suricata alerts. Data can be uploaded via a browser-based interface or forwarded live using lightweight forwarders.
• Analysis Interfaces:
  • OpenSearch Dashboards: Provides data visualization through prebuilt dashboards for protocol overviews.
  • Arkime: Used for deep session identification and investigating suspected security incidents.
• Deployment Model: Operates as a cluster of software containers, making it suitable for both long-term SOC deployment on Linux servers and portable incident response on devices like MacBooks.
• Security: All communications (UI and remote log forwarders) use industry-standard encryption.

Malcolm is a network traffic analysis tool suite designed for network security monitoring. It automates the normalization, enrichment, and correlation of network data to provide visibility into communications.

Key capabilities include:

• Data Ingestion: Accepts PCAP files, Zeek logs, and Suricata alerts via a browser-based interface or via lightweight forwarders for live capture.
• Analysis Interfaces:
  • OpenSearch Dashboards: Provides data visualization and prebuilt dashboards for protocol overviews.
  • Arkime: Used for identifying and investigating specific network sessions during security incidents.
• Deployment: Operates as a cluster of software containers, making it portable for both long-term SOC deployment on Linux servers or incident response on local machines (e.g., MacBook).
• Security: Uses industry-standard encryption for all user interface and remote log forwarder communications.

Malcolm allows you to analyze Zeek logs using the same interface and processes used for native Arkime sessions. Malcolm maps Zeek log values to the Arkime session database schema and creates new "native" Arkime fields for any Zeek values that do not have an equivalent in Arkime.

Key fields for distinguishing and filtering Zeek data in OpenSearch:

• Data Source (event.provider): Use zeek to identify records from Zeek logs (vs arkime for native Arkime sessions).
• Log Type (event.dataset): Corresponds to the specific Zeek .log file (e.g., conn, weird, dns).

Example Queries:

• Restrict to conn.log records: event.provider == zeek && event.dataset == conn
• Restrict to weird.log records: event.provider == zeek && event.dataset == weird

Note: While Zeek records appear in the Sessions view like native Arkime sessions, they do not contain full packet contents. Consequently, PCAP-related buttons (viewing/exporting) will not function for Zeek-sourced records.