flowcontainer

You can extract specific Wireshark fields by passing them in the extension list to extract(). The resulting value.extension attribute is a dictionary where:

• Key: The field name you requested.
• Value: A list of tuples [(value, packet_index), ...].
  • value: The extracted data.
• packet_index: The index of the IP packet in the stream where this value appeared.

Common Extension Fields:

Warning: Do not attempt to re-extract fields that are already provided by default (e.g., ip.src, tcp.port, ip.length), as this will cause parsing errors.

# Example: Extracting TLS ciphersuites
result = extract("file.pcap", extension=["tls.handshake.ciphersuite"])
for key, value in result.items():
    # extension is a dict: {'tls.handshake.ciphersuite': [('value', index), ...]}
    print(value.extension)

flowcontainer provides specialized parsing for common application-layer protocols via the extension mechanism.

HTTP Parsing

Extracts metadata such as User-agent, http-url, and host.

DNS Parsing

Extracts DNS records, such as A records (NAME, TYPE, ADDRESS).

SSL/TLS Parsing

Extracts security-related information including:

• sni: Server Name Indication.
• cipher_suites: List of supported cipher suites.
• certificates: A list of X509 certificates containing issuer, subject, not_valid_before, not_valid_after, seriral_number, and version.

Note: To use these, ensure you pass the appropriate protocol extensions during the extract call.

# Example of what an SSL flow dictionary might look like:
{
  'pcapname': 'tid_ssl.pcap', 
  'src_ip': '119.78.131.162', 
  'sport': 50665, 
  'dst_ip': '23.56.10.10', 
  'dport': 443, 
  'protocol': 'tcp', 
  'ext_proto': 'TLSv1.2', 
  'start': 1597319999.645109, 
  'end': 1597319999.732076, 
  'sni': 'c.go-mpulse.net', 
  'cipher_suites': '49192,49191...', 
  'certificates': [...] 
}

Import extract from flowcontainer.extractor to parse a PCAP file. The function returns a dictionary where the key is a tuple (filename, protocol, stream_id) and the value is a Flow object.

Function Signature: extract(infile, filter='', extension=None, split_flag=False)

• infile: Path to the PCAP file.
• filter: Packet filtering rules (uses Wireshark syntax). Can be empty.
• extension: A list of additional Wireshark fields to extract (e.g., ['tls.handshake.extensions_server_name']). Can be empty.
• split_flag: If True, the input PCAP is split into smaller PCAPs based on the 5-tuple, and parsed concurrently using a thread pool. This significantly improves speed for large files.

from flowcontainer.extractor import extract

# Basic extraction with specific TLS extensions
result = extract(
    r"1592754322_clear.pcap",
    filter='',
    extension=["tls.handshake.extensions_server_name", "tls.handshake.ciphersuite"]
)

Once you have obtained a Flow object from the extract function, you can access various network metadata and packet sequences:

# Example of accessing Flow attributes
for key, value in result.items():
    print('src ip:', value.src)
    print('dst ip:', value.dst)
    print('sport:', value.sport)
    print('dport:', value.dport)
    print('payload lengths :', value.payload_lengths)
    print('payload timestamps:', value.payload_timestamps)
    print('ip packets lengths:', value.ip_lengths)
    print('ip packets timestamps:', value.ip_timestamps)
    print('default length sequence:', value.lengths)
    print('default timestamp sequence:', value.timestamps)
    print('start timestamp:{0}, end timestamp :{1}'.format(value.time_start, value.time_end))
    print('proto:', value.ext_protocol)
    print('extension:', value.extension)

Once you have a Flow object from the extract result, you can access various network attributes.

Note on Packet Lengths: Length sequences use signed integers to indicate direction:

• Positive (+): Client to Server (C->S).
• Negative (-): Server to Client (S->C).

for key, value in result.items():
    # key is (filename, protocol, stream_id)
    print(f'Flow {key} info:')
    
    # IP and Port info
    print('src ip:', value.src)
    print('dst ip:', value.dst)
    print('sport:', value.sport)
    print('dport:', value.dport)

    # Payload sequences (only non-zero TCP/UDP payloads)
    print('payload lengths:', value.payload_lengths)
    print('payload timestamps:', value.payload_timestamps)

    # IP packet sequences (includes handshake/zero-payload packets)
    print('ip packets lengths:', value.ip_lengths)
    print('ip packets timestamps:', value.ip_timestamps)

    # Default sequences (aliases for payload sequences)
    print('default length sequence:', value.lengths)
    print('default timestamp sequence:', value.timestamps)

    # Timing
    print('start timestamp:', value.time_start)
    print('end timestamp:', value.time_end)

    # Protocol and Extensions
    print('proto:', value.ext_protocol)
    print('extension:', value.extension)

Use the flowcontainer.extractor.extract function to parse PCAP files into network flows.

Return Value Structure: The function returns a dictionary where:

• Key: A tuple containing (filename, protocol, stream_id).
• Value: A Flow object (accessible via flowcontainer.flows.Flow attributes).

Arguments:

• file_path (str): Path to the PCAP file (absolute paths are recommended).
• filter (str): Protocol filter (e.g., 'ip', 'tcp or udp').
• extension (list): List of additional protocol extensions to extract.
• split_flag (bool): Whether to enable flow splitting.
• verbose (bool): Enable verbose logging.

from flowcontainer.extractor import extract

result = extract(
    r"1592993485_noise.pcap",
    filter='ip',
    extension=[],
    split_flag=False,
    verbose=True
)

for key, flow in result.items():
    print(f'Flow {key} info:')
    print(f'src ip: {flow.src}')
    print(f'dst ip: {flow.dst}')
    print(f'sport: {flow.sport}')
    print(f'dport: {flow.dport}')