stagesepx

stagesepx is a lightweight, fully automated video analysis tool based on image processing and machine learning. It is designed to analyze videos and automatically split them into a series of distinct stages.

Key capabilities include:

• Automatic Stage Detection: Identifies stable or unstable stages within a video (e.g., detecting the difference between a click action and a page loading state).
• Temporal Mapping: Automatically determines the time intervals for each detected stage.
• Snapshot Generation: Provides visual snapshots (thumbnails) for each stage to help users understand what occurred during that period.
• Zero Configuration: Requires only a video file; no pre-defined templates or prior training/learning is necessary.
• Automated Reporting: Calculates the duration (time cost) for each stage automatically.

stagesepx is designed to act as a preparation layer for AI-driven image processing. It serves as a 'bridge' that allows you to collect and prepare resources from videos without needing to handle raw video file manipulation yourself.

Typical AI Workflow:

1. Video Input: Provide a video containing the target actions.
2. stagesepx Processing: Use the tool to segment or extract relevant frames/resources from the video.
3. AI/Image Processing: Pass the extracted resources directly to your image processing or AI models for analysis.

Because stagesepx analyzes video as its primary subject, it has no runtime dependency on the platform being recorded. You can use it to analyze video captures from:

• Mobile Apps
• PC/Desktop applications
• Web browsers
• Any arbitrary video source

Unlike tools that rely on template matching or OCR (which are sensitive to rotation, resolution, and lighting), stagesepx uses SSIM (Structural Similarity Index) for classification.

• Robustness: By using the SSIM classifier, the tool maintains high stability against environmental interference (rotation, lighting, etc.) because it focuses on structural changes within the video context.
• Accuracy: The tool provides high precision, with errors typically within 0.01s. Accuracy can be further improved by increasing the video's FPS (frames per second).

To achieve high performance, stagesepx utilizes a sampling mechanism that converts continuous time or spatial domains into discrete quantities. This is primarily used in the cutter component to accelerate the cutting process.

• Efficiency: Using a step size of 5 frames can reduce computational load by approximately 80%.
• Trade-off: Sampling introduces a small amount of error. If your video contains very rapid changes or if you require maximum precision, you can disable the sampling feature.

Example log showing high-speed processing (approx. 300ms for a test video):

2019-07-17 10:52:03.429 | INFO     | stagesepx.cutter:cut:200 - start cutting: test.mp4
...
2019-07-17 10:52:03.792 | INFO     | stagesepx.cutter:cut:203 - cut finished: test.mp4

stagesepx is a video analysis tool that can be applied to various testing scenarios by analyzing video content.

Common use cases include:

• Performance Testing: Calculating application startup speeds, page transition speeds, or measuring the duration of specific actions (e.g., time taken for a pen to enter or leave a frame).
• Functional Testing: Using the built-in splitter to cut videos into segments, which can then be passed to image recognition tools like findit to verify functional correctness.
• Game Testing: Analyzing gameplay videos in scenarios where traditional automated testing methods are difficult to implement.

If you need to perform secondary development or custom data processing instead of just viewing a report, you can use stagesepx as a library. By omitting the report generation, you receive a Python object containing raw data.

This data can be converted into a dictionary where each frame contains:

• stage: The classified category
• timestamp: The time offset
• frame_id: The frame number
• video_path: Path to the source video
• data: Additional metadata (can be null)

Example dictionary structure:

{
  "data": [
    {
      "data": null,
      "frame_id": 1,
      "stage": "0",
      "timestamp": 0.0,
      "video_path": "../demo.mp4"
    },
    {
      "data": null,
      "frame_id": 2,
      "stage": "0",
      "timestamp": 0.04,
      "video_path": "../demo.mp4"
    }
  ]
}

Hooks allow you to inject custom logic into the frame-processing pipeline. When a Cutter or Classifier processes a frame, it iterates through a hook_list and applies each hook sequentially.

Key behaviors to note:

• Execution Order: Hooks are executed strictly in the order they are added. Be aware that hooks can interfere with one another.
• The overwrite parameter: This parameter controls whether a hook modifies the original frame or works on a copy.

Common built-in hooks:

• CompressHook: Compresses frames before analysis.
• GreyHook: Converts frames to grayscale.
• FrameSaveHook: Saves individual frames to a specified location during the iteration process.

You can implement your own hooks by following the patterns of these built-in versions to perform personalized frame operations.

Classifiers are used to categorize frames within a video, often leveraging sampling results from a Cutter. stagesepx provides two official classifier types:

• SVM + HoG Classifier: Best suited for videos with complex stages. It can be trained on different video sets to improve recognition accuracy for production environments.
• SSIM Classifier: A traditional, lightweight, and training-free option. It is ideal for simple videos with fewer stages.

While stagesepx favors lightweight machine learning (via sklearn), you can use the output from the Cutter to train your own deep learning models for even higher accuracy.

To get started with stagesepx, you can refer to the official example repository to see practical implementations and usage patterns.

https://github.com/williamfzc/stagesepx/tree/master/example