opencv_dart

repository·main·Indexed 42 days ago

https://github.com/rainyl/opencv_dart

Dart bindings for OpenCV that enable computer vision capabilities within Dart and Flutter applications. The library includes support for Mat data structures, ArUco marker detection via ArucoDictionary, high-quality text rendering with FreeType2, QR code detection using WeChatQRCode, and object detection via the WBDetector WaldBoost implementation.

Tokens
14.8K
Snippets
51
Records
65
Agent score
53%

What's inside opencv_dart

  1. Explore the dart_cli example application

    main
    The dart_cli package within the example directory serves as a sample command-line application. It demonstrates how to implement basic argument parsing and provides a standard entrypoint located in the bin/ directory. This is useful for developers looking to understand how to structure a Dart CLI tool that might interact with opencv_dart functionalities.
  2. Getting started with the dartcv_example Flutter project

    main
    The dartcv_example project serves as a starting point for building Flutter applications using the opencv_dart ecosystem. If you are new to Flutter development, you should consult the official Flutter documentation, including the 'Write your first Flutter app' codelab and the Flutter Cookbook for useful samples.
  3. Customize the iOS launch screen assets

    main

    To change the image displayed during the app's launch on iOS, you can either replace the image files directly in the filesystem or use Xcode.

    Method 1: Filesystem replacement

    Replace the existing image files located in the packages/dartcv/example/flutter/ios/Runner/Assets.xcassets/LaunchImage.imageset/ directory with your own assets.

    Method 2: Using Xcode

    1. Open the iOS workspace in Xcode by running open ios/Runner.xcworkspace from your terminal.
    2. In the Xcode Project Navigator, navigate to Runner/Assets.xcassets.
    3. Drag and drop your desired images into the asset catalog to replace the current launch images.
    open ios/Runner.xcworkspace
  4. Customize iOS launch screen assets

    main

    To customize the launch screen for your iOS application, you can replace the existing image files in the packages/opencv_dart/example/ios/Runner/Assets.xcassets/LaunchImage.imageset/ directory with your own assets.

    Alternatively, you can manage these assets using Xcode:

    1. Open the iOS workspace using: open ios/Runner.xcworkspace.
    2. In the Xcode Project Navigator, select Runner/Assets.xcassets.
    3. Drag and drop your desired images into the asset catalog.
    open ios/Runner.xcworkspace
  5. Use FreeType2 for text rendering

    main

    The FreeType2 class provides high-quality text rendering capabilities using FreeType fonts. You can create a renderer, load font data from files or memory buffers, and draw text onto images. It supports both synchronous and asynchronous operations for loading data and rendering text.

    import 'package:opencv_dart/opencv_dart.dart' as cv;
    
    // Create a FreeType2 instance and load a font file
    final ft2 = cv.FreeType2.create(filename: 'path/to/font.ttf');
    
    // Draw text on an image
    // img: InputOutputArray (e.g., a Mat)
    // text: The string to render
    // org: Point for the text origin
    // fontHeight: Size in pixels
    // color: Scalar color
    // thickness: Negative for filled glyphs, positive for line thickness
    // lineType: e.g., cv.LINE_8
    // bottomLeftOrigin: If true, origin is bottom-left
    ft2.putText(
      img, 
      'Hello World', 
      cv.Point(10, 50), 
      24, 
      cv.Scalar(255, 255, 255), 
      thickness: -1
    );
  6. Manage VecMat collections

    main

    A VecMat is a specialized vector container for Mat objects, useful for handling collections of matrices (like contours or detected objects).

    Key Operations

    • add(Mat element): Appends a matrix to the vector.
    • size() / length: Returns the number of matrices in the vector.
    • operator [](int idx): Accesses a matrix at a specific index.
    • clear(): Removes all elements.
    • resize(int newSize): Changes the capacity.
    • asVec(): An extension on List<Mat> to quickly convert a Dart list to a VecMat.
    • cvd: A shorthand extension property on List<Mat> to get a VecMat.
    List<Mat> myMats = [mat1, mat2, mat3];
    
    // Convert List<Mat> to VecMat using extension
    VecMat vec = myMats.cvd;
    
    // Access elements
    Mat first = vec[0];
    
    // Add new element
    vec.add(mat4);
  7. Convert Mat data types and formats

    main

    Use these methods to change how data is represented or to export it to Dart-native structures.

    Type Conversion

    • convertTo(MatType type, {bool inplace = false, double alpha = 1, double beta = 0}): Converts the matrix to a new type with optional scaling ($\alpha \cdot x + \beta$). This is the preferred method for type casting.
    • reinterpret(MatType type): Resets the type of the matrix. If the element size remains the same, the buffer is reused.

    Data Export

    • toList({bool copy = true}): Converts the matrix to a 2D list List<List<num>>. If copy is false, it returns a zero-copy view (writing to the list writes to the native Mat).
    • toList3D({bool copy = true}): For multi-channel matrices, returns a 3D list List<List<List<num>>> ordered as [row][col][channels].
    • toFmtString({int fmtType = FMT_NUMPY, ...}): Returns a string representation of the matrix in various formats (NumPy, CSV, MATLAB, etc.).
    // Convert to 8-bit unsigned integer with scaling
    Mat gray = colorMat.convertTo(MatType.CV_8U, alpha: 1.0/255.0);
    
    // Export to Dart list (deep copy)
    List<List<num>> data = mat.toList(copy: true);
    
    // Export to NumPy-style string
    String numpyStr = mat.toFmtString(fmtType: Mat.FMT_NUMPY);
  8. Perform Non-Maximum Suppression (NMS)

    main

    When performing object detection, multiple overlapping bounding boxes may be returned for the same object. Use NMSBoxes to filter these using non-maximum suppression.

    Parameters:

    • bboxes: A VecRect containing the candidate bounding boxes.
    • scores: A VecF32 containing the confidence scores for each box.
    • scoreThreshold: Minimum score required to consider a detection.
    • nmsThreshold: Intersection over Union (IoU) threshold for suppression.
    • eta: (Optional) parameter for specific NMS implementations.
    • topK: (Optional) number of top results to return.

    Returns a List<int> of indices of the selected boxes.

    List<int> selectedIndices = NMSBoxes(
      bboxes,
      scores,
      0.5, // scoreThreshold
      0.4, // nmsThreshold
    );
  9. Inspect DNN Layers

    main

    You can inspect the internal structure of a Net using the following methods:

    • getLayerNames(): Returns a List<String> of all layer names in the network.
    • getLayer(int index): Returns a Layer object at the specified index.
    • Layer.name: Gets the string name of the layer.
    • Layer.type: Gets the string type of the layer.
    • Layer.inputNameToIndex(String name): Returns the index of a specific input blob by name.
    • Layer.outputNameToIndex(String name): Returns the index of a specific output blob by name.
    • getUnconnectedOutLayers(): Returns indices of layers that are not connected to any subsequent layer.
  10. Calculate Mat statistics

    main

    Extract statistical information from a Mat:

    • mean({Mat? mask}): Calculates the per-channel average. An optional mask can be provided to limit the calculation to specific pixels.
    • stdDev(): Calculates the standard deviation per channel.
    • variance(): Returns the variance (square of stdDev()).
    • sum(): Calculates the per-channel pixel sum.

    All statistical methods return a Scalar object, where the order of values matches the matrix channels (e.g., BGR $\rightarrow$ BGR).

    Scalar avg = mat.mean();
    print('Mean B: ${avg.val[0]}, Mean G: ${avg.val[1]}, Mean R: ${avg.val[2]}');
    
    Scalar sd = mat.stdDev();
  11. Create a Mat from various data sources

    main

    The Mat class (N-dimensional dense arrays) provides several factory constructors to initialize matrices from different data formats:

    • From List: Mat.fromList(rows, cols, type, data) creates a Mat from a flat list. Note that this involves two copies of the data.
    • From 2D/3D Lists: Mat.from2DList(data, type, {cols, rows}) and Mat.from3DList(data, type) allow using nested iterables.
    • From Buffer (High Performance): Mat.fromBuffer(rows, cols, type, buff) creates a Mat that shares the memory of the provided ffi.Pointer<ffi.Void>.
      • Warning: You are responsible for managing the native pointer. The Mat does not own the buffer. The buffer must remain valid as long as the Mat is in use.
    • From Vector: Mat.fromVec(vec, {rows, cols, type, copyData}) creates a Mat from a Vec instance (e.g., VecPoint, VecU8, etc.).
    • From Scalar: Mat.fromScalar(rows, cols, type, s) initializes every element with the value in Scalar s.
    • Specialized Initializers:
      • Mat.empty(): Creates an empty Mat.
      • Mat.zeros(rows, cols, type): Creates a zero-filled array.
      • Mat.ones(rows, cols, type): Creates an array of 1s.
      • Mat.eye(rows, cols, type): Creates an identity matrix.
      • Mat.randn(rows, cols, type, {mean, std}): Creates a matrix with random values from a normal distribution.
      • Mat.randu(rows, cols, type, {low, high}): Creates a matrix with random values from a uniform distribution.
    // Example: Creating a Mat from a list
    final mat = Mat.fromList(2, 2, MatType.CV_8UC1, [255, 0, 0, 255]);
    
    // Example: High performance buffer sharing
    // Note: You must ensure 'ptr' remains valid!
    final matFromBuf = Mat.fromBuffer(rows, cols, type, ptr);