EloquentTinyML

EloquentTinyML provides a simplified interface for loading TensorFlow Lite for Microcontrollers models and running predictions on Arduino.

Setup Workflow

1. Include Headers: You must include your model header file before the runtime and the EloquentTinyML wrapper.
2. Select Runtime: Include either <tflm_esp32.h> or <tflm_cortexm.h> based on your board.
3. Include Wrapper: Include <eloquent_tinyml.h>.
4. Define Arena Size: Define an ARENA_SIZE constant. This is the memory allocated for the tensor arena. Start with a high value (e.g., 10000) and decrease it until the model stops working to optimize memory usage.
5. Instantiate Model: Use Eloquent::TF::Sequential<TF_NUM_OPS, ARENA_SIZE> tf; to create your model instance.
6. Initialize: In setup(), call tf.begin(yourModelHeader) and check .isOk() to ensure the model loaded correctly. You can inspect errors via tf.exception.toString().
7. Run Inference: Use tf.predict(input) in loop(). The result is accessed via tf.classification for classification tasks.

Configuration Options

If your model was not generated using the everywhereml Python package, you may need to manually configure the following in setup():

• tf.setNumInputs(n): Set the number of input features.
• tf.setNumOutputs(n): Set the number of output classes/values.
• tf.resolver.Add[OpName](): Manually add required operations (e.g., tf.resolver.AddFullyConnected(), tf.resolver.AddSoftmax()).

Benchmarking

You can measure inference performance using tf.benchmark.microseconds(), which returns the time taken for a single prediction in microseconds.

/**
 * Run a TensorFlow model to predict the IRIS dataset
 */
// replace with your own model
// include BEFORE <eloquent_tinyml.h>!
#include "irisModel.h"
// include the runtime specific for your board
#include <tflm_esp32.h>
// now you can include the eloquent tinyml wrapper
#include <eloquent_tinyml.h>

#define ARENA_SIZE 2000

Eloquent::TF::Sequential<TF_NUM_OPS, ARENA_SIZE> tf;

void setup() {
    Serial.begin(115200);
    
    // configure input/output
    tf.setNumInputs(4);
    tf.setNumOutputs(3);
    
    // add required ops
    tf.resolver.AddFullyConnected();
    tf.resolver.AddSoftmax();
    
    while (!tf.begin(irisModel).isOk()) {
        Serial.println(tf.exception.toString());
    }
}

void loop() {
    if (!tf.predict(x0).isOk()) {
        Serial.println(tf.exception.toString());
        return;
    }
    
    Serial.print("predicted class ");
    Serial.println(tf.classification);
    
    Serial.print("Inference time: ");
    Serial.print(tf.benchmark.microseconds());
    Serial.println(" us");
    
    delay(1000);
}

The Eloquent::TF::Sequential class is the primary interface for managing TensorFlow Lite models. It is templated with TF_NUM_OPS and ARENA_SIZE.

Methods

• begin(model_header): Initializes the model with the provided header. Returns a status object that can be checked with .isOk().
• predict(input): Performs inference on the provided input. Returns a status object.
• setNumInputs(int n): Configures the number of input features.
• setNumOutputs(int n): Configures the number of output features/classes.

Properties

• tf.classification: Holds the result of the last successful classification prediction.
• tf.exception.toString(): Returns a string representation of any error encountered during initialization or inference.
• tf.benchmark.microseconds(): Returns the execution time of the last prediction in microseconds.
• tf.resolver: Accesses the operator resolver to manually add required TensorFlow operations (e.g., AddFullyConnected(), AddSoftmax()).

The Eloquent::Extra::Time::Benchmark class provides utilities for measuring the execution time of code blocks or functions in microseconds (us) or milliseconds (ms).

Key Methods

• benchmark(Callback callback): Executes a provided callback function, measures its duration, and returns the elapsed time in milliseconds.
• timeit(Callback callback): Executes a provided callback function and measures its duration, but does not return the value (useful for side-effect only functions).
• start(): Resets and starts the internal timer.
• stop(): Stops the timer and returns the elapsed time in milliseconds.
• microseconds() / us(): Returns the elapsed time in microseconds.
• millis() / ms(): Returns the elapsed time in milliseconds.

Note: While stop() and benchmark() return milliseconds, you can use us() or micros() to retrieve the higher-precision microsecond measurement.

#include <eloquent_tinyml/benchmark.h>

Eloquent::Extra::Time::Benchmark bench;

// Example 1: Using benchmark() to get execution time in ms
size_t durationMs = bench.benchmark([]() {
    // Your code or model inference here
    do_something();
});

// Example 2: Using timeit() for side-effect only tasks
bench.timeit([]() {
    do_something_else();
});

// Example 3: Getting high precision microsecond measurement
size_t durationUs = bench.us();

The PersonDetection class provides a pre-configured implementation for detecting people in 96x96 grayscale images. It manages a TensorFlow Lite model internally and provides high-level methods to initialize the model, run inference on image buffers or camera objects, and check for detection results.

Key Methods

• begin(): Initializes the model and the TensorFlow resolver. Must be called before running detection.
• run(uint8_t *image): Runs detection on a raw 96x96 grayscale image buffer. The input pixels are automatically normalized (centered around 0).
• run(Camera& camera): A template method that runs detection directly using a camera object's frame buffer.
• operator bool(): Returns true if a person is detected based on the internal scores and a default threshold of 180.
• personScore(): Returns the raw score for the 'person' class.
• notPersonScore(): Returns the raw score for the 'not person' class.

Configuration

You can control the memory allocation for the TensorFlow arena using the PERSON_DETECTION_ARENA_SIZE macro. If not defined, it defaults to 90000L.

#include <eloquent_tinyml/zoo/person_detection.h>

// Using the static instance provided in the eloq namespace
void detect(eloq::tinyml::zoo::personDetection& detector, uint8_t* frame) {
    if (detector.begin().isOk()) {
        if (detector.run(frame).isOk()) {
            if (bool(detector)) {
                // Person detected!
            }
        }
    }
}

The Eloquent::TF::Sequential class is the primary interface for loading a TensorFlow Lite Micro model and performing inferences. It manages the interpreter, tensor arena, and operator resolver.

Template Parameters

When instantiating Sequential, you must provide two template arguments:

1. numOps: The maximum number of unique TensorFlow Lite operators your model uses.
2. tensorArenaSize: The size (in bytes) of the memory buffer used for the tensor arena.

Workflow

1. Instantiate: Create a Sequential<numOps, tensorArenaSize> object.
2. Configure Dimensions: Call setNumInputs(n) and setNumOutputs(n) to define the model's I/O shape.
3. Initialize: Call begin(model_data_pointer) to load the model and allocate tensors. This returns an Exception object that should be checked for errors.
4. Inference: Use predict(float *x) for float inputs or predict(int8_t *x) for quantized integer inputs.
5. Retrieve Results: Use output(i) to get the float value of the $i$-th output.

Error Handling

Methods like begin() and predict() return an Exception&. You should check if the model initialized or executed correctly using the exception member.

// Example usage of Sequential
// Assuming 10 ops and 10KB arena
Eloquent::TF::Sequential<10, 10240> model;

model.setNumInputs(12);
model.setNumOutputs(3);

if (model.begin(my_model_data) != Eloquent::Error::Exception::clear()) {
    // Handle error
}

float input_data[12] = { ... };
if (model.predict(input_data) != Eloquent::Error::Exception::clear()) {
    // Handle error
}

float first_output = model.output(0);