enkiTS Task Scheduler

enkiTS is a lightweight, permissively licensed C and C++ Task Scheduler designed for creating parallel programs. It aims to help developers handle both data-level and task-level parallelism to maximize multicore CPU performance. It is designed to be lean, fast on low thread counts, and scalable, with a focus on zero allocations during scheduling.

Key features include:

• Braided Parallelism: Tasks can be issued from other tasks or from the thread that created the Task System.
• Task Pinning: Ability to pin tasks to specific threads.
• Task Priorities: Configurable task priorities (default is 3, but can be increased via ENKITS_TASK_PRIORITIES_NUM).
• External Threads: Ability to register external threads for use with the API.
• Dependencies: Support for setting dependencies between tasks.
• Completion Actions: Perform actions upon task completion to avoid re-scheduling overhead.
• Custom Allocators: API for providing custom allocators.

enkiTS allows you to set priorities for task sets using enki::TASK_PRIORITY_LOW and enki::TASK_PRIORITY_HIGH. When calling WaitforTask, you can pass a priority level to ensure the calling thread only executes tasks of that priority or higher while waiting.

#include "TaskScheduler.h"

enki::TaskScheduler g_TS;

struct ExampleTask : enki::ITaskSet {
    ExampleTask( ) { m_SetSize = size_; }
    void ExecuteRange( enki::TaskSetPartition range_, uint32_t threadnum_ ) override {
        // implementation
    }
};

int main(int argc, const char * argv[]) {
    g_TS.Initialize();

    ExampleTask lowPriorityTask( 10 );
    lowPriorityTask.m_Priority  = enki::TASK_PRIORITY_LOW;

    ExampleTask highPriorityTask( 1 );
    highPriorityTask.m_Priority = enki::TASK_PRIORITY_HIGH;

    g_TS.AddTaskSetToPipe( &lowPriorityTask );
    for( int task = 0; task < 10; ++task ) {
        g_TS.AddTaskSetToPipe( &highPriorityTask );
        // wait for task but only run tasks of the same priority or higher on this thread
        g_TS.WaitforTask( &highPriorityTask, highPriorityTask.m_Priority );
    }
    g_TS.WaitforTask( &lowPriorityTask );

    return 0;
}

If you have an existing thread pool or external threads that you want to participate in the enkiTS scheduling, you can register them using g_TS.RegisterExternalTaskThread().

To support these threads, configure the enki::TaskSchedulerConfig by setting numExternalTaskThreads before calling Initialize().

#include "TaskScheduler.h"

enki::TaskScheduler g_TS;

struct ParallelTaskSet : ITaskSet {
    void ExecuteRange( enki::TaskSetPartition range_, uint32_t threadnum_ ) override {
        // Do something
    }
};

void threadFunction() {
    g_TS.RegisterExternalTaskThread();

    // ... work ...

    ParallelTaskSet task;
    g_TS.AddTaskSetToPipe( &task );
    g_TS.WaitforTask( &task);

    g_TS.DeRegisterExternalTaskThread();
}

int main(int argc, const char * argv[]) {
    enki::TaskSchedulerConfig config;
    config.numExternalTaskThreads = 1;

    g_TS.Initialize( config );

    std::thread exampleThread( threadFunction );
    exampleThread.join();

    return 0;
}

For threads that spend significant time blocked (e.g., IO threads), you can use g_TS.WaitForNewPinnedTasks() inside a loop. This allows the thread to 'sleep' until new pinned tasks are available, preventing high CPU usage while idling.

Combine this with g_TS.RunPinnedTasks() to execute the tasks once they arrive.

#include "TaskScheduler.h"

enki::TaskScheduler g_TS;

struct RunPinnedTaskLoopTask : enki::IPinnedTask {
    void Execute() override {
        while( !g_TS.GetIsShutdownRequested() ) {
            g_TS.WaitForNewPinnedTasks(); // sleep until new pinned tasks arrive
            g_TS.RunPinnedTasks();
        }
    }
};

struct PretendDoFileIO : enki::IPinnedTask {
    void Execute() override {
        // Do file IO
    }
};

int main(int argc, const char * argv[]) {
    enki::TaskSchedulerConfig config;
    config.numTaskThreadsToCreate += 1;

    g_TS.Initialize( config );

    RunPinnedTaskLoopTask runPinnedTaskLoopTasks;
    runPinnedTaskLoopTasks.threadNum = g_TS.GetNumTaskThreads() - 1;
    g_TS.AddPinnedTask( &runPinnedTaskLoopTasks );

    PretendDoFileIO pretendDoFileIO;
    pretendDoFileIO.threadNum = runPinnedTaskLoopTasks.threadNum;
    g_TS.AddPinnedTask( &pretendDoFileIO );

    g_TS.WaitforAllAndShutdown();

    return 0;
}