CoreMark Benchmark Documentation

CoreMark evaluates processor performance using three primary algorithmic workloads designed to exercise different aspects of embedded processing:

1. Linked List: Exercises memory access and pointer manipulation. It uses a custom list_data and list_head structure. The workload involves multiple find operations, sorting the list using merge sort (first by data16, then by idx), and calculating CRCs. The memory layout is intentionally non-sequential to emulate real-world fragmentation.

2. Matrix Multiply: Focuses on tight inner loops and arithmetic optimization. It performs matrix multiplications (A by constant, A by column of B, and A by B) into a result matrix C. This tests the processor's ability to handle data-intensive mathematical operations.

3. State Machine: Exercises branching logic (switch and if statements). It uses a Moore state machine to parse comma-separated strings into various number formats (int, float, scientific). The workload includes parsing, injecting errors into the input, and recovering to the original state.

To ensure results are valid for reporting, adhere to these requirements:

Required:

• The benchmark must run for at least 10 seconds.
• Validation: All validation must succeed for seeds 0,0,0x66 and 0x3415,0x3415,0x66 with a buffer size of 2000 bytes.
  • For performance runs: make XCFLAGS="-DPERFORMANCE_RUN=1" REBUILD=1 run1.log
  • For validation runs: make XCFLAGS="-DVALIDATION_RUN=1" REBUILD=1 run2.log
• Profile Guided Optimization: Use seeds 8,8,8 and a buffer size of 1200 bytes.
  • make XCFLAGS="-DTOTAL_DATA_SIZE=1200 -DPROFILE_RUN=1" REBUILD=1 run3.log
• Consistency: All source files must be compiled with the same flags.
• Data Types: Ensure ee_u8, ee_s16, ee_u16, ee_s32, and ee_u32 match the required bit sizes.

Allowed:

• Changing iterations, toolchains, build/load/run options, memory acquisition methods, seed acquisition methods, and implementations within core_portme.c, core_portme.h, or core_portme.mak.

NOT Allowed:

• Changing any source files other than core_portme* files. Use make check to verify.

To port CoreMark to a bare-bone system (e.g., a microcontroller without an OS), use the files in the barebones folder as a starting point. You must provide your own implementations for printf and a timer with a constant clock frequency.

Prerequisites

Before porting, ensure your environment provides:

1. printf support: For outputting results.
2. Timer support: A timer capable of measuring at least 10 seconds of execution.
   • 32-bit timers: Can often measure the entire duration directly if the frequency and iteration count are appropriate (e.g., a 100MHz 32-bit timer can count for ~42.95 seconds).
   • 24-bit or smaller timers: May require an interrupt-based approach (e.g., a 1KHz interrupt incrementing a software counter) to prevent overflow during the 10-second minimum execution period.

Required Files

Unmodified source files:

• coremark/core_main.c
• coremark/core_list_join.c
• coremark/core_matrix.c
• coremark/core_state.c
• coremark/core_util.c
• coremark/coremark.h

Files requiring modification:

• coremark/barebones/core_portme.c
• coremark/barebones/core_portme.h

In core_portme.c, you must link CoreMark to your hardware's specific functions for timing, I/O, and cache management.

1. Declare Hardware Externals

Declare your hardware-specific functions (e.g., UART init, timer config) as extern.

2. Implement barebones_clock()

This function must return the current time. If your timer increments at a specific frequency (e.g., 100Hz), configure the following constants:

#define CLOCKS_PER_SEC             100
#define GETMYTIME(_t)              (*_t = barebones_clock())
#define MYTIMEDIFF(fin, ini)       ((fin) - (ini))
#define TIMER_RES_DIVIDER          1
#define SAMPLE_TIME_IMPLEMENTATION 1
#define EE_TICKS_PER_SEC           (CLOCKS_PER_SEC / TIMER_RES_DIVIDER)

3. Implement portable_init()

Use portable_init to perform hardware initialization (UART, Cache, Timers) and validate type sizes.

void portable_init(core_portable *p, int *argc, char *argv[])
{
    stdio_init();
    cache_init();
    timer_config();
    
    // Type validation
    if (sizeof(ee_ptr_int) != sizeof(ee_u8 *))
    {
        ee_printf("ERROR! Please define ee_ptr_int to a type that holds a pointer!\n");
    }
    if (sizeof(ee_u32) != 4)
    {
        ee_printf("ERROR! Please define ee_u32 to a 32b unsigned type!\n");
    }
    p->portable_id = 1;
}

extern void timer_config(void);
extern void stdio_init(void);
extern void cache_init(void);
extern unsigned long get_100Hz_value(void);

CORETIMETYPE barebones_clock()
{
  return get_100Hz_value();  
}

void portable_init(core_portable *p, int *argc, char *argv[])
{
    stdio_init();
    cache_init();
    timer_config();
    // ... validation logic ...
    p->portable_id = 1;
}

When setting up a bare-bone project, you must define specific preprocessing macros to tell CoreMark how to operate in a standalone environment.

Project-level Macros

core_portme.h Macros

Update these macros in core_portme.h based on your hardware capabilities:

Additionally, ensure MAIN_HAS_NOARGC is defined:

#ifndef MAIN_HAS_NOARGC
#define MAIN_HAS_NOARGC 1
#endif

To ensure valid results for reporting, follow these rules:

Required

1. Minimum Runtime: The benchmark must run for at least 10 seconds.
2. Validation: All validation must succeed for seeds 0,0,0x66 and 0x3415,0x3415,0x66 with a total buffer size of 2000 bytes.
   • If not using command line arguments, use:

     make XCFLAGS="-DPERFORMANCE_RUN=1" REBUILD=1 run1.log
     make XCFLAGS="-DVALIDATION_RUN=1" REBUILD=1 run2.log

3. Profile Guided Optimization: If using PGO, the profile must be generated using seeds 8,8,8 and a buffer size of 1200 bytes.
   • Use: make XCFLAGS="-DTOTAL_DATA_SIZE=1200 -DPROFILE_RUN=1" REBUILD=1 run3.log
4. Consistency: All source files must be compiled with the same flags.
5. Data Types: Data type sizes must match the following bit widths:
   • ee_u8: 8 bits
   • ee_s16: 16 bits
   • ee_u16: 16 bits
   • ee_s32: 32 bits
   • ee_u32: 32 bits

Allowed

• Changing number of iterations.
• Changing toolchain and build/load/run options.
• Changing method of acquiring data memory blocks or seed values.
• Changing implementation in core_portme.c, core_portme.h, or core_portme.mak.

Not Allowed

• Changing any source files other than core_portme* files (use make check to validate).

Results should be reported following specific formats depending on whether you are reporting standard scores or scaling results.

Standard Reporting Format

CoreMark 1.0 : N / C [/ P] [/ M]

• N: Number of iterations per second (using seeds 0,0,0x66, size 2000).
• C: Compiler version and flags.
• P (Optional): Parameters such as data and code allocation specifics. (Required when reporting CoreMark/MHz).
• M (Optional): Type of parallel execution and number of contexts.

Example: CoreMark 1.0 : 128 / GCC 4.1.2 -O2 -fprofile-use / Heap in TCRAM / FORK:2

Scaling Results Format

CoreMark/MHz 1.0 : N / C / P [/ M]

When reporting scaling, the memory parameter (P) must indicate the memory frequency to core frequency ratio. If the core has a configurable cache frequency ratio, that must also be included.

Example: CoreMark/MHz 1.0 : 1.47 / GCC 4.1.2 -O2 / DDR3(Heap) 30:1 Memory 1:1 Cache