Charm++ Message-Passing Parallel Language and Runtime

AMPI (Adaptive MPI) provides extensions to the standard MPI specification to support advanced features like migration, serialization, and load balancing. All AMPI extensions are prefixed with AMPI_ (e.g., AMPI_Migrate, AMPI_Register_pup).

Most extensions are available in C, C++, and Fortran. However, two specific functions are only available in Fortran:

• AMPI_Command_argument_count
• AMPI_Get_command_argument

Key functional areas provided by these extensions include:

• Serialization: Packing and unpacking program state.
• Load Balancing and Migration: Moving tasks between processors.
• Checkpointing and Fault Tolerance.

The f90charm module provides a Fortran90 interface to the Charm++ parallel object language. It allows developers to write Fortran90 programs that utilize parallel object arrays (Chare Arrays) and message passing, similar to the C++ Charm++ experience.

To develop a program with f90charm, you must follow these steps:

1. Write a Charm Interface file (.ci) to declare Chare arrays and entry methods.
2. Write a Fortran90 program using f90charmmain() as the entry point.
3. Implement Chare entry methods and allocation subroutines in Fortran90.
4. Compile and link with the Charm Fortran library.
5. Run the program.

The Charm++ debugging system consists of a Java GUI client (running on a desktop) and a Charm++ server (running as part of your parallel program). They communicate over a network using a secure protocol.

Key capabilities include:

• State Inspection: View major programmer-visible entities like array elements and messages in queues across the parallel machine. Objects and messages are extracted as raw data and interpreted by the client.
• Breakpoints: Set and remove breakpoints on remote entry points to capture control flow.
• Processor Control: Freeze and unfreeze the execution of specific processors to create consistent snapshots for examination.
• Sequential Debugging: Attach a sequential debugger (like gdb) to a specific subset of processes to manage the debugging scope.

POSE is a tool for Parallel Discrete Event Simulation (PDES) built on top of Charm++. It leverages Charm++'s message-driven execution model but introduces discrete timestamps to control the execution of messages in simulated time.

Key concepts:

• Chares vs. Posers: While Charm++ uses chares (parallel objects), POSE refers to these entities as posers.
• Time Management: POSE manages Object Virtual Time (OVT) for individual posers and a Global Virtual Time (GVT) for the entire simulation.
• Synchronization Strategies: Users can choose synchronization strategies on a per-class basis. POSE is optimized for adaptive synchronization strategies (specifically adapt4 for stability or adept for development), which are optimistic and handle rollbacks, cancellations, and checkpointing automatically.

CkIO is a library designed for parallel I/O in Charm++. It provides high-performance reading and writing by using aggregation to minimize contention on I/O nodes and align with the stripe sizes of parallel file systems (like Lustre).

Key Concepts:

• Output Aggregation: Data is sent to PEs corresponding to the file's stripes and aggregated until a full stripe is ready, then written in a single operation.
• Input Aggregation: Uses intermediate chares called "Buffer Chares" to aggregate read requests. The user should tune the number of Buffer Chares to optimize granularity based on file size and PE count.
• Session-Oriented: CkIO uses a callback-centric, session-based workflow. You open a file, start a session (read or write), perform operations via callbacks, and then close the session/file.

liveViz is a visualization library for Charm++ that acts as a reducer for 2D-image data. It follows a client-server model where image assembly is performed on the server side.

Individual chares compute small chunks (array elements) of a large 2D image. liveViz collects these chunks and combines them into a single large image. The final assembled image is then viewed using a Java-based liveViz client.

Projections is a performance analysis and visualization framework for Charm++ applications. It consists of two main parts:

1. Event Tracing Component: A low-perturbation framework that records performance data (e.g., entry method starts/ends, message sends, scheduler idle time). Most users do not need to manually instrument code as the runtime handles this automatically.
2. Java-based Visualization Tool: A component used to visually study performance data and locate issues.

Workflow:

1. Link your application with appropriate trace generation modules (tracemodes) and execute it.
2. Use the Java-based tool to analyze the generated trace data.

The Topological Routing and Aggregation Module (TRAM) is a library designed to optimize many-to-many and all-to-all collective communication patterns in Charm++ applications. It works by mapping communication onto a user-defined N-dimensional virtual grid topology of Processing Elements (PEs).

Key characteristics:

• Topological Routing: PEs communicate directly only with their 'peers' (neighbors in the grid). Non-peer communication is routed through intermediate destinations.
• Aggregation: TRAM reduces network overhead by buffering and aggregating multiple 'data items' of a single type into fewer, larger messages.
• Implementation: TRAM is implemented as a Charm++ group, meaning an instance of TRAM has one 'local instance' object on every PE used in the run.