repository·develop·Indexed 43 days ago
https://github.com/rocm/amdmigraphxA graph inference engine designed to accelerate machine learning model inference on AMD hardware. It includes support for various Stable Diffusion versions (1.5, 2.1, 3, and XL) and ControlNet, providing tools for ONNX model conversion, FP16 quantization, and performance benchmarking on ROCm 6.0.
The migraphx-driver is a command-line tool used to interact with the MIGraphX execution engine without writing custom code. It allows you to load, compile, run, and profile deep learning models.
Installation Locations:
/opt/rocm/bin/migraphx-driverAMDMIGraphX/build/bin/migraphx-driverThe examples/migraphx directory contains practical implementations and guides for working with AMD MIGraphX. You can find examples for:
MIGraphX provides several inference examples for vision tasks, implemented in both C++ and Python. These examples demonstrate how to run specific models using the MIGraphX runtime.
The examples/ directory provides practical implementations of common MIGraphX use cases across different domains. You can find specialized examples for the following categories:
MIGraphX is a deep learning (DL) compiler designed to improve inference performance on AMD GPUs by analyzing a model's compute graph and applying program transformations.
Key capabilities include:
torch_migraphx integration). It also provides an ONNX Runtime execution provider.The compiler improves latency and throughput primarily through kernel fusion. This process merges compatible operations into a single kernel launch, which reduces launch overhead and minimizes unnecessary host-to-device memory reads/writes.
Op Builders are abstractions used to simplify the creation of complex operations by composing multiple low-level MIGraphX primitive operators (e.g., dot, multibroadcast, slice, add) into a single high-level operation (e.g., GEMM, Convolution, MatMul).
Key characteristics:
Einsum builder might internally call a MatMul builder).Matchers are a way to compose multiple predicates to identify specific instructions within a program. They allow for complex pattern matching by combining simple matchers. For example, a composite matcher m(m1, m2) first checks for a match of m, followed by a match for m1 and then m2 in sequence.
Commonly used matchers include:
name(string): Matches an instruction where the operator equals the specified name.arg(index): Matches a specific argument of an instruction.args(...): A convenience matcher that matches a sequence of arguments, removing the need to manually specify indices like arg(0), arg(1), etc.// Match a 'sum' operator
auto match_sum = name("sum");
// Match a 'sum' operator with a specific output shape
auto match_sum = name("sum")(standard_shape());
// Match a 'sum' operator with two arguments that are literals
auto match_sum = name("sum")(args(name("@literal"), name("@literal")));migraphx::internal::program represents the parsed model graph. The graph is composed of migraphx::internal::instruction objects, which represent individual operations. To reference specific operations within a program, use migraphx::internal::instruction_ref.The program::run_trace API allows you to inspect operator output buffers during program evaluation. You provide a callback function that receives a trace_info object. This object exposes:
This is useful for debugging or verifying the data flowing through specific parts of a compiled graph.
To add a new dataset (e.g., Video), create a new Python file within the sample_generator/dataset/ directory and implement a class that inherits from Base.
Existing supported datasets include:
imagenetsquadlibrispeechNote: After adding a dataset, you must update generate.py to include it in the generation logic.
You can improve performance and reduce memory usage by quantizing your graph program.
Apply simple 16-bit floating point precision using migraphx::quantize_fp16(prog).
INT8 requires a calibration step to map floating point ranges to integer values. You must provide calibration data for the input parameters using migraphx::quantize_int8_options and migraphx::program_parameters.
// FP16
migraphx::quantize_fp16(prog);
// INT8
std::vector<float> calib_dig; // ... load data
migraphx::quantize_int8_options quant_opts;
migraphx::program_parameters quant_params;
auto param_shapes = prog.get_parameter_shapes();
for(auto&& name : param_shapes.names())
{
quant_params.add(name, migraphx::argument(param_shapes[name], calib_dig.data()));
}
quant_opts.add_calibration_data(quant_params);
migraphx::quantize_int8(prog, targ, quant_opts);