Kalosm Llama provides the transformer implementation for Llama, Mistral, Phi, and Qwen models. It is the primary engine for running these models within the Kalosm ecosystem.
Core Components
Llama struct: The main entrypoint for interacting with the models.Llama::builder(): Used to configure and create a model instance.ChatModelExt trait: Provides high-level methods like .chat() to start chat sessions or .task() to start specific tasks.Parse macro: Enables structured generation by allowing you to define Rust types (structs and enums) that the model must adhere to when generating output.
Structured Generation Workflow
To force the model to output data in a specific format (like JSON), you can use the .with_constraints() method on a task. This requires implementing the Parse derive macro on your target data structures and passing a parser created via <Type as Parse>::new_parser().
// Example of structured generation with constraints
#[derive(Debug, Clone, Parse)]
struct Pet {
name: String,
description: String,
color: String,
size: Size,
diet: Diet,
}
// ... define enums with #[parse(rename = "...")] ...
let task = llm
.task("You generate realistic JSON placeholders")
.with_constraints(Arc::new(<[Pet; 4] as Parse>::new_parser()));
let stream = task.run(prompt);
