repository·main·Indexed 23 days ago
https://github.com/ai-hero-dev/ai-heroAn educational resource and codebase designed to transition developers from traditional software engineering to AI engineering. It provides runnable examples, exercises, and libraries focusing on concepts such as connecting Vercel's AI SDK to various models, implementing text streaming, managing conversation history with CoreMessage arrays, and generating structured outputs using Zod and OpenAI-compatible APIs.
examples/ directory contains self-contained, runnable code samples designed to demonstrate specific AI engineering concepts or techniques. Each example is intended to be paired with educational articles available at AI Hero.Retrieval Augmented Generation (RAG) is a technique used to provide Large Language Models (LLMs) with specific, relevant context that was not part of their original training dataset.
Instead of feeding an entire massive document into a prompt (which is token-intensive and may exceed context windows), RAG follows a three-step workflow:
The Model Context Protocol (MCP) is an open standard that acts as a standardized interface layer between Large Language Models (LLMs) and external data sources or APIs (like Slack, GitHub, or local filesystems).
Instead of writing custom 'glue code' for every unique API to connect it to an LLM, developers can use MCP to create a reusable connection. An MCP server contains the logic to interact with specific APIs and exposes a set of 'tools' in an LLM-friendly format. This allows any MCP-compliant client (such as Cursor, Windsurf, or Claude Code) to interact with those tools without needing to know the underlying API implementation.
A 'Hard' Human In The Loop implementation is used to prevent an LLM from performing irreversible or destructive actions (like deleting files or sending emails) without explicit authorization.
Instead of allowing the model to execute a tool call directly, the workflow should:
This pattern ensures the probabilistic nature of the LLM cannot result in unintended side effects.
When building an agentic loop, you need to define a schema that allows an LLM to choose between different tasks. While z.union is a common approach in Zod, it translates to a JSON Schema oneOf structure which can be difficult for LLMs to parse reliably.
Instead, use a single z.object with a type field defined as an enum, and make the specific payload fields (like query or urls) .optional(). This approach is more robust for Structured Outputs.
import { z } from "zod";
export const actionSchema = z.object({
type:
z
.enum(["search", "scrape", "answer"])
.describe(
`The type of action to take.
- 'search': Search the web for more information.
- 'scrape': Scrape a URL.
- 'answer': Answer the user's question and complete the loop.`,
),
query:
z
.string()
.describe(
"The query to search for. Required if type is 'search'.",
)
.optional(),
urls:
z
.array(z.string())
.describe(
"The URLs to scrape. Required if type is 'scrape'.",
)
.optional(),
});When selecting a model to act as a judge, consider two primary strategies:
gemini-1.5-flash) if the task is primarily classification. This is faster and more cost-effective.Example using Google's Gemini via the AI SDK:
import { google } from "@ai-sdk/google";
export const factualityModel = google(
"gemini-1.5-flash",
);XML tags are used to provide clear delimiters within prompts, helping the LLM distinguish between different types of information or instructions.
Use tags to wrap data, instructions, or examples to prevent the model from confusing them. For example, wrapping spreadsheet data in <data> tags and instructions in <instructions> tags.
Instruct the LLM to wrap specific parts of its response in XML tags. This allows you to programmatically parse multiple distinct outputs (like a <summary>, <critique>, and <suggested-improvements>) from a single response.
Review the article below and provide a summary, critique, and suggested improvements.
Wrap the summary in <summary> tags.
Wrap the critique in <critique> tags.
Wrap the suggested improvements in <suggested-improvements> tags.
<article>
{{ARTICLE}}
</article>When deploying to serverless platforms (like Next.js on Vercel), in-memory tracking (e.g., using a Map) fails because multiple Node.js processes are spun up. Use a centralized Redis instance to sync rate limits across all processes.
This is achieved by using a Redis key based on the current time window (e.g., prefix:windowStart) and utilizing Redis pipelines to atomically increment the counter (INCR) and set an expiration (EXPIRE) for the window.
To implement rate limiting, define a configuration object that specifies the maximum number of requests allowed within a specific time window (in milliseconds).
interface RateLimitConfig {
// Maximum number of requests
maxRequests: number;
// Time window in milliseconds
windowMs: number;
keyPrefix?: string;
// Maximum number of retries before failing
maxRetries?: number;
}When building LLM applications, you need observability to monitor long conversational threads, tool calls, and database queries. This is critical for both local development and production to manage costs (LLM usage) and user experience.
Key considerations for choosing a platform:
While both involve LLMs invoking functions, they differ in architecture and decoupling:
MCP Architecture Flow: