DOCSEARCHbyAlgolia
Submit Docs

Palico Documentation

website·Indexed 31 days ago

https://docs.palico.ai

Documentation for building AI agents with Palico, featuring guides on tool integration, long-term memory, streaming, and prompt management. Includes a client SDK, React SDK, and API reference for managing conversations, along with integrations for LangChain, LlamaIndex, and various vector databases and model providers.

Tokens
11.6K
Snippets
64
Records
82
Agent score
53%

What's inside Palico

  1. Overview of AI Gateways

    An AI Gateway is a service that allows you to interact with different LLM models through a single endpoint. Examples of AI Gateways include Portkey, GCP Vertex, and AWS Bedrock.

  2. Integrate LangChain components for RAG and Prompting in Palico

    When building LLM applications with Palico, LangChain can be used for specific implementation utilities while Palico manages the overall architecture (REST API, logging, tracing, and experiments). Common LangChain integrations include:

    • RAG Applications: Using LangChain tools to index and retrieve documents from vector databases like Pinecone or PGVector.
    • Prompt Templates: Using LangChain's templating tools to manage complex prompt inputs.
    • Unified Model Interface: Using LangChain to switch between different LLM providers such as OpenAI and Claude.

  3. Implement LLM Agents in Palico

    Agents are the core building blocks of a Palico App, encapsulating prompt logic and LLM model calls. To maintain systematic accuracy improvements, each agent should contain only a single LLM model call. For applications requiring multiple models, create multiple agents and orchestrate them using workflows.

  4. Use Palico Studio for prototyping and analysis

    Palico Studio is a local web application that serves as the control center for a Palico App. It provides the following tools:

    • Chat: A chat interface for prototyping and interacting with Agents or Workflows.
    • Quick Lab: A side-by-side comparison tool for Agents or Workflows across multiple use cases.
    • Experiments: A management interface to run and analyze LLM experiments.
    • Traces: A viewer for runtime traces (implemented via OpenTelemetry) to debug events, errors, and performance.

  5. Manage long-term conversation memory

    Palico provides getConversationState and setConversationState to persist and restore conversation state across multiple turns without managing the underlying storage infrastructure. Use the isNewConversation flag to determine whether to initialize a new state or restore an existing one using the conversationId.
    import { Chat, getConversationState, setConversationState } from "@palico-ai/app";

const handler: Chat = async ({ conversationId, isNewConversation, userMessage }) => {
  let state: { messages: OpenAIMessage[] };

  if (isNewConversation) {
    state = { messages: [] };
  } else {
    state = await getConversationState(conversationId);
  }

  state.messages.push({ role: "user", content: userMessage });
  const response = await openai.chat.completions.create({
    model: "gpt-3.5-turbo-0125",
    messages: state.messages,
  });
  
  const responseMessage = response.choices[0].message.content;
  state.messages.push({ role: "agent", content: responseMessage });

  await setConversationState(conversationId, state);
  return { message: responseMessage };
};

  7. Implement streaming responses in Chat handlers

    To stream responses to the client, use the stream.push() method provided in the ChatRequest object. This allows you to send messages, intermediate steps, or other data chunks back to the user in real-time.
    const handler: Chat = async ({ userMessage, stream }) => {
  const response = await openai.chat.completions.create({
    model: 'gpt-3.5-turbo-0125',
    stream: true,
    messages: [{ role: 'user', content: userMessage }],
  });
  for await (const chunk of response) {
    if (chunk.choices[0].delta.content) {
      stream.push({
        message: chunk.choices[0].delta.content,
        data: { /* additional data */ },
        intermediateSteps: [ /* intermediate steps */ ],
      });
    }
  }
};

  8. Dynamically swap LLM models using Portkey and App Config

    By combining Portkey with App Config, you can swap LLM models and providers (e.g., for A/B testing or experiments) without changing your source code. Pass the model and provider values from your application configuration directly into the Portkey client and request.
    import { Portkey } from "portkey";

interface AppConfig {
  model: string;
  provider: string;
}

const handler: Chat = async ({ userMessage, appConfig }) => {
  const { model, provider } = appConfig;
  
  const portkey = new Portkey({
    Authorization: "Bearer sk-xxxxx",
    provider: provider,
    // ...additional authorization params
  });

  const response = await portkey.chat.completions.create({
    messages: [{ role: "user", content: userMessage }],
    model: model,
  });

  return {
    message: response.choices[0].message.content,
  };
};

  9. Setup a server-side authorization proxy in NextJS

    To securely communicate with Palico agents, create a server-side API route in NextJS. This prevents exposing the PALICO_SERVICE_KEY to the client. The route should verify the user, initialize the @palico-ai/client-js client, and stream the agent's chat response.
    import { createClient } from "@palico-ai/client-js";

export const maxDuration = 120;

export async function POST(req: Request) {
  await verifyUser(); // replace with your authorization logic
  const agentAPIURL = process.env.PALICO_AGENT_API_URL;
  const serviceKey = process.env.PALICO_SERVICE_KEY;
  if (!agentAPIURL || !serviceKey) {
    throw new Error("Missing Palico environment variables");
  }
  const request = await req.json();
  const client = createClient({
    apiURL: agentAPIURL,
    serviceKey: serviceKey,
  });
  const responseStream = await client.agent.chat({
    ...request,
    stream: true,
  });
  return responseStream.response;
}

  10. Integrate Anthropic models with Palico

    Use the @anthropic-ai/sdk package to call Anthropic models. The implementation requires an ANTHROPIC_API_KEY environment variable.
    npm install @anthropic-ai/sdk
    import { AnthropicClient } from "@anthropic-ai/sdk";

const chat: Chat = async ({ userMessage }) => {
  const client = new AnthropicClient({
    apiKey: process.env.ANTHROPIC_API_KEY,
  });
  const response = await client.messages.create({
    messages: [{ role: "user", content: userMessage }],
    model: "claude-3-opus-20240229",
    max_tokens: 1000,
  });
  return {
    message: response.content,
  };
};

  11. Deploy Palico app using Coolify

    Palico apps can be hosted on any server supporting Docker. Coolify is recommended for managing hosting, CI/CD, and Pull Request Previews. Note that an HTTPS domain is typically required to connect to the hosted app from Palico Studio or other clients.

  12. Handle client-side tool execution

    The useChat hook provides pendingToolCalls to identify tools the agent wants to execute on the client and addResult to send the execution results back to the agent.
    import { useChat } from "@palico-ai/react";

const Assistant: React.FC = () => {
  const { addResult, pendingToolCalls } = useChat({
    agentName: "assistant",
    apiURL: "/api/palico",
  });
  const { showDialog } = useDialog();

  if(pendingToolCalls.length > 0) {
    const toolCall = pendingToolCalls[0];
    if(toolCall.name === "askForConfirmation") {
      showDialog({
        title: "Confirmation",
        message: toolCall.parameters.message,
        onConfirm: () => {
          addResult(toolCall, { confirmed: true });
        },
        onCancel: () => {
          addResult(toolCall, { confirmed: false });
        },
      });
    }
  }

  return (<div>{/* UI */}</div>);
};