DOCSEARCHbyAlgolia
Submit Docs

Flutter AI Toolkit

repository·main·Indexed 42 days ago

https://github.com/flutter/ai

A set of ready-to-use chat widgets for Flutter applications featuring a pluggable LLM provider architecture. It supports multi-turn chat interfaces with streaming responses, multimedia attachments, and voice notes, with built-in integration for Firebase Vertex AI using the LlmChatView and FirebaseProvider.

Tokens
6.3K
Snippets
18
Records
29
Agent score
54%

What's inside Flutter AI Toolkit

  1. Overview of FatIcons.ttf generation

    main

    The lib/fonts/FatIcons.ttf font file is a custom icon font generated using fluttericon.com. It combines specific SVG assets and selected Material Icons into a single font file to allow Cupertino apps to use these icons without requiring the full Material Icons dependency.

    Included SVG Assets

    • font_svg/spark-icon.svg (from The Noun Project)
    • font_svg/submit-icon.svg (a horizontally flipped version of the Google Symbols 'reply' icon)
    • font_svg/reply_24dp_5F6368_FILL0_wght400_GRAD0_opsz24.svg (from Google Symbols)

    Included Material Icons

    The following icons are pulled from the Material library and packaged into FatIcons.ttf:

    • Icons.add
    • Icons.attach_file
    • Icons.stop
    • Icons.mic
    • Icons.close
    • Icons.camera_alt
    • Icons.image
    • Icons.edit
    • Icons.copy

  2. How the AI Toolkit and LLM Providers work together

    main

    The AI Toolkit is built around an abstract LLM provider API. This architecture allows you to swap out different Large Language Model (LLM) providers easily.

    • LlmChatView: The primary widget used to display the chat interface.
    • Provider: An abstraction (like FirebaseProvider) that connects the LlmChatView to a specific LLM implementation.
    • Pluggable Support: You can implement the provider interface to plug in your own custom LLM.

  3. Customize iOS launch screen assets

    main

    To change the image displayed during the app's launch on iOS, you can either replace the existing image files directly in the example/ios/Runner/Assets.xcassets/LaunchImage.imageset/ directory or use Xcode for a more visual approach.

    Using Xcode:

    1. Open your Flutter project's iOS workspace by running open ios/Runner.xcworkspace in your terminal.
    2. In the Xcode Project Navigator, navigate to Runner/Assets.xcassets.
    3. Drag and drop your desired images into the asset catalog to replace the current launch images.
    open ios/Runner.xcworkspace

  4. Install the Flutter AI Toolkit

    main

    Add the following dependencies to your pubspec.yaml file to install the toolkit and its default Firebase AI integration:

    $ flutter pub add flutter_ai_toolkit firebase_ai firebase_core

  5. Configure device permissions for AI features

    main

    To support advanced features like voice input and media attachments, you must configure platform-specific permissions:

    Network Access

    • macOS: Add com.apple.security.network.client to your *.entitlements files.
    • Android: Add <uses-permission android:name="android.permission.INTERNET"/> to your AndroidManifest.xml.

    Voice and Media

    • Microphone: Follow the setup instructions for package:record to enable voice input.
    • File Selection: Follow the usage instructions for package:file_selector to enable file attachments.
    • Image Selection: Follow the installation instructions for package:image_picker to enable camera/gallery access.

  6. Implement a chat interface using Firebase AI

    main

    To use Firebase AI, first ensure Firebase is initialized in your main() function. Then, use LlmChatView and pass a FirebaseProvider configured with a model from FirebaseAI.vertexAI().

    Note: You must provide a model name (e.g., 'gemini-2.0-flash') that corresponds to available Firebase models.

    import 'package:firebase_core/firebase_core.dart';
import 'package:firebase_vertexai/firebase_ai.dart';
import 'package:flutter_ai_toolkit/flutter_ai_toolkit.dart';
import 'firebase_options.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await Firebase.initializeApp(options: DefaultFirebaseOptions.currentPlatform);
  runApp(const App());
}

class ChatPage extends StatelessWidget {
  const ChatPage({super.key});

  @override
  Widget build(BuildContext context) => Scaffold(
    appBar: AppBar(title: const Text('App Title')),
    body: LlmChatView(
      provider: FirebaseProvider(
        model: FirebaseAI.vertexAI().generativeModel(
          model: 'gemini-2.0-flash'
        ),
      ),
    ),
  );
}

  7. Augment messages with a custom messageSender

    main

    If you need to perform prompt engineering, implement Retrieval-Augmented Generation (RAG), or log user prompts before they reach the LLM, you can provide a custom messageSender to LlmChatView.

    When provided, LlmChatView will use this LlmStreamGenerator instead of the default sendMessageStream method on the LlmProvider. It is the developer's responsibility to ensure the provided sender returns a stream that properly represents the LLM's response.

  8. Use LlmChatView to display a chat interface

    main

    The LlmChatView widget is the primary UI component for building an AI chat interface. It provides a complete user experience including message history, a chat input area, and support for attachments and voice notes. It requires an LlmProvider to manage the underlying LLM interactions.

    Key configuration options:

    • provider: An LlmProvider instance (required).
    • style: An LlmChatViewStyle to customize colors and appearance.
    • suggestions: A list of strings to display as quick-action buttons when the chat is empty.
    • welcomeMessage: A message to show when the chat is first opened.
    • enableAttachments: Boolean to toggle file/image attachments (defaults to true).
    • enableVoiceNotes: Boolean to toggle voice recording (defaults to true).
    • messageSender: An optional LlmStreamGenerator used to intercept and augment messages (useful for RAG or prompt engineering).
    LlmChatView(
  provider: MyLlmProvider(),
  style: LlmChatViewStyle(
    backgroundColor: Colors.white,
    // ... other style properties
  ),
  suggestions: ['Hello!', 'Tell me a joke'],
  welcomeMessage: 'How can I help you today?',
)

  9. Use the ChatInput widget for user input

    main

    The ChatInput widget provides a complete chat interface for users to enter text, record audio for speech-to-text (STT), and attach media files.

    Key Features

    • Text Input: Standard text entry for messages.
    • Speech-to-Text: Support for recording audio and translating it into text.
    • Attachments: Support for adding and removing media attachments.
    • Message Editing: Support for pre-populating the input with an existing message to allow for non-destructive edits.

    Required Callbacks

    To use ChatInput, you must provide two primary callbacks:

    1. onSendMessage: Triggered when the user submits a text message. It provides the message text as a String and an Iterable<Attachment> for any attached files.
    2. onTranslateStt: Triggered when an audio recording is finished. It provides an XFile containing the audio and the current Iterable<Attachment>.

    Implementation Example

    ChatInput(
  onSendMessage: (text, attachments) {
    // Handle sending the message
    print('Sending: $text with ${attachments.length} attachments');
  },
  onTranslateStt: (file, attachments) {
    // Handle the audio file for speech-to-text
    print('Transcribing file: ${file.path}');
  },
  initialMessage: myExistingMessage, // Optional: for editing existing messages
  onCancelEdit: () {
    // Optional: revert to original message if user cancels edit
  },
)```

    ```dart
ChatInput(
  onSendMessage: (text, attachments) {
    // Handle sending the message
  },
  onTranslateStt: (file, attachments) {
    // Handle the audio file for speech-to-text
  },
)```

  10. Modify LlmMessageStyle using copyWith

    main

    To create a new LlmMessageStyle based on an existing one while only changing specific fields, use the copyWith method. This is useful for creating variations of a theme (e.g., changing only the color or the icon) without re-specifying every property.

    final baseStyle = LlmMessageStyle.defaultStyle();

// Create a new style that is identical to default but with a different icon color
final modifiedStyle = baseStyle.copyWith(
  iconColor: Colors.red,
);

  11. Resolve LlmMessageStyle with fallbacks

    main

    If you are providing a partial style and want to ensure all properties are populated, use LlmMessageStyle.resolve. This method merges a custom style with a default style, using the custom style's values where provided and falling back to the default for any missing properties.

    If no defaultStyle is explicitly passed to the resolve method, it defaults to LlmMessageStyle.defaultStyle().

    // Merges 'customStyle' with the project's default style
final resolvedStyle = LlmMessageStyle.resolve(customStyle);

// Merges 'customStyle' with a specific provided fallback style
final resolvedStyleWithCustomFallback = LlmMessageStyle.resolve(
  customStyle,
  defaultStyle: myAlternativeStyle,
);