gollama

repository·main·Indexed 23 days ago

https://github.com/sammcj/gollama

A terminal UI application and CLI tool for managing and interacting with Ollama models, built using the Bubble Tea framework. It provides features for listing, searching, pulling, and deleting models, as well as editing Modelfiles, inspecting model metadata, and estimating VRAM usage for different quantizations.

Tokens
5.9K
Snippets
14
Records
30
Agent score
83%

What's inside gollama

  1. Delete selected models

    main

    To delete models:

    1. Use the Space key to toggle selection for one or multiple models in the list.
    2. Press the Delete key.
    3. A confirmation view will appear showing the names of all selected models.
    4. Press the ConfirmYes key to execute the deletion or ConfirmNo to cancel.
  2. Understand the AppModel lifecycle and views

    main

    The AppModel is the central state manager for the Gollama TUI, implementing the tea.Model interface from Bubble Tea. It manages several distinct views to control what the user sees and how they interact with the application:

    • MainView: The default list view of available Ollama models.
    • TopView: A view showing currently running models, their VRAM usage, and expiration times.
    • HelpView: An expanded view displaying all available keybindings.
    • ExternalEditorView: A mode used when editing a Modelfile in an external editor.
    • Filtering: An active state within the list where the user is typing to narrow down models.
    • Inspecting: A view that displays detailed metadata for a specific model in a table format.
    • ConfirmDeletion: A state requiring user confirmation before deleting selected models.
  3. Manage model views and navigation

    main

    The AppModel uses a view field to switch between different UI states. Navigation is primarily handled via q or esc keys, which behave contextually:

    • In TopView, HelpView, or ExternalEditorView: Pressing q or esc returns the user to the MainView.
    • In MainView (with no active filter): Pressing q or esc quits the application.
    • When a filter is applied: Pressing q or esc clears the filter and returns to the unfiltered list.
    • In ExternalEditorView: Pressing s saves changes, while other keys return to the main view.
  4. Inspect model metadata

    main

    When a model is selected and the inspection mode is triggered, the application displays a detailed table of the model's properties. This includes basic info (Name, ID, Size) and enhanced metadata retrieved from the Ollama API, such as:

    • Parameter Size
    • Quantization Level
    • Format
    • Family
    • Context Length
    • Embedding Length
    • Vocab Size
    • Capabilities (e.g., vision, etc.)

    To return to the main list, press q or esc.

  5. Pull new models

    main

    Gollama supports two ways to pull models from Ollama:

    1. Pull an existing model: Select a model from the list and use the PullModel keybinding. This shows a progress bar.
    2. Pull a new model by name: Use the PullNewModel keybinding. This opens a text input where you can type the model name (e.g., llama3:8b-instruct).

    Note: There is a known issue where the progress bar might require a keypress (like an arrow key) to refresh visually.

  6. Edit Modelfiles via internal or external editors

    main

    You can modify a model's configuration using the EditModel keybinding. The behavior depends on your system configuration:

    • Internal Editor: Uses a terminal-based editor (like Vim) directly within the TUI.
    • External Editor: If an external editor is configured, the application creates a temporary file and opens it in your system editor.
      • While in ExternalEditorView, press s to save changes and return to the main view.
      • Pressing any other key will discard changes and return to the main view.
  7. Implement a progress bar using progressModel

    main

    The progressModel struct wraps a github.com/charmbracelet/bubbles/progress model to integrate it into a Bubble Tea application. It handles window resizing to maintain a maximum width and manages periodic updates via a tickMsg.

    Key behaviors:

    • Resizing: Responds to tea.WindowSizeMsg by adjusting the progress bar width, capped at a maxWidth of 80 characters.
    • Updating: Uses m.progress.IncrPercent(float64) to increment progress. You can also use m.progress.SetPercent(float64) to set an explicit value.
    • Animation: Must handle progress.FrameMsg in the Update loop to allow the progress bar to animate its internal state.
    • Termination: The model automatically quits when m.progress.Percent() == 1.0 or when a key is pressed.
    func (m progressModel) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
    	switch msg := msg.(type) {
    	case tea.KeyMsg:
    		return m, tea.Quit
    
    	case tea.WindowSizeMsg:
    		m.progress.Width = msg.Width - padding*2 - 4
    		if m.progress.Width > maxWidth {
    			m.progress.Width = maxWidth
    		}
    		return m, nil
    
    	case tickMsg:
    		if m.progress.Percent() == 1.0 {
    			return m, tea.Quit
    		}
    		cmd := m.progress.IncrPercent(0.25)
    		return m, tea.Batch(tickCmd(), cmd)
    
    	case progress.FrameMsg:
    		progressModel, cmd := m.progress.Update(msg)
    		m.progress = progressModel.(progress.Model)
    		return m, cmd
    
    	default:
    		return m, nil
    	}
    }
  8. View running models

    main

    The TopView provides a real-time look at which models are currently loaded in Ollama's memory. It displays a table with:

    • Name: The model name.
    • Size (GB): Memory footprint.
    • VRAM (GB): VRAM usage.
    • Until: Estimated time until unloading (if applicable).

    This view can be toggled using the Top keybinding and refreshes automatically every 2 seconds.

  9. Sort the model list

    main

    The application provides several ways to sort the list of models. These are triggered by specific keybindings (defined in the KeyMap):

    • Sort by Name: SortByName
    • Sort by Size: SortBySize (largest first)
    • Sort by Modified Date: SortByModified (newest first)
    • Sort by Quantization: SortByQuant
    • Sort by Family: SortByFamily
    • Sort by Parameter Size: SortByParamSize (largest first, handles 'B' suffix parsing)
  10. Use the gollama CLI to manage Ollama models

    main

    gollama is a CLI tool for managing Ollama models. It provides a TUI (Terminal User Interface) for interactive management, as well as various flags for specific tasks like listing models, searching, unloading models, and estimating VRAM usage.

    Common CLI Flags

    FlagDescription
    -lList all available Ollama models and exit
    -s <term>Search for models containing the search term in their name
    -uUnload all currently running models and exit
    -e <model>Edit a specific model's modelfile
    -vPrint the version and exit
    -HShortcut to connect to http://localhost:11434
    --ollama-dir <dir>Specify a custom Ollama models directory
    --host <url>Override the configured Ollama API host (e.g., http://localhost:11434)
    --log-level <level>Override log level (debug, info, warn, error)
    --log <level>Alias for --log-level

    VRAM Estimation Flags

    Use these flags to estimate how much VRAM different quantizations of a model will consume.

    FlagDescription
    --vram <model>The model to estimate VRAM usage for (e.g., qwen2:q4_0 or meta-llama/Llama-2-7b)
    --fits <GB>Highlight quant sizes and context sizes that fit in this amount of vRAM (in GB)
    --context <size>Maximum context length (e.g., 32k or 128k)
    --quant <level>Specific quantization level (e.g., Q4_0, Q5_K_M)
    --vram-to-nth <size>Top context length to search for (e.g., 65536, 32k, 2m)

    Examples

    List all models:

    gollama -l

    Search for a model:

    gollama -s llama

    Estimate VRAM for a model:

    gollama --vram qwen2:q4_0 --fits 8

    Unload all running models:

    gollama -u
  11. Run a progress bar demo

    main

    To run a progress bar that updates automatically, initialize a progress.Model (e.g., using progress.WithDefaultGradient()), wrap it in your progressModel, and start the Bubble Tea program. Use tea.Tick to send periodic messages to your Update function to drive the progress increments.

    func progressDemo() {
    	tickCmd := tea.Tick(time.Second*1, func(t time.Time) tea.Msg {
    		return tickMsg(t)
    	})
    
    	p := progress.New(progress.WithDefaultGradient())
    	m := progressModel{progress: p}
    
    	t := tea.NewProgram(m)
    
    	if err := func() error {
    		_, err := t.Run()
    		return err
    	}(); err != nil {
    		panic(err)
    	}
    
    	tickCmd()
    	t.Kill()
    }
  12. Initialize a new KeyMap with default bindings

    main
    Use NewKeyMap() to create a pointer to a KeyMap struct containing the default keyboard shortcuts for the application. These bindings use the github.com/charmbracelet/bubbles/key package and include mappings for model management (push, pull, edit), sorting, and navigation.