Databricks Apps Cookbook

repository·main·Indexed 41 days ago

https://github.com/databricks-solutions/databricks-apps-cookbook

A collection of recipes and sample applications for building data and AI apps on the Databricks Apps runtime. Includes multi-framework support for FastAPI, Dash, Streamlit, and Reflex, with demonstrations for integrating Databricks SQL Warehouses, Lakebase PostgreSQL, Unity Catalog, AI/ML model serving, Vector Search, and Databricks Workflows.

Tokens
66.6K
Snippets
164
Records
263
Agent score
52%

What's inside Databricks Apps Cookbook

  1. Overview of Databricks Apps Cookbook - Dash App

    main

    The Dash App is an interactive cookbook demonstrating common Databricks integration patterns. It uses a sidebar-navigated, multi-page layout where each recipe page provides three distinct views:

    • Try it: An interactive demo to test the integration.
    • Code snippet: An implementation reference for developers.
    • Requirements: A list of necessary permissions and resources.

    Navigation is categorized into sections such as Tables, Volumes, AI/ML, Business Intelligence, Workflows, Compute, Authentication, and External Services.

  2. Overview of the Reflex App cookbook structure

    main

    The Reflex App is an interactive cookbook designed to demonstrate Databricks integration patterns. Each 'recipe' in the application is organized into a consistent three-tab interface to facilitate learning and implementation:

    1. Try It: An interactive demo where you can execute the pattern.
    2. Code Snippet: The implementation reference containing the actual code used.
    3. Requirements/Details: Documentation regarding the specific permissions and Databricks resources required to run the recipe.

    The application uses a sidebar-navigated layout with a collapsible menu.

  3. Overview of Databricks Apps Cookbook

    main

    The Databricks Apps Cookbook provides ready-to-use code snippets for building interactive data and AI applications using Databricks Apps.

    Common use cases covered include:

    • Reading from and writing to tables and volumes.
    • Invoking traditional ML models and GenAI.
    • Triggering workflows.

    Each snippet includes the source code, required permissions, and a list of dependencies. While the cookbook provides specific recipes for several frameworks, they can be adapted to any Python framework.

  4. Overview of the Databricks Apps Cookbook

    main

    The Databricks Apps Cookbook provides ready-to-use code snippets and recipes for building data and AI applications using Databricks Apps.

    Key features include:

    • 10+ recipes for common use cases like reading/writing to Delta tables and volumes, invoking ML/GenAI models, and triggering workflows.
    • Multi-framework support: Snippets are available for Dash, Streamlit, Reflex, and FastAPI.
    • Deployment flexibility: Recipes can be deployed directly to Databricks Apps or run locally.
    • Detailed requirements: Each recipe includes descriptions of necessary permissions, resources, and dependencies.

    For interactive exploration and code copying, visit apps-cookbook.dev.

  5. Understand the Streamlit App layout and navigation

    main

    The Databricks Apps Cookbook Streamlit application is a multi-page interactive demo designed to showcase Databricks integration patterns.

    Navigation Structure:

    • Sidebar (Left): Contains grouped navigation sections organized into over 10 categories (e.g., Tables, Volumes, AI/ML, Unity Catalog).
    • Main Content Area: Displays the selected recipe page.

    Recipe Page Layout: Each recipe page is organized into three distinct tabs to facilitate learning and implementation:

    1. Try it: An interactive demo where you can execute the pattern.
    2. Code snippet: An implementation reference containing the source code for the pattern.
    3. Requirements: A list of the specific permissions and Databricks resources needed to run the recipe.
  6. How the FastAPI App architecture works

    main

    The application uses a dual-database architecture to separate analytical and transactional workloads:

    1. Databricks SQL Warehouse: Used for analytical queries against Unity Catalog tables.
    2. Lakebase PostgreSQL Database: Used for transactional CRUD operations on orders data.

    Conditional Endpoint Registration: Endpoints related to the Lakebase database (e.g., /api/v1/orders/*) are only registered if a Lakebase instance is detected. If resources haven't been provisioned, the app runs in a 'limited mode' providing only the health check and Unity Catalog table query endpoints.

    Background Processes:

    • OAuth Token Refresh: Automatically refreshes tokens for persistent database connections every 50 minutes.
    • Health Checks: Runs background health checks every 300 seconds.
    • Performance Tracking: Tracks request performance via response headers.
  7. Configure on-behalf-of-user authentication for Databricks Apps

    main

    To access the X-Forwarded-Access-Token header and use the current_user.me() API to retrieve detailed user information (such as User ID, Display Name, and Groups), you must enable on-behalf-of-user authentication for your Databricks App.

    Without this configuration, you will only have access to basic identity headers, and any SDK calls using the app's default identity will return the app's service principal information rather than the active user's information.

  8. Understand the dual database architecture

    main

    The application utilizes two distinct data layers depending on the workload:

    1. Databricks SQL Warehouse: Handles Unity Catalog table queries and analytics workloads via the /api/v1/table endpoint.
    2. Lakebase PostgreSQL Database: Handles transactional operations and orders management via the /api/v1/orders/* endpoints.

    Note: The Lakebase PostgreSQL database includes automatic token refresh for Databricks database instances using OAuth authentication.

  9. Use On-Behalf-Of (OBO) User Authentication

    main

    The app demonstrates the difference between running queries as the application's service principal versus the logged-in user:

    1. Select the required data hierarchy (warehouse, catalog, schema, table).
    2. Choose an authentication mode:
      • "On-behalf-of-user (OBO)": Uses the logged-in user's token (respects user permissions).
      • "Service principal": Uses the app's own credentials.
    3. Click "Run query" to execute a SELECT * FROM table LIMIT 10 using the selected identity.
  10. Explore the Databricks Apps Cookbook recipe categories

    main

    The application provides recipes organized into the following functional categories:

    • Introduction
    • Tables: Read a Lakebase table, Read a Databricks table, Edit a Databricks table
    • Volumes: Upload, Download
    • AI/ML: Invoke Model, Multi-Modal LLM, Vector Search, MCP Connect
    • Business Intelligence: AI/BI Dashboard, Genie
    • Workflows: Trigger Job, Retrieve Results
    • Compute: Connect to Cluster, Connect to Serverless
    • Unity Catalog: List Catalogs & Schemas
    • Authentication: Get Current User, On-Behalf-Of User
    • Visualizations: Charts, Maps
    • External Services: External Connections, Retrieve Secret
  11. Configure on-behalf-of-user authentication for user identity

    main

    To access the x_forwarded_access_token header and use the current_user.me() API to fetch the identity of the person interacting with your app, you must configure your Databricks App with on-behalf-of-user authentication.

    Without this configuration:

    1. The x_forwarded_access_token header will not be available.
    2. Any Databricks SDK calls using the available credentials will act as the app's service principal rather than the end-user.