Amazon Nova Act Python SDK

repository·main·Indexed 21 days ago

https://github.com/aws/nova-act

A Python SDK for Amazon Nova Act, a service for building and managing fleets of AI agents that automate production UI workflows in a browser. The SDK supports script, interactive, and async modes, and provides features for human-in-the-loop escalation, structured data extraction via act_get(), custom tool integration using the @tool decorator, and MCP tool support via Strands MCP Client.

Tokens
93.4K
Snippets
279
Records
397
Agent score
74%

What's inside nova-act

  1. Overview of Browser Commands in Nova Act

    main

    The act browser CLI provides interactive browser automation. It allows users to execute browser actions using either natural language prompts or specific commands. The command suite is organized into four main functional groups:

    1. Browsing: Commands for navigation and interaction (e.g., ask, click_target, fill_form, goto, type_text).
    2. Extraction: Commands for retrieving data and page state (e.g., extract, get_content, screenshot, query).
    3. Session: Subcommands for managing browser sessions (e.g., create, list, close, export).
    4. Setup: Diagnostic and configuration commands (e.g., doctor, setup).

    For complete command usage, flags, and examples, run act browser --help in your terminal.

    act browser --help
  2. What the Nova Act CLI does

    main

    The Nova Act CLI is a Python command-line tool designed to deploy Python workflows to the AWS AgentCore Runtime. It automates the lifecycle of a workflow, including containerization, AWS service integration, and management.

    Core Capabilities:

    • Deployment: Deploy Python scripts to AWS AgentCore with automatic containerization.
    • Image Management: Manage ECR (Elastic Container Registry) repositories and Docker images.
    • State Tracking: Track workflow state across multiple AWS regions and accounts.
    • Execution: Run workflows and stream logs in real-time.
    • Lifecycle Management: Handle the creation, deployment, execution, and deletion of workflows.
  3. Implement State Guardrails

    main

    State guardrails allow you to control which URLs the agent can visit. You provide a callback function that receives a GuardrailInputState and returns a GuardrailDecision. If the callback returns GuardrailDecision.BLOCK, the act() method will raise an ActStateGuardrailError.

    This is useful for preventing the agent from navigating to unauthorized domains or sensitive pages.

    from nova_act import NovaAct, GuardrailDecision, GuardrailInputState
    from urllib.parse import urlparse
    import fnmatch
    
    def url_guardrail(state: GuardrailInputState) -> GuardrailDecision:
        hostname = urlparse(state.browser_url).hostname
        if not hostname:
            return GuardrailDecision.BLOCK
    
        # Example URL block-list
        blocked = ["*.blocked-domain.com", "*.another-blocked-domain.com"]
        if any(fnmatch.fnmatch(hostname, pattern) for pattern in blocked):
            return GuardrailDecision.BLOCK
    
        # Example URL allow-list
        allowed = ["allowed-domain.com", "*.another-allowed-domain.com"]
        if any(fnmatch.fnmatch(hostname, pattern) for pattern in allowed):
            return GuardrailDecision.PASS
    
        return GuardrailDecision.BLOCK
    
    with NovaAct(starting_page="https://allowed-domain.com", state_guardrail=url_guardrail) as nova:
        # This will be blocked if agent tries to visit a blocklisted domain
        nova.act("Navigate to the homepage")
  4. Security Best Practices for Browser Automation

    main

    Because this tool controls a real browser with access to your authenticated state (cookies, local storage, etc.), follow these security guidelines:

    • API Keys: Never commit keys to source control. Use act browser setup to store them in ~/.act_cli/browser/config.yaml (permissions 0o600).
    • JavaScript Execution: The evaluate command can read cookies and modify the DOM. Never run untrusted JavaScript.
    • Sensitive Data in Logs: network-log and console-log capture data in memory. While network-log hides Authorization and Cookie headers from output, they remain in process memory. Use export with caution as it bundles sensitive content.
    • HTTPS Security: By default, --ignore-https-errors is enabled. For security-sensitive workflows, use --no-ignore-https-errors to enforce strict certificate validation.
    • Chrome Arguments: Avoid dangerous flags like --disable-web-security or --no-sandbox unless you fully understand the security implications.
  5. Enable Human-in-the-loop (HITL) for task clarification

    main
    Nova Act can be configured to use Human-in-the-loop (HITL) capabilities. This allows the model to ask the user for clarification or confirmation when it encounters uncertain tasks during a workflow.
  6. How to handle multi-threading in Nova Act workflows

    main

    The Workflow class is compatible with multi-threading, but because the @workflow decorator relies on ContextVars (which are thread-specific), you must explicitly propagate the context to new threads.

    There are two ways to handle this:

    1. Using copy_context(): Capture the current context and use ctx.run() to execute the helper function in the new thread.
    2. Manual Injection: Use get_current_workflow() to retrieve the active workflow and pass it directly to the helper function, which then passes it to the NovaAct constructor.

    Note: Multi-processing is currently not supported because the Workflow object contains non-pickleable boto3 Session and Client instances.

    from contextvars import copy_context
    from threading import Thread
    from nova_act import NovaAct, workflow, get_current_workflow
    
    # Option 1: Using copy_context (for decorated functions)
    @workflow(workflow_definition_name="my-workflow", model_id="nova-act-latest")
    def multi_threaded_workflow():
        ctx = copy_context()
        t = Thread(target=ctx.run, args=(multi_threaded_helper,))
        t.start()
        t.join()
    
    # Option 2: Manual injection (using get_current_workflow)
    def multi_threaded_helper(workflow):
        with NovaAct(..., workflow=workflow) as nova:
           pass
    
    @workflow(workflow_definition_name="my-workflow", model_id="nova-act-latest")
    def multi_threaded_workflow_manual():
        t = Thread(target=multi_threaded_helper, args=(get_current_workflow(),))
        t.start()
        t.join()
  7. Nova Act CLI codebase structure and organization

    main

    The CLI is organized into several functional layers:

    • cli.py: The main entry point for the CLI.
    • core/: Contains foundational infrastructure including configuration utilities, AWS identity resolution, logging, region management, and state management. It also houses specialized AWS service clients for agentcore, ecr, iam, nova_act, and s3.
    • workflow/: The primary management system for workflows.
      • commands/: Contains the implementation of CLI commands such as create, delete, deploy, list, run, show, and update.
      • services/: Manages external service integrations, specifically orchestration for AgentCore deployment, IAM role management, and image building.
      • utils/: Provides helper utilities for ARN validation, S3 bucket management, AWS Console deep links, Docker building, and CloudWatch log tailing.
  8. Nova Act CLI Command Groups

    main

    The Nova Act CLI is organized into two primary command groups:

    1. main: The top-level entry point for the CLI.
    2. workflow: A command group dedicated to managing the lifecycle of workflows.

    Commands are implemented using the click library and utilize a StyledGroup class to provide formatted help output.

    @click.group(cls=StyledGroup)
    def workflow() -> None:
        # Main workflow command group
    
    @click.group(cls=StyledGroup)
    @click.version_option(version=VERSION)
    def main() -> None:
        # Nova Act CLI main entry point
  9. Implement Human-in-the-loop (HITL) patterns

    main

    Nova Act supports Human-in-the-loop (HITL) to allow human supervision within autonomous workflows. You implement this by extending the HumanInputCallbacksBase class and providing implementations for the required abstract methods. Pass an instance of your class to the human_input_callbacks argument in the NovaAct constructor.

    Supported Patterns:

    • Human approval: Triggered via the approve method. Used for asynchronous decisions like Approve/Reject or Yes/No. The system captures a screenshot for the reviewer.
    • UI takeover: Triggered via the ui_takeover method. Enables real-time human control of the remote browser session (e.g., solving CAPTCHAs) via a live-streaming interface.
    from nova_act import NovaAct
    from nova_act.tools.human.interface.human_input_callback import (
        ApprovalResponse, HumanInputCallbacksBase, UiTakeoverResponse,
    )
    
    class MyHumanInputCallbacks(HumanInputCallbacksBase):
        def approve(self, message: str) -> ApprovalResponse:
            # Implement logic to handle approval
            ...
    
        def ui_takeover(self, message: str) -> UiTakeoverResponse:
            # Implement logic to handle UI takeover
            ...
    
    with NovaAct(
        starting_page="https://example.com",
        tty=False,
        human_input_callbacks=MyHumanInputCallbacks(),
    ) as nova:
        # Your workflow here
        pass
  10. How Browser Commands and Services are Architected

    main

    The browser CLI follows a layered architecture designed to separate command wiring from business logic:

    • Commands Layer: A thin layer using Click decorators. Commands do not manage browser state directly; instead, they use the command_session() context manager to access services and delegate work to BrowserActions methods.
    • Services Layer: Contains the core business logic.
      • BrowserActions: A mixin-based class decomposed into domains like exploration, inspection, interaction, and navigation.
      • SessionManager: Orchestrates the lifecycle of a browser session.
    • Utils Layer: Provides shared utilities. The command_session() context manager is the primary way to handle session preparation, NovaAct instantiation, and cleanup.

    This separation ensures that the business logic remains independent of the CLI framework.

  11. Understand the Nova Act deployment workflow

    main

    When you run a deployment command (e.g., act workflow deploy), the CLI orchestrates several steps to move your local Python code into an AWS runtime environment:

    1. Validation: AgentCoreSourceValidator checks your source directory and entry point.
    2. IAM Setup: AgentCoreIAMRoleManager ensures the necessary execution roles are configured.
    3. Image Building: BuildContextPreparer prepares the build context for a container image.
    4. Image Storage: ECRClient pushes the container image to an Amazon ECR repository.
    5. Runtime Creation: AgentCoreDeploymentService creates the AgentCore runtime environment.
    6. State Persistence: StateManager records the deployment state in a local JSON file.
    7. Artifact Storage: BucketManager ensures S3 buckets exist for any required artifacts.
  12. Known limitations of Nova Act

    main

    When building automations, be aware of the following constraints:

    • Application Scope: act() cannot interact with non-browser applications.
    • Browser Modals: act() cannot interact with the browser window itself (e.g., location access permission modals). These must be manually acknowledged if required.
    • Screen Size: Nova Act is optimized for resolutions between 864×1296 and 1536×2304. Performance may degrade outside this range. You can adjust dimensions using screen_width and screen_height parameters.
    • Prompt Injection: Nova Act may encounter unauthorized commands (prompt injections) in third-party website content, which could cause the model to ignore instructions or perform unauthorized actions.