Tensorlake SDK

It is important to distinguish between the Git-based workflow and the Filesystem-based workflow:

tl fs Specifics

• Snapshots: Use tl fs snapshot [PATH] -m "message" to record a permanent generation. Without -m, it is an ephemeral autosave.
• History: Use tl fs history [FILESYSTEM|PATH] to view permanent snapshots and the ephemeral WAL. The output separates these into snapshots and autosaves arrays.
• Status: tl fs status [PATH] reports local changes, last autosave time, and permanent snapshot count.

You can extract specific data from a document into a structured format by defining a Pydantic model and passing it to the DocumentAI.parse method via StructuredExtractionOptions.

1. Define a BaseModel representing your desired schema.
2. Initialize DocumentAI.
3. Create StructuredExtractionOptions using your schema.
4. Call doc_ai.parse() with the file path and options.
5. Use doc_ai.wait_for_completion(parse_id) to retrieve the results.

from tensorlake.documentai import DocumentAI
from tensorlake.documentai.models import ParsingOptions, StructuredExtractionOptions, ChunkingStrategy
from pydantic import BaseModel, Field
from typing import List, Optional

# 1. Define Schema
class ResearchPaperSchema(BaseModel):
    title: str = Field(description="Title of the research paper")
    authors: List[str] = Field(description="List of author names")
    abstract: str = Field(description="Abstract of the paper")

# 2. Initialize Client
doc_ai = DocumentAI()
file_path = "https://example.com/paper.pdf"

# 3. Configure Options
parsing_options = ParsingOptions(chunking_strategy=ChunkingStrategy.PAGE)
structured_extraction_options = StructuredExtractionOptions(
    schema_name="Research Paper Analysis", 
    json_schema=ResearchPaperSchema
)

# 4. Parse
parse_id = doc_ai.parse(
    file_path,
    parsing_options=parsing_options,
    structured_extraction_options=[structured_extraction_options],
)

# 5. Wait for results
result = doc_ai.wait_for_completion(parse_id)

When using tl git mount, edits are first stored in a crash-safe local journal and continuously uploaded as ordered, idempotent checkpoints in a workspace WAL. These are not repository commits. To manage these changes, use the following workflow:

Core Commands

• tl git status [/code]: Check the status of the mount.
• tl git snapshot [/code] --message "<msg>": Flushes the current WAL state and materializes it as an immutable commit on the workspace's private line.
• tl git sync [/code] [BRANCH|TAG|FULL_COMMIT]: Refreshes the current source. If a target is provided, it switches the view or workspace. If the workspace has snapshots, use rebase instead to change the base.
• tl git rebase [/code] BRANCH|TAG|FULL_COMMIT: Runs server-side to replay materialized snapshots and the unsnapshotted WAL tail onto a new base. Conflicts are shown as diff3 markers.
• tl git promote [/code] BRANCH [--merge]: The command to land changes. It autosaves edits, creates a snapshot, and lands it onto a real branch. By default, it performs a squash landing. Use --merge for a true merge.
• tl git log [/code|REPOSITORY] and tl git smartlog [/code|REPOSITORY] [--project]: View history and retained recovery chains.

The --publish Workspace

A --publish workspace continuously serves a fixed target branch.

• Snapshots: Explicit snapshots are reconciled directly onto the target branch.
• Restrictions: sync can refresh the target but cannot retarget it. rebase is rejected for --publish workspaces.
• Usage: Use a normal workspace if you need explicit rebase or retarget control.

tl git status [/code]
tl git snapshot [/code] --message "checkpoint"
tl git sync [/code] [BRANCH|TAG|FULL_COMMIT]
tl git rebase [/code] BRANCH|TAG|FULL_COMMIT
tl git promote [/code] BRANCH [--merge]
tl git log [/code|REPOSITORY]
tl git smartlog [/code|REPOSITORY] [--project]

After defining your application, deploy it using the tl deploy command. Once deployed, you can invoke the application via HTTP requests. You can retrieve results using the request_id or stream results using Server-Sent Events (SSE).

# 1. Set secrets
export TENSORLAKE_API_KEY="your-api-key"
tl secrets set OPENAI_API_KEY "your-openai-key"

# 2. Deploy
tl deploy examples/readme_example/city_guide.py

# 3. Invoke via HTTP
curl https://api.tensorlake.ai/applications/city_guide_app \
  -H "Authorization: Bearer $TENSORLAKE_API_KEY" \
  --json '"San Francisco"'

# 4. Get result using request_id
curl https://api.tensorlake.ai/applications/city_guide_app/requests/{request_id}/output \
  -H "Authorization: Bearer $TENSORLAKE_API_KEY"

# 5. Stream results with SSE
curl https://api.tensorlake.ai/applications/city_guide_app \
  -H "Authorization: Bearer $TENSORLAKE_API_KEY" \
  -H "Accept: text/event-stream" \
  --json '"San Francisco"'