BricksLLM Documentation

You can enforce usage limits, rate limits, and model access for specific users by leveraging the userId field in your API requests. This is useful for internal applications (access based on email) or SaaS applications (tier-based limits).

To implement this, follow these steps:

1. Create a provider setting: Register your LLM provider (e.g., openai) via /api/provider-settings and capture the returned id.
2. Create a Bricks API key: Create a key via /api/key-management/keys, associating it with the provider setting id using the settingIds array.
3. Create a User: Define a user via /api/users. You must provide a unique userId and matching tags that correspond to the API key created in step 2. In this step, you can configure:
   • costLimitInUsd: Total spend limit.
   • costLimitInUsdOverTime: Spend limit over a specific time unit.
   • costLimitInUsdUnit: Time unit for spend limits (e.g., m for minute).
   • rateLimitOverTime: Request rate limit.
   • rateLimitUnit: Time unit for rate limits (e.g., m for minute).
   • allowedPaths: An array of objects specifying allowed API path and method.
   • allowedModels: An array of model strings (e.g., ["gpt-4"]).
   • userId: Your custom identifier for the user.
   • tags: Must match the tags on the API key to link the user to that key.
4. Make Requests: When calling the Bricks gateway, include the Bricks API key in the Authorization header and the userId in the JSON request body.

If a user attempts to access a model or path not defined in their allowedModels or allowedPaths, the request will return a 401 error.

### Step 1 - Create a provider
```bash
curl -X PUT http://localhost:8001/api/provider-settings \
   -H "Content-Type: application/json" \
   -d '{
          "provider":"openai",
          "setting": {
             "apikey": "YOUR_OPENAI_API_KEY"
          }
      }'   

Step 2 - Create a Bricks API key

curl -X PUT http://localhost:8001/api/key-management/keys \
   -H "Content-Type: application/json" \
   -d '{
	      "name": "My Secret Key",
	      "key": "my-secret-key",
	      "tags": ["team-one"],
	      "settingIds": ["ID_FROM_STEP_ONE"]
      }'   

Step 3 - Create a User

curl -X POST http://localhost:8001/api/users \
   -H "Content-Type: application/json" \
   -d '{
            "name": "Spike Lu",
            "costLimitInUsd": 1,
            "costLimitInUsdOverTime": 0.002,
            "costLimitInUsdUnit": "m",
            "rateLimitOverTime": 5,
            "rateLimitUnit": "m",
            "allowedPaths": [
                {
                "path": "/api/providers/openai/v1/chat/completions",
                "method": "POST"
                }
            ],
            "allowedModels": ["gpt-4"],
            "userId": "my-user-id",
            "tags": ["team-one"]
      }'   

Usage Example

curl -X POST http://localhost:8002/api/providers/openai/v1/chat/completions \
   -H "Authorization: Bearer my-secret-key" \
   -H "Content-Type: application/json" \
   -d '{
          "model": "gpt-4",
          "messages": [
              {
                  "role": "system",
                  "content": "hi"
              }
          ],
          "user": "my-user-id"
    }'

Create a managed API key by sending a PUT request to /api/key-management/keys. This allows you to enforce rate limits and cost limits on specific users or applications.

Required fields:

• name: A descriptive name for the key.
• key: The actual string used for Bearer authentication.
• settingIds: An array containing the id obtained from the provider setting configuration.
• rateLimitOverTime: The number of requests allowed.
• rateLimitUnit: The time unit for the rate limit (e.g., m for minutes).
• costLimitInUsd: The maximum spend allowed in USD.

curl -X PUT http://localhost:8001/api/key-management/keys \
   -H "Content-Type: application/json" \
   -d '{
	      "name": "My Secret Key",
	      "key": "my-secret-key",
	      "tags": ["mykey"],
        "settingIds": ["ID_FROM_STEP_FOUR"],
        "rateLimitOverTime": 2,
        "rateLimitUnit": "m",
        "costLimitInUsd": 0.25
      }'   

You can restrict BricksLLM API keys to specific models, endpoints (paths), and rate/spend limits. This is achieved in two steps: first, creating a provider setting that defines allowed models, and second, creating a Bricks API key that references that setting and defines specific path/method restrictions and usage limits.

### Step 1 - Create a provider
curl -X PUT http://localhost:8001/api/provider-settings \
   -H "Content-Type: application/json" \
   -d '{
          "provider":"openai",
          "setting": {
             "apikey": "YOUR_OPENAI_KEY"
          },
          "allowedModels": ["gpt-3.5-turbo"]
      }'   

### Step 2 - Create a Bricks API key
curl -X PUT http://localhost:8001/api/key-management/keys \
   -H "Content-Type: application/json" \
   -d '{
	      "name": "My Secret Key",
	      "key": "my-secret-key",
	      "tags": ["mykey"],
        "settingIds": ["ID_FROM_STEP_ONE"],
        "rateLimitOverTime": 2,
        "rateLimitUnit": "m",
        "costLimitInUsd": 0.25,
        "allowedPaths": [{
            "path": "/api/providers/openai/v1/chat/completions",
            "method": "POST"
        }]
      }'