Scweet Documentation

Scweet uses a structured exception hierarchy. You can catch specific errors or use ScweetError as a catch-all.

• ScweetError (Base)
  • AccountPoolExhausted: No eligible accounts (all cooled down or at daily limits).
  • EngineError: Engine-level runtime error.
    • RunFailed: Run completed but couldn't produce results.
      • RateLimitError: All accounts rate-limited (429).
      • AuthError: Credentials invalid or expired (401/403).
      • NetworkError: Network/connectivity failure.
      • ProxyError: Proxy misconfiguration or connectivity failure.

When you initialize a Scweet instance, it provisions your accounts into a local SQLite database (default: scweet_state.db).

Key behaviors:

• Automatic Storage: Cookies are imported, validated, and stored to manage rate limits, cooldowns, and daily caps.
• Persistence: You only need to provide credentials once. Subsequent runs can reuse the database.
• Deduplication: If an account already exists in the DB (matched by username or auth_token), it is updated rather than duplicated.
• Control: Provisioning is enabled by default (provision=True).

Reusing an existing database: To use accounts already in your DB without re-providing cookies:

s = Scweet(db_path="scweet_state.db")

To skip provisioning and only work with existing accounts:

s = Scweet(db_path="scweet_state.db", provision=False)

Scweet provides two layers of limits to protect your accounts:

1. Per-call limit: A parameter passed to pagination methods (search, get_profile_tweets, get_followers, get_following). It defines the maximum number of items to collect in that specific call. It is highly recommended to always set this to avoid burning through account quotas.
2. Account daily caps: Configured via ScweetConfig (daily_requests_limit, daily_tweets_limit). These act as safety nets for the total number of requests/tweets allowed per account per UTC day.

Example usage of per-call limit:

tweets = s.search("python", limit=200)              # stop after 200 tweets
tweets = s.get_profile_tweets(["elonmusk"], limit=100)
users  = s.get_followers(["elonmusk"], limit=500)

Scweet uses the standard Python logging module under the "Scweet" namespace. By default, no output is produced. To see logs, configure a handler for the "Scweet" logger.

import logging

logging.basicConfig(level=logging.INFO)
# or target only Scweet:
logging.getLogger("Scweet").setLevel(logging.INFO)
logging.getLogger("Scweet").addHandler(logging.StreamHandler())

To use Scweet, install it using pip. It is recommended to use the -U flag to ensure you have the latest version.

pip install -U Scweet

Twitter/X rotates GraphQL query IDs periodically. To ensure your requests use current IDs, you can enable manifest_scrape_on_init=True in the Scweet constructor. This fetches the current main.js bundle from X on initialization and extracts the latest IDs. This adds a few seconds to startup time.

# Python
s = Scweet(cookies_file="cookies.json", manifest_scrape_on_init=True)

# CLI
scweet --auth-token TOKEN --manifest-scrape-on-init search "query" --limit 100

Initialize the Scweet class with your auth_token. You can also provide a proxy for increased security and reduced ban risk.

Note: Always set a limit to prevent scraping until your account's daily cap is hit. All methods have async variants (e.g., asearch(), aget_profile_tweets()).

from Scweet import Scweet

# Credentials are stored in scweet_state.db automatically on first run
s = Scweet(auth_token="YOUR_AUTH_TOKEN", proxy="http://user:pass@host:port")

# Search tweets — save to CSV (save_format="json" or "both" also works)
tweets = s.search("bitcoin", since="2025-01-01", limit=500, save=True)

# Reuse provisioned accounts on subsequent runs using the state database
s = Scweet(db_path="scweet_state.db")
tweets = s.search("ethereum", limit=500, save=True)

For one-off runs or simple scripts, you can pass cookies directly into the Scweet constructor.

Single account:

s = Scweet(cookies={"auth_token": "...", "ct0": "..."})

Multiple accounts:

s = Scweet(cookies=[
    {"auth_token": "tok1", "ct0": "ct0_1"},
    {"auth_token": "tok2", "ct0": "ct0_2"},
])

The recommended way to provide Twitter/X authentication is via a cookies.json file. You can obtain your auth_token and ct0 values from your browser's DevTools (Application > Cookies > https://x.com).

Scweet can handle multiple accounts, each with its own optional proxy to reduce ban risk and enable concurrent scraping.

Example cookies.json structure:

[
  {
    "username": "your_account",
    "cookies": { "auth_token": "..." }
  }
]

Usage:

s = Scweet(cookies_file="cookies.json")

If you are upgrading from version 4, note the following API changes:

You can resume a search from a previous checkpoint using the resume=True parameter. This works by matching a hash of the query parameters. To resume correctly, you must provide the exact same since, until, query, lang, and display_type used in the original run.

# First run — gets interrupted or completes partially
tweets = s.search("query", since="2024-01-01", until="2024-06-01", limit=1000)

# Resume — picks up from last saved checkpoint
tweets = s.search("query", since="2024-01-01", until="2024-06-01", limit=1000, resume=True)

If you only have an auth_token, Scweet will automatically bootstrap the ct0 value for you. You can also provide a global proxy directly during initialization.

Single account:

s = Scweet(auth_token="YOUR_AUTH_TOKEN")

With global proxy:

s = Scweet(auth_token="YOUR_AUTH_TOKEN", proxy="http://user:pass@host:port")