DOCSEARCHbyAlgolia
Submit Docs

trends-checker

repository·master·Indexed 18 days ago

https://github.com/akvise/trends-checker

A Python CLI tool (v0.2.0) for analyzing Google Trends interest across Web, YouTube, Images, News, and Shopping categories. It features reliability layers including cookie authentication, exponential backoff, and a DataForSEO backend to bypass rate limits. The tool supports continuous monitoring via a Watch Mode with configurable polling intervals, alert thresholds, and webhook integration, and can export results in JSON, CSV, and Markdown formats.

Tokens
3.3K
Snippets
11
Records
14
Agent score
13%

What's inside trends-checker

  1. How trends-checker works

    master

    The tool queries the unofficial Google Trends API via pytrends and applies several reliability layers:

    1. Request: Sends keyword batches to Google Trends (/explore, /multiline).
    2. Cookie auth: Optionally injects browser cookies to avoid cold-start 429 errors.
    3. Rate limiting: Uses configurable sleep and jitter between geo requests.
    4. Retry logic: Implements exponential backoff on 429/503 errors.
    5. DataForSEO fallback: Allows swapping the backend entirely for zero rate limits and real search volumes.

    Results are returned as a normalized interest score (0–100) per keyword per region.

  2. Handle 429 Rate Limiting errors

    master

    Since the Google Trends API is unofficial and rate-limited, use one of these two methods to avoid 429 errors:

    Option 1: Browser Cookies (Free)

    Inject cookies from your browser to warm up the session.

    • Use --cookie-file <path> to point to a cookie file.
    • Use --cookie "<raw_header>" for the raw value.
    • Alternatively, set the TRENDS_COOKIE environment variable.

    Option 2: DataForSEO API (Recommended for automation)

    Swap to the DataForSEO backend for zero rate limits and real search volumes.

    • Use --dataforseo-key <user>:<password>.
    • Alternatively, set the DATAFORSEO_KEY environment variable.
    # Using cookie file
trends-checker --cookie-file cookie.txt --geo US

# Using environment variable for cookies
TRENDS_COOKIE="NID=...;" trends-checker --geo US

# Using DataForSEO key
trends-checker --keywords "vibe coding" --dataforseo-key login@email.com:password

# Using environment variable for DataForSEO
DATAFORSEO_KEY="login@email.com:password" trends-checker --keywords "AI agents"

  3. Use Watch Mode for continuous monitoring

    master

    Enable continuous polling and alerts using the --watch flag. This is useful for tracking keyword velocity over time.

    Key features:

    • --interval: Set how often to poll (e.g., 6h, 30m, 1d).
    • --threshold: Set the percentage change required to trigger an alert.
    • --watchlist: Provide a TOML file for keyword/geo configurations.
    • --watch-snapshot: Persist a baseline snapshot (JSON) between runs to track changes.
    • --watch-output: Save watch events to a JSON file.
    # Poll every 6h, alert on >20% change
trends-checker --keywords "AI agents,vibe coding" --watch --interval 6h --threshold 20

# Watch mode with JSON output for external monitoring
trends-checker --keywords "AI agents" --watch --interval 1d --watch-output watch.json --format json

# Using a watchlist config and persistent baseline
trends-checker --watch --watchlist watchlist.toml --watch-snapshot watch-snapshot.json

  4. Install trends-checker

    master

    You can install trends-checker via pip for quick access, or build it from source.

    Via pip:

    pip install trends-checker

    From source:

    git clone https://github.com/akvise/trends-checker
cd trends-checker
make install
    pip install trends-checker

  5. Configure Watch Mode for continuous monitoring

    master

    Watch mode allows you to continuously poll Google Trends and trigger alerts when keyword interest changes significantly.

    • --watch: Enable continuous polling.
    • --watchlist: Path to a TOML configuration file containing keywords, geos, etc. CLI flags will override TOML values.
    • --interval: Polling interval (e.g., 30m, 6h, 1d). Default is 6h.
    • --threshold: Percentage change threshold to trigger an alert (e.g., 20.0 for 20%).
    • --webhook: POST watch alerts as JSON to a specified URL (e.g., Slack/Discord webhook).
    • --watch-snapshot: Path to persist the baseline JSON snapshot between runs.
    • --watch-output: Path to write all watch events as a JSON file for external monitoring.
    trends-checker --watch --watchlist config.toml --threshold 25 --webhook https://hooks.slack.com/services/... --watch-snapshot baseline.json

  6. Run trends-checker CLI

    master

    The trends-checker CLI analyzes Google Trends interest across different search categories (Web, YouTube, Images, News, Shopping) for a set of keywords across various geographic regions. It supports standard Google Trends scraping via pytrends or a rate-limit-free backend via the DataForSEO API.

    # Basic usage with default keywords and geos
trends-checker

# Custom keywords and regions
trends-checker --keywords "AI, Coding" --geo "US,BR"

# Using a keywords file
trends-checker --keywords-file path/to/keywords.txt

  7. Use DataForSEO for rate-limit-free analysis

    master

    To avoid Google Trends rate limits (429 errors) and get real search volumes, provide DataForSEO API credentials. This mode bypasses the standard pytrends scraping logic.

    • --dataforseo-key: Provide credentials in the format username:password.
    • Alternatively, set the DATAFORSEO_KEY environment variable.
    trends-checker --dataforseo-key "your_username:your_password"
# OR
export DATAFORSEO_KEY="your_username:your_password"
trends-checker

  8. Basic usage of trends-checker

    master

    Run a basic trend analysis for specific keywords in a specific geographic region using the default Google Trends backend.

    trends-checker --keywords "AI agents,vibe coding" --geo US
    trends-checker --keywords "AI agents,vibe coding" --geo US

  9. Configure search categories and timeframes

    master

    You can specify which Google search property to query and the time range for the analysis.

    • --group: Choose the search category: web (default), youtube, images, news, or shopping.
    • --timeframe: Set the timeframe (e.g., 'today 12-m', 'today 5-y').
    • --since: Provide a specific start date in YYYY-MM-DD format. This overrides --timeframe by setting the range from the start date to today.
    trends-checker --group youtube --timeframe "today 12-m"
trends-checker --since 2024-01-01

  10. Configure output formats and filtering

    master

    Customize how results are displayed or saved using the following flags:

    • --format json: Outputs JSON data (ideal for automation/piping).
    • --format csv: Outputs CSV data to stdout.
    • --output <path>: Saves the output to a specific file (e.g., results.csv).
    • --top <n>: Shows only the top n keywords by mean interest.
    • --no-color: Disables unicode bars and colors (safe for CI/CD, cron jobs, and log files).
    trends-checker --keywords "AI agents,vibe coding" --format json
trends-checker --keywords "AI agents,vibe coding" --format csv --output results.csv
trends-checker --keywords "AI agents,vibe coding,cursor ide,llm,rag" --top 3
trends-checker --keywords "AI agents,vibe coding" --no-color

  11. Reference: trends-checker CLI parameters

    master

    Full list of available CLI parameters:

    ParameterDescriptionDefault
    --keywordsComma-separated terms (max 5)AI agents, vibe coding, ...
    --keywords-fileFile with keywords (one per line)-
    --groupweb, youtube, images, news, shoppingweb
    --geoISO country codes or WWWW,US,BR,ES,IN,ID,RU
    --timeframeTime period ("today 12-m", "today 5-y")today 12-m
    --displayvertical or widevertical
    --outputCSV export path-
    --relatedShow rising related queriesfalse
    --sleepSeconds between geo requests1.2
    --retriesRetry attempts on 429 errors3
    --backoffExponential backoff base (seconds)1.5
    --jitterRandom jitter added to delays0.6
    --cookie-fileBrowser cookie file-
    --cookieRaw cookie header value-
    --proxyHTTP/HTTPS proxy URLs (comma-separated)-
    --dataforseo-keyDataForSEO credentials (user:pass)$DATAFORSEO_KEY
    --hlUI language (e.g., en-US)en-US
    --watchEnable watch mode (continuous polling + alerts)false
    --watchlistPath to watchlist TOML (keywords/geos/timeframe/etc)-
    --intervalPolling interval in watch mode (e.g. 6h, 30m, 1d)6h
    --thresholdPercentage change to trigger an alert20
    --watch-outputPath to write watch events as JSON-
    --watch-snapshotPath to persist baseline snapshot between runs (JSON)-

  12. Select search categories (groups)

    master

    Use the --group flag to specify which Google Trends category to query. Available options are:

    • web (default)
    • youtube
    • images
    • news
    • shopping
    trends-checker --group youtube --keywords "cursor ide tutorial"
trends-checker --group images --keywords "AI generated art"
trends-checker --group news --keywords "artificial intelligence"
trends-checker --group shopping --keywords "mechanical keyboard"