DOCSEARCHbyAlgolia
Submit Docs

SerpBear Documentation

repository·main·Indexed 24 days ago

https://github.com/towfiqi/serpbear

An open-source Search Engine Position Tracking and Keyword Research application. SerpBear allows users to track unlimited domains and keywords in Google, integrate with Google Search Console and Ads, and receive position change notifications via SMTP. It supports various third-party SERP scraping services (such as serper.dev, serpapi.com, and scrapingrobot.com) or custom proxy IPs, and can be deployed using Docker Compose.

Tokens
1.3K
Snippets
1
Records
6
Agent score
84%

What's inside SerpBear

  1. Overview of SerpBear

    main

    SerpBear is an open-source Search Engine Position Tracking and Keyword Research application. It enables users to track website keyword positions in Google and receive notifications regarding position changes.

    Key capabilities include:

    • Unlimited Tracking: Track unlimited domains and keywords.
    • SERP API: Access built-in APIs for marketing and data reporting.
    • Keyword Research: Generate keyword ideas via Google Ads integration.
    • Google Search Console Integration: View actual search visits, impressions, and performance data for tracked keywords.
    • Email Notifications: Receive daily, weekly, or monthly updates on keyword position changes via SMTP.
    • PWA Support: Install as a Progressive Web App for mobile use.

  2. Quickstart guide for SerpBear

    main

    Follow these steps to set up and start tracking keywords with SerpBear:

    1. Deploy & Run: Deploy the application (e.g., using Docker or hosting on mogenius.com or Fly.io).
    2. Access & Login: Open the application in your browser and log in.
    3. Add Domain: Add your first domain to the tracker.
    4. Configure Scraping: Obtain an API key from a provider (like ScrapingRobot) or prepare your Proxy IPs. If using a provider, skip to step 5.
    5. Setup Scraping API/Proxy: Navigate to the App's Settings interface to configure your chosen scraping service or proxy.
    6. Track Keywords: Add your keywords to begin tracking.
    7. Configure Notifications (Optional): In the Settings panel, configure SMTP details (e.g., using ElasticEmail or Sendpulse) to receive email updates.

  3. Deploy SerpBear using Docker Compose

    main

    To run SerpBear, use the provided docker-compose.yml. The service uses a named volume serpbear_data mapped to /app/data inside the container to ensure data persistence.

    By default, the service pulls the towfiqi/serpbear:latest image. If you wish to build from source, uncomment the build: . line in the app service definition.

    services:
  app:
    image: towfiqi/serpbear:latest
    restart: unless-stopped
    ports:
      - "${PORT:-3000}:3000"
    environment:
      - USER_NAME=${USER:-admin}
      - PASSWORD=${PASSWORD}
      - SECRET=${SECRET}
      - APIKEY=${APIKEY}
      - SESSION_DURATION=${SESSION_DURATION:-24}
      - NEXT_PUBLIC_APP_URL=${NEXT_PUBLIC_APP_URL:-http://localhost:3000}
      - SEARCH_CONSOLE_CLIENT_EMAIL=${SEARCH_CONSOLE_CLIENT_EMAIL:-}
      - SEARCH_CONSOLE_PRIVATE_KEY=${SEARCH_CONSOLE_PRIVATE_KEY:-}
    volumes:
      - serpbear_data:/app/data

volumes:
  serpbear_data:

  4. Configure SerpBear via environment variables

    main

    When deploying SerpBear using Docker Compose, you can configure the application behavior and security using environment variables. These variables should be defined in your shell or an .env file.

    Required Variables

    • PASSWORD: The password for the admin user.
    • SECRET: A secret key used for application security.
    • APIKEY: The API key for the application.

    Optional Variables

    • PORT: The host port to map to the container's port 3000. Defaults to 3000.
    • USER_NAME: The username for the admin account. Defaults to admin.
    • SESSION_DURATION: The session duration in hours. Defaults to 24.
    • NEXT_PUBLIC_APP_URL: The public URL of the application. Defaults to http://localhost:3000.
    • SEARCH_CONSOLE_CLIENT_EMAIL: Email address for Google Search Console integration.
    • SEARCH_CONSOLE_PRIVATE_KEY: Private key for Google Search Console integration.

  5. Configure SERP scraping services

    main

    SerpBear does not scrape Google directly; it relies on third-party scrapers or proxy IPs to fetch search results. You can configure these in the application settings.

    Supported services include:

    • serper.dev: Pay As You Go ($1.00/1000 req), no API.
    • serply.io: $49/mo, includes API.
    • serpapi.com: From $50/mo, includes API.
    • SearchApi.io: From $40/mo, includes API.
    • valueserp.com: Pay As You Go ($2.50/1000 req), no API.
    • hasdata.com: From $29/mo, includes API.
    • crazyserp.com: From $47/mo, includes API.
    • scrapingant.com: From $19/mo, includes API.
    • scrapingrobot.com: Free tier (5000/mo), includes API.
    • Proxy IPs: You can provide your own proxy IPs instead of using a third-party service.

  6. Available SERP scraping services

    main
    The scrapers/index.ts module exports a default array containing all supported SERP (Search Engine Results Page) scraping service implementations. Developers can iterate over this list to access different provider modules, which include specialized services like serpapi, serper, scrapingant, and others. Each service in this list follows a consistent interface for performing scraping tasks.