DOCSEARCHbyAlgolia
Submit Docs

Skill Seekers

repository·development·Indexed 12 days ago

https://github.com/yusufkaraaslan/skill_seekers

A data layer for AI systems that transforms 18 different source types—including documentation websites, GitHub repositories, PDFs, and videos—into structured knowledge assets. It allows developers to export data to 22 different targets such as LLM platforms, RAG pipelines, and AI coding assistants. Features include a FastAPI backend, a Claude Code plugin with MCP support, and a GitHub Action (v3) for automating the creation and updating of AI skills.

Tokens
378.4K
Snippets
1.1K
Records
1.6K
Agent score
96%

What's inside Skill Seekers

  1. Overview of Skill Seekers capabilities

    development

    Skill Seekers is a tool designed to transform any knowledge source—such as documentation websites, PDFs, codebases, GitHub repositories, Office documents, and more—into AI skills (e.g., for Claude). It provides both a CLI and an MCP (Model Context Protocol) interface for creating, enhancing, and packaging these skills for various AI platforms.

    Key Capabilities (v3.6.0):

    • 18 Supported Source Types: Documentation, GitHub, PDF, video, Word, EPUB, Jupyter, local HTML, OpenAPI, AsciiDoc, PowerPoint, RSS/Atom, man pages, Confluence, Notion, Slack/Discord, and local codebases.
    • Multi-Platform Support: Automatically uploads to 21 platforms including Claude, Gemini, OpenAI, DeepSeek, Qwen, LangChain, LlamaIndex, and various vector databases (Pinecone, ChromaDB, etc.).
    • Unified CLI: A single create command that uses auto-detection to handle all 18 source types.
    • MCP Integration: 40+ functional MCP tools for interacting with skills and configurations.
    • Advanced Analysis: Includes a codebase analysis suite (C3.x) for pattern recognition and architectural overview generation.

  2. Overview of Skill Seekers

    development
    Skill Seekers is a Python CLI tool and MCP server designed to convert 18 different source types (including documentation sites, GitHub repositories, PDFs, videos, notebooks, and wikis) into structured, AI-ready skills. It supports 22 export targets, covering 12 LLM platforms, 8 RAG/vector targets, Atlas, and IBM Bob. It is published on PyPI as skill-seekers.

  3. Overview of the Skill Seekers Intelligence System

    development

    The Skill Seekers Intelligence System is an auto-updating, context-aware, multi-skill codebase intelligence system designed to act as a self-maintaining RAG (Retrieval-Augmented Generation) system for your codebase.

    Key capabilities include:

    • Automatic Tech Stack Detection: Identifies frameworks like FastAPI, React, or PostgreSQL.
    • Skill Generation: Creates separate skills for specific libraries and codebase modules.
    • Automated Updates: Uses git-based triggers to update skills when branches merge.
    • Intelligent Clustering: Loads only the relevant skills based on the current work context.
    • AI Integration: Integrates with Claude Code via a plugin system.

  4. Overview of the Skill Seekers ecosystem

    development

    Skill Seekers is a distributed project across several repositories:

    • Skill_Seekers: The core CLI and MCP server (this repository). Available on PyPI.
    • skillseekersweb: The official website and documentation.
    • skill-seekers-configs: A repository for community-contributed configurations.
    • skill-seekers-action: A GitHub Action for integration and continuous deployment.
    • skill-seekers-plugin: A plugin for Claude Code.
    • homebrew-skill-seekers: A Homebrew tap for macOS users.

  5. Overview of Skill Seekers capabilities and outputs

    development

    Skill Seekers is a tool designed to transform various data sources (documentation, GitHub repositories, PDFs, videos) into high-quality knowledge assets for AI.

    Depending on your use case, it produces different outputs:

    • AI Skills: Detailed SKILL.md files and reference files for LLMs like Claude Code, Gemini, or GPT.
    • RAG Pipelines: Chunked documents with rich metadata for frameworks like LangChain, LlamaIndex, or Haystack.
    • Vector Databases: Pre-formatted data ready for upserting into Pinecone, Chroma, Weaviate, FAISS, or Qdrant.
    • AI Coding Assistants: Context files designed for IDE AI agents like Cursor, Windsurf, Cline, or Continue.dev.

  6. Integration Strategy Document Map

    development

    The docs/strategy/ directory contains the following core documents for implementing the integration strategy:

    • INTEGRATION_STRATEGY.md: The master strategy document containing Tier 1-3 opportunities, an implementation priority matrix, and a 4-week action plan.
    • DEEPWIKI_ANALYSIS.md: Analysis of how the DeepWiki-open tool successfully positioned Skill Seekers, providing a replication template.
    • INTEGRATION_TEMPLATES.md: Copy-paste templates for integration guides, case studies, outreach emails, and social media.
    • ACTION_PLAN.md: A detailed execution plan with week-by-week breakdowns and daily checklists.
    • ../case-studies/deepwiki-open.md: A reference case study detailing a successful integration workflow and metrics.

  7. Navigate the Skill Seekers documentation

    development

    The Skill Seekers documentation is organized into four main categories based on user needs and expertise levels. Use this guide to find the information most relevant to your current goal:

    1. Getting Started (getting-started/): For new users. Contains installation steps, quick starts, and your first skill walkthrough.
    2. User Guide (user-guide/): For common tasks. Covers core concepts, scraping (all 18 source types), AI enhancement, packaging, and troubleshooting.
    3. Reference (reference/): For technical lookup. Provides authoritative details on the CLI, MCP tools, configuration JSON schemas, and environment variables.
    4. Advanced (advanced/): For power users. Covers MCP server setup, custom workflows, and multi-source scraping.

  8. Skill Seekers Ecosystem overview

    development

    Skill Seekers is part of a larger ecosystem of repositories:

    • Skill_Seekers: The main CLI and MCP server (this repository).
    • skillseekersweb: The official website and documentation.
    • skill-seekers-configs: Community-contributed configurations.
    • skill-seekers-action: GitHub Action for CI/CD integration.
    • skill-seekers-plugin: Plugin for Claude Code.
    • homebrew-skill-seekers: Homebrew tap for macOS users.

  9. Review Skill Seekers release history and capabilities

    development

    Skill Seekers has evolved from a documentation scraper to a comprehensive codebase analysis and AI skill generation suite.

    Key capabilities available in current versions (v2.6.0+):

    • C3.x Codebase Analysis Suite: Includes pattern detection, test extraction, and architecture analysis.
    • Multi-platform Support: Adaptors for Claude, Gemini, OpenAI, and Markdown.
    • MCP Integration: Provides up to 40 MCP tools (as of v3.6.0) to interface with AI agents.
    • Unified Scraping: Supports GitHub repository scraping, PDF extraction/conversion, and multi-source synthesis.
    • Large Scale Support: Capable of handling documentation with 40K+ pages.
    • Preset Configurations: Includes 12+ preset configurations for various scraping scenarios.

  10. Known issues and bug audit in Skill Seekers

    development

    The docs/BUG_AUDIT.md file contains a detailed log of identified bugs, regressions, and investigated issues across various scrapers (Document, Media, Enhancement, and Infrastructure). This document serves as a reference for developers to understand historical defects and the status of fixes in the project.

    Key categories of issues include:

    • Document Scrapers: Issues with llms.txt overwriting, parallel scraping bounds, EPUB image handling, and HTML-to-Markdown link extraction.
    • Media/Structured/Remote Scrapers: Issues with OpenAPI $ref resolution, RSS time budgets, Confluence pagination, GitHub rate-limiting, and Slack/Discord chat scraping.
    • Enhancement + Builder: Issues with AI provider fallbacks, quality metric thresholds, and PDF content assembly.
    • Args / Infra: Issues with orphan CLI flags, Azure storage authentication, and unused create command presets.

  11. Explore the Skill Seekers ecosystem

    development

    Skill Seekers is a multi-repository ecosystem. Depending on your needs, you may want to interact with different parts of the project:

    • Skill_Seekers (this repository): Contains the core CLI and the MCP (Model Context Protocol) server. Available on PyPI.
    • skillseekersweb: The official website and documentation.
    • skill-seekers-configs: A repository for community-contributed configurations.
    • skill-seekers-action: A GitHub Action designed for CI/CD workflows.
    • skill-seekers-plugin: A specific plugin for Claude Code.
    • homebrew-skill-seekers: A Homebrew Tap for macOS users.

  12. Understand the Skill Seekers Architecture

    development

    Skill Seekers is organized into several core functional modules that work together to transform various source types into AI-ready skills.

    Core Modules

    • CLICore: Handles command dispatching and argument parsing via SubcommandParser subclasses.
    • Scrapers: Uses SkillConverter and get_converter() to extract data from 18+ source types.
    • Scan: Performs codebase analysis using the C3.x pipeline.
    • Adaptors: Provides standardized interfaces for storage (S3, GCS, Azure) and skills.
    • Analysis (C3.x Pipeline): A multi-stage process including pattern recognition, test extraction, guide building, config extraction, and architectural pattern detection.
    • Enhancement: AI-driven refinement of skills based on an --enhance-level parameter.
    • Packaging: Prepares skills for specific AI platforms.
    • MCP Server: Allows AI agents (like Claude Code or Cursor) to invoke scraping, packaging, and analysis tools via JSON-RPC.