DOCSEARCHbyAlgolia
Submit Docs

llms-tools: Curated AI Tools and Research

repository·main·Indexed 43 days ago

https://github.com/petroivaniuk/llms-tools

A curated collection of AI tools, open-source resources, and research projects focusing on Large Language Models (LLMs), Computer Vision, and Large Visual Language Models (LVLMs). Includes directories of frontier and open-source models, local model runners like ollama and LM Studio, evaluation benchmarks such as SWE-bench and AgentBench, and educational resources from DeepLearning.AI and LangChain Academy.

Tokens
12.8K
Snippets
2
Records
44
Agent score
6%

What's inside llms-tools

  1. Overview of LLMs Tools & Research Projects

    main
    This repository serves as a curated collection of ready-to-use AI tools, open-source projects, and research papers. While the primary focus is on Large Language Models (LLMs) and chatbots, it also includes research from other AI domains such as Computer Vision.

  3. Explore AI Monetization and Distribution models

    main

    Information on how creators can monetize or distribute AI models and bots:

    • Poe: Uses a price-per-message revenue model for AI bot creators.
    • GPTs Store: Allows creating custom versions of ChatGPT with specific instructions and knowledge.
    • Voice Library (ElevenLabs): Allows users to share their voice and earn rewards when it is used.
    • HuggingChat: Provides community access to high-quality AI chat models.

  4. Explore Computer Vision models and frameworks

    main

    A collection of state-of-the-art Computer Vision models is provided, covering several domains:

    • Segmentation & Tracking: The SAM (Segment Anything Model) family (SAM, SAM2, SAM3) by MetaAI for detection and tracking. Other tools include OMG-Seg for unified segmentation tasks, Track-Anything for interactive video tracking, and EdgeSAM/EfficientSAM for optimized/lightweight execution.
    • Self-Supervised Learning: DINOv2/DINOv3 for large-scale vision learning and I-JEPA (Image Joint Embedding Predictive Architecture) for predicting image representations.
    • Motion & Point Tracking: CoTracker/CoTracker3 for multi-point video tracking, TAPIR (Google DeepMind) for Tracking Any Point, and Omnimotion for dense long-range motion estimation.
    • Depth & Reconstruction: Depth-Anything for depth estimation and Behind the Scenes for single-view reconstruction via density fields.
    • Multimodal & Prompting: ImageBind (MetaAI) for binding multiple modalities (Audio, Image, Text) and Visual Prompting for applying text-prompting concepts to computer vision.

  5. Explore AI hardware devices and wearables

    main

    The repository maintains a curated list of AI-powered hardware devices. These include:

    • General AI Wearables & Companions: Devices like Omi (conversation management), NotePin (AI memory capsule), Friend (AI necklace), Limitless (personalized AI), and Rabbit r1 (natural language OS).
    • Specialized Assistive & Industrial Tech: Biped.ai (wearable vest for visually impaired), Mytra (warehouse robotics), and Codex Micro (Codex Keyboard).
    • Computing & Inference: 01 Project (open-source language model computer) and LPU Inference Engine (Groq's Language Processing Units).

  7. Explore Science & Climate AI tools

    main

    The repository lists several AI research projects for scientific discovery and environmental monitoring:

    • Materials & Geospatial Science: MatterGen (generative materials design), GNoME (predicting material stability), OlmoEarth Platform (planetary scale geospatial inference), and AlphaEarth Foundations (virtual satellite modeling).
    • Quantum Computing: AlphaQubit (quantum error correction).
    • Climate & Weather Forecasting:
      • WeatherNext 2 & 3 (high-resolution weather forecasting with satellite data).
      • GraphCast (global weather forecasting).
      • MetNet-3 (high-resolution neural weather model).
      • ClimaX (foundation model for weather and climate).
      • Weather Lab (experimental cyclone predictions).
    • Carbon Capture: OpenDAC (research for reducing Direct Air Capture costs).

  8. Explore Bio & Med AI tools

    main

    The repository maintains a curated list of AI tools and research projects focused on biology and medicine. Key categories include:

    • Genomics & Protein Research: Tools like AlphaFold 3 (predicting protein/DNA/RNA structures), AlphaGenome (predicting DNA variant impacts), AlphaMissense (classifying genetic mutations), and evodiff (protein sequence generation).
    • Medical Reasoning & Diagnostics: AMIE (diagnostic reasoning), meditron (open-source medical LLM), Med-PaLM (medical Q&A), and MentalLLaMA (mental health analysis).
    • Brain-Computer Interface (BCI) & Reconstruction: Brain2Qwerty v2 (real-time sentence decoding), Brain2Music (music reconstruction), and Mind-Video (video reconstruction from brain activity).
    • Medical Imaging: SAM-Med2D (Segment Anything Model for 2D medical images) and Med-Flamingo (multimodal medical vision-language model).

  9. Explore AI Image Editing and Generation tools

    main

    The repository maintains a curated list of tools for image manipulation, generation, and enhancement. Key categories include:

    • Image Editing & Customization: Tools like Qwen-Image-Edit for editing, AnyDoor for zero-shot object-level customization, PhotoMaker for realistic human photo customization, and Textify for replacing AI-generated text gibberish.
    • Generation & Design: SkyReels for generating comics, Designer (Microsoft) for AI-assisted design, Scribble Diffusion for turning sketches into images, and VDraw for effortless image editing.
    • Enhancement & Restoration: Magnific for upscaling/enhancing, UnblurImage for removing blur, and Interpolating between Images with Diffusion Models for controllable interpolation.
    • Specialized Tasks: Material Palette for extracting PBR materials, DiffusionLight for lighting estimation, and Matting Anything for alpha matte estimation.

  10. Explore 3D Generative AI and Reconstruction tools

    main

    The llms-tools repository maintains a curated list of research projects and tools focused on 3D content generation, reconstruction, and manipulation. These tools cover several categories:

    3D Asset Generation (Text/Image to 3D)

    • Stable Fast 3D / Stable Zero123 / TripoSR (Stability AI): High-quality 3D asset generation from single images.
    • 3DGen (MetaAI): Fast pipeline for text-to-3D asset generation with high prompt fidelity.
    • Genie (LumaLabs): A foundational 3D model.
    • Shap-E (OpenAI): A conditional generative model for 3D assets.
    • Masterpiece X: Generative text-to-3D app for meshes, textures, and animations.
    • Spline: Generate 3D objects from text prompts and images.

    3D Reconstruction and NeRFs

    • Neuralangelo (NVIDIA): High-fidelity 3D surface reconstruction from RGB video.
    • 3D Gaussian Splatting: A technique for real-time (≥ 100 fps) novel-view synthesis and 3D reconstruction.
    • Mip-NeRF 360: Unbounded anti-aliased Neural Radiance Fields for large-scale scenes.
    • InstantMesh: Efficient 3D mesh generation from a single image using sparse-view models.
    • AutoRecon: Automated discovery and reconstruction from multi-view images.

    3D Design and CAD

    • cadwithai: Use an AI chatbot to create and edit CAD models.
    • backflip: AI 3D design tools for physical world applications.

    3D Datasets

    • Objaverse-XL: Open dataset containing over 10 million 3D objects.
    • OmniObject3D: Large vocabulary dataset of high-quality real-scanned 3D objects.
    • VGGHeads: Synthetic dataset for 3D human heads.

  11. Explore Agentic Frameworks and Tools

    main

    The llms-tools repository maintains a curated list of resources for building, deploying, and interacting with AI agents. This collection includes frameworks for multi-agent orchestration, specialized agent protocols, and tools for specific domains like finance, software development, and browser automation.

    Key Categories of Agent Tools:

    • Multi-Agent Orchestration & Frameworks: Tools for managing complex interactions between multiple agents.

      • swarm (OpenAI): An educational framework for lightweight multi-agent orchestration.
      • Multi-Agent Orchestrator (AWS): A framework for managing multiple agents and complex conversations.
      • BeeAI Framework: Build production-ready multi-agent systems in Python or TypeScript.
      • ChatDev: A framework for studying collective intelligence through LLM-based software development.
      • rowboat: An AI-powered multi-agent builder using OpenAI's Agents SDK.

    • Specialized Agent Capabilities:

      • Browser & UI Control: Browser-Use (control browsers) and AppAgent (operate smartphone apps via multimodal LLMs).
      • Software Development: OpenHands (platform for software development agents).
      • Computer Use: agent.exe (enables Claude's computer use capabilities) and Agent-S (uses computers like a human).
      • Multimodal & Real-time: TEN-Agent (real-time conversational agents with voice, vision, and avatars) and UI-TARS-1.5 (multimodal agent for virtual worlds).

    • Protocols & Standards:

      • SCP: Science Context Protocol: A standard for autonomous scientific agents.
      • Agent Payments Protocol (AP2): A protocol for secure agent-led payments.
      • Agent2Agent (A2A): A protocol for interoperability between agentic applications.

    • No-Code & Production Tools:

      • Kiln AI: A desktop app for building AI products (evals, RAG, agents) without coding.
      • Pearl (MetaAI): A production-ready RL AI Agent Library.

  12. Explore Large Language Model (LLM) research and tools

    main

    This repository serves as a curated directory of various Large Language Model (LLM) research projects, models, and tools. It categorizes resources into several areas including:

    • Models: A wide range of models from frontier class (e.g., Kimi, Gemini, Muse Spark) to open-source reproductions (e.g., Open-R1, OpenLLaMA) and specialized small models (e.g., SmolLM, Granite Nano).
    • Research & Tools: Frameworks for tool-use (ToolBench), factual error detection (FacTool), symbolic math computation (Math Swarm), and multi-agent workflows (MetaGPT).
    • Applications: AI research assistants (Paperguide), no-code chatbot platforms (ChatFlow), and specialized writing tools (TextFX).
    • Techniques & Algorithms: Speed-up algorithms (FlashAttention), diffusion models (Gemini Diffusion, LLaDA), and scaling recipes (s1).

    Users can use this list to find specific models for deployment, research papers for study, or tools to augment their existing AI workflows.