DOCSEARCHbyAlgolia
Submit Docs

PLUS-WAVE Blog

website·Indexed 30 days ago

https://blog.pluswave.top/

A personal technical blog by Jiawei Wang covering computer vision (CV), 3D vision, VLM, and reasoning. The site features detailed guides and implementations on OpenGL (including fluid, water, and elastic object simulations), CUDA, PyTorch, and system configurations like WSL2 and Nginx deployment.

Tokens
104.6K
Snippets
373
Records
578
Agent score
52%

What's inside PLUS-WAVE

  1. Overview of the SEAL (Show, Search, and Tell) Framework

    SEAL is an MLLM framework designed to mimic human visual search by actively seeking key details in complex or high-resolution images. It consists of two core components: a VQA LLM and a Visual Search Model, which collaborate via a Visual Working Memory (VWM). The VWM acts as a dynamic record storing the original image, the question, identified target crops, and their locations.

    The operational flow is as follows:

    1. Show: The VQA LLM attempts to answer the question. If it cannot, it lists the missing key information (e.g., "need to find a red cup").
    2. Search: The Visual Search Model is triggered to locate the missing targets in the image using a priority queue.
    3. Tell: Found target crops and locations are added to the VWM. The VQA LLM then regenerates the final answer based on the enriched VWM information.

  2. Overview of T2I-R1 Bi-Level CoT Framework

    T2I-R1 introduces a Bi-Level Chain-of-Thought (BiCoT) mechanism to coordinate semantic understanding and pixel-level synthesis in image generation. It is built upon Unified Generation and Understanding LMMs (e.g., Janus-Pro). The framework consists of two reasoning layers:

    1. Semantic-level CoT: A high-level language reasoning process that occurs before image generation. It parses the prompt to plan the overall structure, object layout, and deep intent (e.g., inferring that a flower from Amsterdam should be a 'tulip').
    2. Token-level CoT: The process of generating image tokens/patches step-by-step. Each token depends on previous tokens, ensuring local consistency and visual coherence. Note that this is treated as a sequence of decisions for RL optimization rather than an explicit symbolic reasoning chain.

  3. Overview of the ZoomEye framework for MLLM visual reasoning

    ZoomEye is a training-free, model-agnostic framework that addresses the limitation of MLLMs (which often overlook details due to fixed input resolutions) by structuring an image as a tree. The root node represents the full image, and child nodes represent progressively finer local patches. The system uses a search algorithm to 'Zoom in' (explore deeper local details) and 'Zoom out' (backtrack to other regions) to find visual evidence needed to answer a query.

  4. Overview of C++ STL Container Classifications

    C++ Standard Template Library (STL) containers are broadly classified into two categories:

    1. Sequence Containers: Elements are arranged in the order they are inserted.

      • std::array: Fixed size (C++11).
      • std::vector: Dynamic size, automatically expands.
      • std::deque: Double-ended queue, expandable at both ends.
      • std::list: Doubly linked list.
      • std::forward_list: Singly linked list (C++11).

    2. Associative Containers: Use keys and values for fast lookup, typically implemented via Red-Black Trees or Hash Tables.

      • std::set: Key is the value; unique elements.
      • std::map: Separate key and value; unique keys.
      • std::multiset / std::multimap: Allows duplicate elements.
      • std::unordered_*: Implemented using Hash Tables with Separate Chaining (C++11).

  5. Overview of Barlow Twins Self-Supervised Learning

    Barlow Twins is a self-supervised learning method based on information theory designed to reduce redundancy between neurons. The goal is to ensure that neurons are invariant to data augmentations but remain independent of each other. This is achieved by training a neural network such that the cross-correlation matrix of the feature representations of two augmented versions of the same input approaches the identity matrix (diagonal elements close to 1, off-diagonal elements close to 0).

  6. Overview of Chain of Manipulations (CoM) for Visual Reasoning

    Chain of Manipulations (CoM) is a method to improve fine-grained visual reasoning in VLMs. Instead of directly providing an answer, the model performs a sequence of iterative image manipulations (e.g., marking regions, zooming, cropping) and outputs intermediate results (e.g., bounding boxes, local images) to reach a final conclusion, mimicking human problem-solving behavior.

  7. Overview of GAGAvatar Architecture

    GAGAvatar is designed for single-image driven animatable head avatar reconstruction. It addresses the slow rendering of NeRF-based methods and the lack of generalizability in standard 3D Gaussian Splatting (3DGS) methods (which usually require per-identity training). It utilizes a Dual-lifting method combined with 3DMM priors to construct 3D Gaussian models from a single image, enabling real-time rendering and cross-identity expression transfer.

  8. Overview of the Portrait4D-v2 Learning Framework

    Portrait4D-v2 is a framework designed to generate animatable 3D head representations (Tri-plane TT) from a single source image. It aims to reduce dependency on inaccurate 3DMM reconstructions by using a pseudo-multi-view video learning approach. The process consists of two main stages: first, learning a static 3D head synthesizer to convert monocular videos into multi-view data, and second, activating motion-related components to learn a full 4D head synthesizer via cross-view self-reenactment.

  9. Overview of the Barlow Twins self-supervised learning method

    Barlow Twins is a self-supervised learning approach based on information theory designed to reduce redundancy between neurons. The core objective is to make the network invariant to data augmentations while ensuring that different neurons remain independent. This is achieved by training the network so that the cross-correlation matrix of the feature representations from two augmented versions of the same input approaches the identity matrix: diagonal elements should be close to 1 (invariance), and off-diagonal elements should be close to 0 (redundancy reduction).

  10. Overview of GVHMR for World-Grounded Human Motion Recovery

    GVHMR is designed to recover human motion in a world coordinate system from monocular video. Unlike traditional methods that recover motion in camera space (which varies with camera movement) or autoregressive methods (which suffer from error accumulation), GVHMR introduces a Gravity-View (GV) coordinate system. This system is naturally aligned with gravity and provides a unique definition for every frame, reducing ambiguity when mapping images to poses. Key contributions include the GV coordinate system, a Transformer-based architecture using Rotary Positional Embeddings (RoPE) for long sequences, and the prediction of stillness labels for hands and feet to correct foot sliding and global trajectory errors.

  11. Use Siamese Neural Networks for similarity assessment

    Siamese networks consist of two identical sub-networks sharing the same weights and biases. They are used to extract feature representations from two inputs and compare them to determine similarity.

    There are two primary ways to evaluate similarity:

    1. Cosine Similarity: Calculates the cosine of the angle between two feature vectors (range -1 to 1). It is computationally fast as it uses normalization and dot products without requiring a Multi-Layer Perceptron (MLP).
    2. Learned Similarity: Uses a trained neural network (typically an MLP) to map feature vectors to a scalar similarity score. This can capture complex non-linear relationships but is computationally slower than cosine similarity.

  12. CogCoM Model Architecture

    CogCoM is based on the CogVLM architecture with a total of approximately 17B parameters. It consists of four core components:

    1. Visual Encoder: EVA2-CLIP-E (4B parameters) for feature extraction.
    2. MLP Adapter: A two-layer SwiGLU MLP that maps visual features to the language space.
    3. LLM Backbone: Vicuna-7B-v1.5 for language understanding and generation.
    4. Visual Expert Module: Integrated into the attention and FFN layers of the LLM, adding 6.5B parameters for deep visual feature fusion.