MNN (Mobile Neural Network)

The MNN Python extension consists of two main parts:

1. MNN: Handles inference, training, image processing, and numerical computation. Key sub-modules include:
   • MNN.expr: Expression engine
   • MNN.nn: Neural network components (loss, compress)
   • MNN.data: Data handling
   • MNN.optim: Optimization
   • MNN.cv: Computer vision
   • MNN.numpy: NumPy-like operations (linalg, random)
2. MNNTools: Wraps MNN utilities, primarily providing mnn, mnnconvert, and mnnquant.

MNN Chat is a full-featured multimodal Large Language Model (LLM) Android application. It allows users to interact with various models locally on their devices, ensuring data privacy.

Key Features:

• Multimodal Support: Includes text-to-text, image-to-text, audio-to-text, and text-to-image (based on diffusion models).
• High Performance: Optimized for CPU inference on Android. MNN-LLM demonstrates significantly faster pre-filling and decoding speeds compared to llama.cpp and fastllm.
• Wide Model Compatibility: Supports models from providers such as Qwen, Gemma, Llama (including TinyLlama and MobileLLM), Baichuan, Yi, DeepSeek, InternLM, Phi, ReaderLM, and SmolLM.
• Local Execution: All processing happens on-device without uploading information to external servers.

MNN-TaoAvatar provides a full-featured, 100% offline, and privacy-focused multimodal AI avatar experience on Android. It integrates several local AI components:

• Conversational AI: Powered by a local Large Language Model (LLM).
• Speech-to-Text: Uses embedded Automatic Speech Recognition (ASR) models.
• Voice Synthesis: On-device Text-to-Speech (TTS).
• Avatar Behavior Animation: Uses Audio-to-BlendShape (A2BS) to animate the avatar based on audio input.
• Real-time Neural Rendering (NNR): Provides expressive, real-time visual rendering of the avatar.

MNN is a lightweight deep neural network engine designed for both inference and training. It is optimized for a wide range of devices, including servers, PCs, mobile phones, and embedded systems. MNN is widely used in large-scale production environments (e.g., Alibaba's Taobao, Tmall, and Youku) for scenarios like live streaming, short video, search recommendation, and security risk control.

Key capabilities include:

• Versatility: Supports mainstream formats like TensorFlow, Caffe, ONNX, and Torchscripts.
• High Performance: Optimized for various CPU architectures (x86, ARM) using SIMD and hand-written assembly, and supports GPU acceleration via Metal, OpenCL, Vulkan, and CUDA.
• Lightweight: Minimal dependencies, making it ideal for mobile and embedded deployment.

MNN is an efficient and lightweight deep learning inference engine. It supports a wide range of model architectures, including:

• CNN (Convolutional Neural Networks)
• Transformer
• LLM (Large Language Models)
• Diffusion models

It is designed to run across multiple platforms, including mobile devices and servers.

MNN-TaoAvatar is a mobile application designed for local, fully offline, multi-modal interactive smart digital humans. It integrates several advanced technologies directly onto the mobile device, including:

• Local Chatbot: Powered by a locally running Large Language Model (LLM).
• Speech Recognition (ASR): Built-in models to convert speech to text in real-time.
• Speech Synthesis (TTS): Models that allow the digital human to speak with natural and realistic voices.
• Audio-to-Body/Speech (A2BS): Technology that automatically generates rich facial expressions and movements driven by audio.
• Neural Rendering (NNR): Provides detailed and realistic facial expressions for enhanced interaction.
• 100% Offline Operation: All processes run locally on the device to ensure privacy and offline availability.

MNN is a lightweight deep learning engine designed for inference and training. It is optimized for deployment on servers, PCs, mobile devices (iOS/Android), and various embedded devices. It is widely used in large-scale production environments for scenarios like live streaming, short video, search recommendation, and image search.

Key capabilities include:

• Lightweight: Minimal dependencies; small binary sizes (e.g., ~800KB for Android shared library; ~12MB for iOS static library).
• Versatile: Supports TensorFlow, Caffe, ONNX, and Torchscripts formats, as well as CNN, RNN, GAN, and Transformer architectures.
• High Performance: Optimized with SIMD, hand-written assembly, and algorithms like Winograd and Strassen for various CPU/GPU architectures.
• Ease of Use: Supports Python calls, provides an MNN CV module for image processing, and supports on-device training.

sherpa-onnx provides local execution for a wide range of speech and audio processing tasks, including:

• Speech-to-text (ASR): Both streaming and non-streaming modes.
• Text-to-speech (TTS)
• Speaker Analysis: Diarization, identification, verification, and spoken language identification.
• Audio Processing: Audio tagging, VAD (Voice Activity Detection, e.g., silero-vad), and Keyword spotting.

MNN-LLM is an LLM inference engine built on top of MNN that supports mainstream open-source Large Language Models. It consists of two primary functional components:

1. Model Export: A process to convert PyTorch models into ONNX and subsequently into MNN models, while also exporting necessary tokenizer, embedding, and configuration files.
2. Model Inference: The runtime component that supports the inference of exported models and enables text generation.