teddygood Technical Wiki

OrchestrationBench is a benchmark designed to evaluate the 'orchestration' capabilities of LLM agents when handling real-world service requests. Unlike standard Q&A benchmarks, it focuses on the agent's ability to decompose complex user requests into manageable workflows and execute tools while adhering to specific service constraints. It supports both English and Korean, covering 17 domains with approximately 100 virtual tools. The benchmark separates evaluation into three distinct areas: workflow planning, call/reject decisions (constraint-aware execution), and function calling.

vLLM is a serving system designed to optimize LLM throughput by redefining KV cache management as a memory allocation problem. It introduces PagedAttention, which applies the concepts of virtual memory and paging to the KV cache. Instead of storing KV caches in contiguous buffers—which leads to waste via reserved slots and fragmentation—vLLM divides the KV cache into fixed-size blocks. A block table maps logical blocks to physical blocks in GPU memory, allowing the cache to be non-contiguous. This approach increases GPU memory utilization (up to 96.3% compared to 20.4%-38.2% in traditional systems) and enables larger batch sizes, resulting in 2-4x higher throughput than systems like FasterTransformer or Orca at similar latency levels.

Black is an "uncompromising" Python code formatter that automatically rewrites code into a consistent style. Unlike tools that offer extensive styling options, Black focuses on ensuring that the same input always results in the same output, reducing manual debates over code formatting. It is commonly used via CLI, editor integrations, pre-commit hooks, and CI pipelines.

Amazon Bedrock is a fully managed generative AI platform from AWS that provides access to various models from partners including AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, and Stability AI. Key features include a web-based playground for testing models, prompt engineering tools, and the ability to implement generative AI models via code.

The book 'GAN 첫걸음' (authored by Tariq Rashid) is an introductory guide to Generative Adversarial Networks (GANs). It focuses on providing a solid foundation through a combination of conceptual explanations and step-by-step implementations using PyTorch. The content is structured into three parts: Part 1 covers basic concepts and foundations; Part 2 focuses on core GAN concepts and building a model; and Part 3 explores advanced applications such as Convolutional GANs and Conditional GANs.

This book is a Korean translation of 'Python Machine Learning 3rd Edition' by Sebastian Raschka and Vahid Mirjalili. It serves as a comprehensive textbook covering both theoretical concepts and practical implementation of machine learning algorithms, including advanced topics like GANs and Transformers. A notable feature is the implementation of algorithms from scratch using only NumPy to deepen conceptual understanding.

This book is a comprehensive guide for beginners to develop, deploy, and operate a web project using Django and Bootstrap. It focuses on a 'learning by doing' approach, guiding the reader through the creation of a community site featuring CRUD functionality, user authentication, and comments. Key technical areas covered include the MTV pattern, Function-Based Views (FBV), Class-Based Views (CBV), and database relationships (One-to-Many, Many-to-Many).

PyPy is an alternative implementation and execution environment for the Python language. It allows developers to run the same Python code as CPython but uses a different runtime. Its primary goals are to maintain high compatibility with CPython and to improve the execution speed of pure Python code using a tracing Just-In-Time (JIT) compiler.

vLLM is a serving system designed to increase LLM throughput by optimizing KV cache memory management. It introduces PagedAttention, which treats the KV cache as a memory management problem rather than a kernel speed problem. By dividing the KV cache into fixed-size blocks and using a block table to map logical blocks to physical blocks, vLLM eliminates the need for contiguous memory allocation. This approach significantly reduces memory waste caused by reserved slots, internal fragmentation, and external fragmentation, allowing for larger batch sizes and 2-4x higher throughput compared to systems like FasterTransformer or Orca at similar latency levels.

The book 'Make Your Own Neural Network' (Korean title: 신경망 첫걸음) by Tariq Rashid is an introductory guide to deep learning and neural network principles. It is specifically designed to be accessible to beginners, including those with limited mathematical backgrounds, by replacing complex formulas with simple arithmetic and extensive visual aids. The book transitions from conceptual explanations to practical examples implemented in a Jupyter Notebook (or IPython) environment.