repository·master·Indexed 12 days ago
https://github.com/benfred/py-spyA low-overhead sampling profiler for Python programs (v0.4.2) that allows developers to visualize performance bottlenecks without restarting the program or modifying source code. It includes subcommands for real-time monitoring (top), recording profiles to SVG flamegraphs or speedscope files (record), and dumping call stacks (dump). Supports profiling native C/C++ extensions, subprocesses, and GIL-holding threads.
Alpine Python does not support standard manylinux wheels via pip. To install py-spy on Alpine, you can either:
echo 'manylinux1_compatible = True' > /usr/local/lib/python3.7/site-packages/_manylinux.py2. **Download a musl binary** directly from the [GitHub releases page](https://github.com/benfred/py-spy/releases).
You can install py-spy using several methods depending on your environment:
libunwind on Linux and Windows).# Via pip
pip install py-spy
# Via cargo (builds from source)
cargo install py-spy
# Via Homebrew (macOS)
brew install py-spy
# Via AUR (Arch Linux)
yay -S py-spy
# Via Alpine Linux
apk add py-spy --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/testing/ --allow-untrustedKubernetes drops the SYS_PTRACE capability by default. To profile a container, you must add this capability to the security context in your Deployment spec.
Deployment Configuration:
securityContext:
capabilities:
add:
- SYS_PTRACEUsing Ephemeral Containers:
You can also use kubectl debug to attach an ephemeral container with the necessary permissions to a running Pod:
kubectl debug --profile=general \
-n your-namespace \
--target=app-container-name \
pod-name \
--image=python:3.12-slim \
-it -- bashRunning py-spy in Docker requires elevated permissions because it uses the process_vm_readv system call. You must grant the SYS_PTRACE capability.
Using Docker CLI:
docker run --cap-add SYS_PTRACE ...Using Docker Compose:
your_service:
cap_add:
- SYS_PTRACEThe Sampler is the core engine that periodically captures stack traces from Python processes. It implements Iterator, meaning you can consume it by calling .next() or using it in a loop.
Each iteration yields a Sample object, which contains:
traces: A vector of StackTrace objects captured during that interval.sampling_errors: An optional list of errors encountered while sampling specific PIDs, formatted as (Pid, Error).late: An optional Duration indicating if the sampling was delayed (jitter/latency in the sampling timer).If the subprocesses configuration is enabled, the Sampler automatically monitors the process tree and includes traces from any new Python child processes it discovers.
// Conceptual usage of the Sampler iterator
for sample in sampler {
for trace in sample.traces {
// Process stack traces
}
if let Some(errors) = sample.sampling_errors {
// Handle sampling errors
}
}A StackTrace represents the call stack for a single Python thread. It contains metadata about the thread's state and the sequence of function calls (frames) that led to the current point.
Key fields include:
pid: The process ID.thread_id: The Python thread ID.os_thread_id: The operating system thread ID.active: Boolean indicating if the thread was active.owns_gil: Boolean indicating if the thread held the Global Interpreter Lock (GIL).frames: A vector of Frame objects representing the call stack.process_info: Metadata about the process command line and parent process.You can determine the thread's status using the status_str() method, which returns one of: "idle", "active+gil", or "active".
match stack_trace.status_str() {
"idle" => println!("Thread is waiting"),
"active+gil" => println!("Thread is running and holds the GIL"),
"active" => println!("Thread is running but does not hold the GIL"),
_ => {}
}The way py-spy interacts with the target Python process is determined by the locking strategy. This is primarily controlled via the --nonblocking flag.
py-spy pauses the Python process to take samples. This ensures high accuracy and complete stack traces.--nonblocking flag, py-spy does not pause the process. This reduces the performance impact on the target application but can lead to incorrect results, such as partial stack traces or a higher sampling error rate.The dump command provides a snapshot of the current Python stack traces. The output includes:
subprocesses configuration is enabled, the dump will recursively include stack traces for child processes.A Frame represents a single function call within a StackTrace. It provides details about the function being executed and its context.
Key fields include:
name: The function name (or qualname if available).filename: The full path to the file containing the function.line: The line number within the file (0 if line information is unavailable).locals: An optional list of LocalVariable objects associated with this frame.is_entry: Indicates if this is an entry frame (corresponds to a native frame in Python 3.11+).is_shim_entry: Used in Python 3.12+ to identify shim frames inserted before Python blocks.By using the --native flag, py-spy can collect stack traces from native extensions written in Cython, C, or C++.
Important Constraints:
--native with the --nonblocking option, as py-spy must pause the process to collect native stack traces accurately.--subprocesses option.dump command, you can output the captured stack traces in a structured JSON format instead of the default human-readable text. This is useful for programmatic analysis of the trace data. This behavior is controlled by the dump_json configuration flag.A LocalVariable describes a variable within a specific Frame.
Key fields include:
name: The name of the variable.addr: The memory address of the variable.arg: Boolean indicating if the variable is a function argument.repr: An optional string representation of the variable's value.