PyAudioWPatch

repository·master·Indexed 42 days ago

https://github.com/s0d3s/pyaudiowpatch

A fork of PyAudio that enables WASAPI loopback recording on Windows, allowing developers to record audio directly from output devices like speakers or Bluetooth headphones. It supports Python versions 3.7 through 3.13 and includes support for WMME, WASAPI, and DirectX (DSound) APIs, while introducing context manager support for PyAudio and Stream objects.

Tokens
5.1K
Snippets
19
Records
29
Agent score
53%

What's inside PyAudioWPatch

  1. What is the WDM-KS host API and when to use it

    master

    The WDM-KS (Windows Driver Model - Kernel Streaming) host API provides a low-latency audio path by connecting PortAudio directly to the Windows Kernel Streaming API.

    Key Characteristics

    • Latency: Offers latency comparable to ASIO drivers, avoiding the mixing penalties associated with MME and DirectX.
    • Compatibility: Works with any Windows audio hardware available via MME/DirectX without requiring specialized ASIO drivers.
    • Exclusivity: Using the KS interface blocks the device from being used by other applications via higher-level APIs (and vice versa). If the system is using a device, the KS API cannot access it.

    When to use it

    Use WDM-KS only if your application has a strict requirement for very low latency audio (typically <20ms), such as:

    • Real-time sound generation based on user input.
    • Real-time audio processing of recorded streams.

    Best Practice: Microsoft recommends keeping the device open only when your application has focus. In PortAudio, this means only having a stream Open on a WDMKS device while the application is active.

  2. How WASAPI loopback recording works in PyAudioWPatch

    master

    PyAudioWPatch extends the Windows Audio Session API (WASAPI) to allow recording from output devices (speakers) by using their loopback analogues.

    Key Concept: WASAPI loopback devices are treated as input devices. In the device list, loopback devices appear as duplicates of the original output devices at the end of the list. To record what you hear through your speakers, you must select the loopback version of that device rather than the output device itself.

  3. Compile WASAPI API with Microsoft Visual Studio

    master

    To compile the WASAPI API using Microsoft Visual Studio (2005 SP1 or higher), no specific configuration is required within the IDE. However, you must ensure the appropriate Windows SDK is installed on your system prior to compilation:

    • For basic WASAPI functionality: Install at least the Windows Vista SDK (v6.0A).
    • For WASAPI functionality specific to Windows 8 and higher: Install at least the Windows 8 SDK.
  4. Verify WASAPI loopback support

    master

    To confirm that the WASAPI loopback functionality was successfully integrated during the build process:

    1. During the build step, use make loopback instead of the standard make command.
    2. Once the build completes, navigate to the ./bin directory at the PortAudio root.
    3. Run the executable paloopback.exe.
    4. Check the output list of devices. If you see devices with the [Loopback] suffix, the build was successful. If not, WASAPI loopback will not function.
    make loopback
    ./bin/paloopback.exe
  5. Link PortAudio to your Visual Studio project

    master

    Once PortAudio is built, integrate it into your own project by:

    1. Linking: Link against portaudio_x86.lib or portaudio_x64.lib depending on your target architecture.
    2. Headers: Include the necessary headers:
      • portaudio.h
      • pa_asio.h (if using ASIO)
      • pa_x86_plain_converters.h (if using x86 plain converters)
    3. Runtime: Ensure portaudio_xXX.dll is available in your execution environment.
    // Example include directives
    #include "portaudio.h"
    #include "pa_asio.h"
    #include "pa_x86_plain_converters.h"
  6. Use Callback Mode Audio I/O

    master

    Callback mode is used to generate audio data on the fly or process recorded audio data immediately. PyAudio calls a user-defined callback function in a separate thread whenever new audio data is needed (for playback) or available (for recording).

    Callback Signature: The callback function must follow this signature: callback(input_data, frame_count, time_info, status_flag)

    Return Value: The callback must return a tuple containing:

    1. frame_count frames of audio data.
    2. A flag signifying whether there are more frames to play/record.

    Workflow:

    1. Define the callback function.
    2. Open a stream using pyaudio.PyAudio.open() and provide the callback function.
    3. Start the stream using pyaudio.Stream.start_stream(). This triggers the repeated calls to your callback.
    4. The callback loop continues until it returns pyaudio.paComplete.
    5. Important: The main thread must remain active (e.g., using time.sleep()) to prevent the process from terminating while the callback thread is running.
    # Based on play_wave_callback.py logic
    def callback(in_data, frame_count, time_info, status):
        # Return (data, status)
        return (data, pyaudio.paContinue)
    
    pyaudio = pyaudio.PyAudio()
    stream = pyaudio.open(format=pyaudio.paInt16, channels=1, rate=44100, output=True, callback=callback)
    stream.start_stream()
    # Keep main thread alive
    while stream.is_active():
        time.sleep(0.1)
  7. Use Blocking Mode Audio I/O

    master

    In blocking mode, audio operations wait for the requested frames to be processed before continuing execution. This is suitable for simple playback or recording tasks where you do not need to generate or process audio data on the fly.

    Workflow:

    1. Instantiate pyaudio.PyAudio() to set up the PortAudio system.
    2. Open a stream using pyaudio.PyAudio.open() with your desired device and parameters to get a pyaudio.Stream object.
    3. Use pyaudio.Stream.write() to play audio data or pyaudio.Stream.read() to record audio data. Each call blocks until the frames are processed.
    4. Use pyaudio.Stream.stop_stream() to pause and pyaudio.Stream.close() to terminate the stream.
    5. Call pyaudio.PyAudio.terminate() to end the PortAudio session.
    # Based on play_wave.py logic
    pyaudio = pyaudio.PyAudio()
    stream = pyaudio.open(format=pyaudio.paInt16, channels=1, rate=44100, output=True)
    stream.write(data)
    stream.stop_stream()
    stream.close()
    pyaudio.terminate()
  8. Build the PortAudio fork for WASAPI loopback (Windows x64)

    master

    This fork of PortAudio is specifically used to add WASAPI loopback functionality to PyAudio. To build it on Windows x64, follow these steps:

    1. Install Cygwin and ensure the following packages are installed via the installer:
      • autoconf, automake, cmake, make, libtool, mingw64-x86_64-gcc-g++ (or mingw64-i686-gcc-g++ for 32-bit).
    2. Fix Python version mismatch (Only required if using python <= 3.6):
      • Open path/to/python/Lib/distutils/cygwinccompiler.py.
      • Replace gcc -dumpversion with gcc --version.
    3. Configure and Build:
      • Navigate to the PortAudio root directory in Cygwin.
      • Run the configuration command:
        ./configure --with-winapi=wasapi --enable-shared=no --host=x86_64-w64-mingw32
        (Note: For 32-bit systems, use i686 instead of x86_64 in the host flag).
      • Build the library:
        make onlylib
    4. Rebuilding: If you need to rebuild, run make clean followed by make.
    ./configure --with-winapi=wasapi --enable-shared=no --host=x86_64-w64-mingw32
    make onlylib
  9. Compile WASAPI API with MinGW (GCC 32/64-bit)

    master

    To compile the WASAPI API using MinGW, you must include the mingw-include directory in your build process. This directory contains modified files derived from the Windows Vista SDK (v6.0A) that are specifically adjusted for compatibility with the MinGW compiler.

    Tested Minimum Versions:

    • MinGW (32-bit): gcc version 4.4.0 (GCC)
    • MinGW64 (64-bit): gcc version 4.4.4 20100226 (prerelease) (GCC)
  10. Install PyAudioWPatch

    master

    Install the library using pip. Precompiled wheels are available for Windows and support Python versions 3.7 through 3.13. The installation includes support for WMME, WASAPI, and DirectX (DSound) APIs.

    pip install PyAudioWPatch
  11. Build PortAudio with ASIO support in Visual Studio

    master

    To build PortAudio (v19-devel) into a Win32 DLL and lib file with ASIO support, follow these steps:

    1. Prepare ASIO SDK: Copy the source directories from your ASIO SDK into src\hostapi\asio\ASIOSDK. The structure should look like this:

      • src\hostapi\asio\ASIOSDK\common
      • src\hostapi\asio\ASIOSDK\host
      • src\hostapi\asio\ASIOSDK\host\sample
      • src\hostapi\asio\ASIOSDK\host\pc (Note: The driver directory is not required).
    2. Open Project:

      • For Visual Studio 2005 or later: Open the portaudio.sln file. This contains two projects (portaudio and portaudio_static) with configurations for Win32/x64 in Debug, Release, and ReleaseMinDependency (which minimizes OS system DLL dependencies).
      • For Visual Studio 7.0 or 7.1: Open portaudio.dsp and convert it if necessary.
      • For Visual Studio 6.0: Ensure you have Service Pack 5, the Processor Pack (for SSE intrinsics), the Platform SDK (Feb 2003), and the DirectX 9.0 SDK (Summer 2003) installed and registered with Visual Studio.
    3. Compile: Run the build command in Visual Studio.

    Open portaudio.sln and hit compile.