You can extend Marble with new encoders using either Internal or External extension modes.
Internal Extension
Implement your encoder by subclassing BaseAudioEncoder within the marble/encoders/ directory.
from marble.core.base_encoder import BaseAudioEncoder
class MyEncoder(BaseAudioEncoder):
def __init__(self, arg1, arg2):
super().__init__()
# initialize your model
def forward(self, waveforms):
# return List[Tensor] of shape (batch, layer, seq_len, hidden_size)
# or return a dict of representations
Then, reference it in your YAML configuration:
model:
encoder:
class_path: marble.encoders.my_encoder.MyEncoder
init_args:
arg1: 123
arg2: 456
External Extension
Place your encoder file anywhere in your project (e.g., ./my_project/my_encoder.py) and use the full Python import path in your YAML configuration:
model:
encoder:
class_path: my_project.my_encoder.MyEncoder
init_args:
arg1: 123
Custom Transforms
If your encoder requires specific transforms, you can register them in the YAML:
- Embedding-level transforms: Subclass
BaseEmbTransform and register under emb_transforms. - Audio preprocessing: Subclass
BaseAudioTransform and register under audio_transforms (can be specific to train or general).
emb_transforms:
- class_path: marble.modules.transforms.MyEmbTransform
init_args:
param: value
audio_transforms:
train:
- class_path: marble.modules.transforms.MyAudioTransform
init_args:
param: value
from marble.core.base_encoder import BaseAudioEncoder
class MyEncoder(BaseAudioEncoder):
def __init__(self, arg1, arg2):
super().__init__()
def forward(self, waveforms):
# return List[Tensor] of shape (batch, layer, seq_len, hidden_size)
pass
