TorchSharp

The following breaking changes were introduced in version 0.95.4:

• Property Renaming: The Weight and Bias properties on some modules were renamed to lowercase weight and bias.
• LRScheduler: The LRScheduler.LearningRate property was removed. To log the learning rate, retrieve it directly from the optimizer currently in use.

If you need a dynamic number of submodules (e.g., to parameterize architecture or avoid defining many fields), use ModuleList.

Important: Unlike Sequential, calling .forward() on a ModuleList will throw an exception. You must manually iterate through the modules within your custom module's forward() implementation.

private class TestModule1 : Module<Tensor, Tensor>
{
    public TestModule1()
        : base("TestModule1")
    {
        RegisterComponents();
    }

    public override Tensor forward(Tensor input)
    {
        for (int i = 0; i < submodules.Count; i++) {
            input = submodules[i].forward(input); 
        }
        return input;
    }

    private ModuleList submodules = new ModuleList(Linear(100, 10), Linear(10, 5));
}

For more granular debugging, especially when working with Recurrent Neural Networks (RNNs), you can drill down into specific statistics via:

• DisposeScopeManager.Statistics.TensorStatistics
• DisposeScopeManager.Statistics.PackedSequenceStatistics

Note that a PackedSequence uses tensors internally. These internal tensors are immediately detached from any active scope, which will increment the DetachedFromScopeCount property. When the PackedSequence is disposed, its internal tensors are also disposed.

The following breaking changes were introduced in version 0.96.3:

• Optimizers: All APIs to create optimizers now require both parameters() and named_parameters().
• Parameter Groups: Support for parameter groups was added to most optimizers and LR schedulers.

When deriving from OptimizerState (starting in v0.101.3), you must follow these two requirements to maintain correct linkage and state management:

1. Pass Parameter to Base Constructor: You must explicitly pass the related torch.nn.Parameter object to the OptimizerState base constructor.
2. Implement Initialize: You must implement an Initialize function to set up the state's properties. Since this function can be called for re-initialization, ensure you properly dispose of any existing tensor objects before re-assigning them to prevent memory leaks.

The following breaking changes were introduced in version 0.96.0:

• Return Types: Module.named_parameters(), parameters(), named_modules(), and named_children() now return IEnumerable instances instead of arrays.
• Naming Convention: Methods were lower-cased to match PyTorch conventions: Module.Train $\rightarrow$ Module.train and Module.Eval $\rightarrow$ Module.eval.

If a module requires tensors that are not updated during back-propagation (e.g., a dropout mask), you should declare them as 'buffers'.

To ensure they are properly registered for tasks like disk storage, declare them as fields of type Tensor (rather than Parameter). This ensures they are correctly handled when RegisterComponents() is called.

TorchSharp uses a DisposeScope system to manage the lifecycle of native resources like Tensors. To avoid memory leaks and ensure proper disposal, you should use torch.NewDisposeScope() instead of manually constructing dispose scopes.

Key features:

• Attachment: Use Attach() to add objects to the current scope.
• Accessing Current Scope: You can retrieve the active scope using torch.CurrentDisposeScope.
• LIFO Order: Scopes can be disposed out of LIFO (Last-In, First-Out) order.
• Statistics: DisposeScopeManager.Statistics provides metrics for debugging memory leaks, including DisposedOutsideScopeCount, AttachedToScopeCount, and ThreadTotalLiveCount.