Scalismo

repository·main·Indexed 42 days ago

https://github.com/unibas-gravis/scalismo

A Scala-based library for statistical shape modeling and model-based image analysis. Designed for both research prototyping and industrial-scale distributed imaging pipelines, it provides functionality for image registration, discrete and continuous image representations, and parametric transformations including rigid and similarity transforms.

Tokens
13.9K
Snippets
50
Records
57
Agent score
54%

What's inside Scalismo

  1. Overview of Scalismo

    main
    Scalismo is a Scala library designed for statistical shape modeling and model-based image analysis. It is built to support a range of use cases from rapid research prototyping to full-scale industrial applications and deployment in complex, distributed imaging pipelines. The library's design for image registration is influenced by ITK and Elastix, aiming to provide core functionality without the high complexity typically associated with those toolkits.
  2. Combine transformations using composition

    main

    You can compose two transformations $t_O$ and $t_I$ to create a CompositeTransformation representing $t_O(t_I(x))$.

    • Use CompositeTransformation(t1, t2) for standard composition.
    • Use CompositeDifferentiableTransformation(t1, t2) if the resulting transformation must be differentiable with respect to position.

    To compose multiple spaces, use ProductTransformationSpace or ProductTransformationSpaceWithDifferentiableTransforms. This allows you to generate a single composite transformation from a single concatenated parameter vector.

    // Basic composition
    val rotationThenTranslation = CompositeTransformation(translation, rotation)
    
    // Composition that preserves position differentiability
    val rotationThenTranslationDifferentiable = CompositeDifferentiableTransformation(translation, rotation)
    
    // Composing spaces
    val compositeSpace = ProductTransformationSpace(TranslationSpace3D, RotationSpace3D(Point3D(0, 0, 0)))
    val ct : CompositeTransformation[_3D, Translation, Rotation] = compositeSpace.transformationForParameters(DenseVector(0.0, 0.0, 0.0, 0.0, 0.0, 0.0))
  3. Use TransformationSpaces for optimization

    main

    A TransformationSpace is a factory associated with a parametric transformation. It is primarily used for optimization tasks (e.g., searching for a transformation that minimizes a cost function).

    Key capabilities of a TransformationSpace:

    • identityTransformation: Returns the identity transformation.
    • transformationForParameters(parameters): Returns a transformation corresponding to a specific set of parameters (e.g., a DenseVector).
    import breeze.linalg.DenseVector
    
    val translationSpace = TranslationSpace3D
    val parameters = DenseVector(1.0, 0.0, 3.0)
    
    // Create a specific transformation from parameters
    val translationFromParameters = translationSpace.transformationForParameters(parameters)
    
    // Get the identity transformation
    val ident = translationSpace.identityTransformation
  4. Interpolate and discretize images

    main

    To move between discrete and continuous representations:

    Continuous Representation (Interpolation)

    Use the .interpolate method on a discrete image and provide an interpolator (e.g., BSplineImageInterpolator3D). This returns a Field defined on all points within the image's bounding box.

    Discrete Representation (Discretization)

    To convert a continuous Field back into a discrete image, use the .discretize method. You must provide a new DiscreteImageDomain and an outsideValue to be assigned to points that fall outside the continuous field's domain.

    // 1. Interpolate: Discrete -> Continuous
    val continuousImage: Field[_3D, Short] = image.interpolate(BSplineImageInterpolator3D(degree = 3))
    
    // 2. Discretize: Continuous -> Discrete
    val newDomain = DiscreteImageDomain3D(image.domain.boundingBox, size = IntVector(32, 32, 32))
    val resampledImage: DiscreteImage[_3D, Short] = continuousImage.discretize(newDomain, outsideValue = 0.toShort)
  5. Use StructuredPoints for regular grid point sets

    main

    The StructuredPoints[D] class defines a set of points in $D$ dimensions aligned on a regular grid. The grid is defined by an origin, a spacing (distance between points in mm), and a size (number of points in each direction). This is useful for representing discrete image domains or regular sampling grids.

    Key capabilities include:

    • Mapping between grid indices (IntVector[D]) and physical coordinates (Point[D]).
    • Mapping between a unique PointId and physical coordinates.
    • Finding the closest grid point to an arbitrary physical point.
    • Determining if a physical point belongs to the grid.
    • Calculating the boundingBox of the grid.
    // Example of the conceptual interface
    val points: StructuredPoints[_2D] = StructuredPoints(origin, spacing, size)
    val physicalPt = points.indexToPoint(IntVector(10, 20))
    val closest = points.findClosestPoint(someArbitraryPoint)
    val id = points.pointId(physicalPt)
  6. Interpolate and blend RGB colors

    main

    Scalismo provides interpolation capabilities for RGB colors through the RGBInterpolator (an implicit ValueInterpolator[RGB]).

    • blend(obj1, obj2, l): Linearly interpolates between two RGB colors based on a factor l (where l=0 is obj1 and l=1 is obj2).
    • average(first, rest*): Calculates the arithmetic mean of multiple RGB colors.
    • barycentricInterpolation(v1, f1, v2, f2, v3, f3): Performs barycentric interpolation using three colors and their respective weights.
    import scalismo.color.RGB
    
    val c1 = RGB(1.0, 0.0, 0.0)
    val c2 = RGB(0.0, 1.0, 0.0)
    
    // Linear interpolation
    val mid = RGBInterpolator.blend(c1, c2, 0.5)
    
    // Averaging
    val avg = RGBInterpolator.average(c1, c2, RGB(0.0, 0.0, 1.0))
  7. Understand ClosestPointWithType details

    main

    When using getClosestPointOnSurface, the returned ClosestPointWithType describes the specific geometric entity where the closest point resides. For TriangleMesh3D implementations, this can be one of the following:

    • ClosestPointIsVertex: The closest point is a vertex of the mesh. Includes the PointId of that vertex.
    • ClosestPointOnLine: The closest point lies on an edge. Includes the PointIds of the two vertices forming the edge and the barycentric coordinate along that line.
    • ClosestPointInTriangle: The closest point is within the face of a triangle. Includes the TriangleId and the BarycentricCoordinates of the point within that triangle.
  8. Representing structured points with StructuredPoints

    main

    In Scalismo, a set of points on a regular grid is represented by the StructuredPoints class. This allows for efficient access to domain points by implicitly representing the grid via a mathematical formula rather than explicit storage.

    Grids are aligned to coordinate axes by default, but you can specify a rotation using Euler angles (yaw, pitch, roll).

    val origin = Point3D(10.0, 15.0, 7.0)
    val spacing = EuclideanVector3D(0.1, 0.1, 0.1)
    val size = IntVector3D(32, 64, 92)
    
    // Create an axis-aligned grid
    val points = StructuredPoints3D(origin, spacing, size)
    
    // Create a rotated grid using Euler angles
    val yaw = Math.PI / 2
    val pitch = 0.0
    val roll = Math.PI
    val points2 = StructuredPoints3D(origin, spacing, size, yaw, pitch, roll)
  9. Create a Transformation

    main

    A Transformation[D] is a function that maps points from a domain $D \subset \mathbb{R}^d$ to other points in $\mathbb{R}^d$. You can create a transformation by specifying a function f : Point[D] => Point[D] and its domain. If no domain is specified, the transformation is assumed to be defined on the entire EuclideanSpace.

    ParametricTransformations are a subclass of transformations defined by a finite set of parameters (e.g., a translation vector or Euler angles).

    import scalismo.common._
    import scalismo.geometry._
    import scalismo.transformations._
    
    // Creating a transformation with an explicit domain
    val domain : Domain[_3D] = EuclideanSpace[_3D]
    val myTransform = Transformation3D(domain, p => p + EuclideanVector3D(1, 0, 0))
    
    // Creating a parametric transformation (Translation)
    val translation = Translation3D(EuclideanVector3D(1, 3, 0.5))
    
    // Creating a parametric transformation (Rotation with center)
    val rotation = Rotation3D(0, 0.1, 0.3, Point3D(0, 0, 0))
  10. Invert and differentiate transformations

    main

    All ParametricTransformations are differentiable with respect to their parameters. Additionally, transformations can be made invertible or differentiable with respect to their position in space by implementing specific traits:

    • Invertibility: Implement CanInvert[D, T] to access the .inverse method.
    • Position Differentiation: Implement CanDifferentiateWRTPosition[D] to access the .derivativeWRTPosition method (which returns a function from a Point to a SquareMatrix).
    // Inverting a transformation
    val tInvertible : Transformation[_3D] with CanInvert[_3D, Transformation] = Translation3D(EuclideanVector3D(1, 0, 0))
    val tinv : Translation[_3D] = tInvertible.inverse
    
    // Getting the derivative with respect to position
    val tDifferentiable : Transformation[_3D] with CanDifferentiateWRTPosition[_3D] = Translation3D(EuclideanVector3D(1, 0, 0))
    val tDerivative : Point[_3D] => SquareMatrix[_3D] = tDifferentiable.derivativeWRTPosition
  11. Create discrete images

    main

    A DiscreteImage is a discrete field with an imageDomain. You can create one by providing a function that maps each point in the domain to a value, or by providing an array of values.

    Note: A DiscreteImage is conceptually equivalent to a DiscreteField that uses an imageDomain.

    // Create an image by assigning a value via a function
    val image = DiscreteImage3D(imageDomain, p => 0.toShort)
    
    // Create an image using an array of values
    val image2 = DiscreteImage3D(imageDomain, Array.fill(imageDomain.pointSet.numberOfPoints)(0.toShort))
    
    // Equivalent to creating a DiscreteField
    val image3 = DiscreteField3D(imageDomain, p => 0.toShort)