gocesiumtiler

repository·v2·Indexed 43 days ago

https://github.com/mfbonfigli/gocesiumtiler

A high-performance Go-based tool and library for converting LAS point cloud files into Cesium 3D Tiles, supporting versions 1.0 (.pnts) and 1.1 (glTF/GLB). It features coordinate conversion via Proj, automatic subsampling, and attribute preservation. The tool provides a CLI for processing single files or folders and can be integrated as a Go module using cgo.

Tokens
4.3K
Snippets
12
Records
18
Agent score
54%

What's inside gocesiumtiler

  1. How to handle vertical coordinate conversion

    v2

    In V2, vertical datum conversion is delegated to the Proj library. To convert elevations (e.g., from EGM to WGS84), you must provide a CRS definition that includes the vertical datum.

    For example, to use EPSG:32633 with elevations referred to EGM2008 (EPSG:3855), use the composite CRS string:

    --crs EPSG:32633+3855

    Important: This may require downloading specific grid files from the Proj CDN and placing them in the share folder.

    gocesiumtiler folder -o C:\out -e EPSG:32633+3855 -r 10 -m 1000 -d 12 C:\las
  2. Understand the sampling algorithm

    v2

    The library uses a hybrid, lazy octree data structure for sampling:

    1. Storage: Points are stored in a linked tree backed by an array for cache efficiency and fast manipulations. Coordinates are converted to EPSG 4978 and then to a local CRS.
    2. Cell Creation: Octree cells are created based on a provided resolution (grid spacing).
    3. Traversal: Points are assigned to cells. The point closest to the cell center is kept; others are parked in the node's octant.
    4. Roll-up: If an octant has fewer points than min-points-per-node or reaches max-depth, points are rolled up to the parent node.
    5. Lazy Creation: Child nodes are created on demand using parked points.
    6. Output: Final artifacts use local CRS coordinates, with a global transform applied at the tileset root to return to EPSG 4978 for Cesium compatibility.
  3. How mutators work in gocesiumtiler

    v2

    Mutators allow you to manipulate or discard input points during the tiling process. They implement the mutator.Mutator interface and are passed to the tiler via tiler.WithMutators().

    Built-in Mutators:

    • mutator.NewZOffset(offset): Performs vertical translation of point clouds.
    • mutator.NewSubsampler(ratio): Thins down the points in the output.

    Important: Mutators must be goroutine safe.

    mut := []mutator.Mutator{
       mutator.NewZOffset(10),
       mutator.NewSubsampler(0.5),
    }
    
    err = t.ProcessFiles([]string{"myinput.las"}, "/tmp/myoutput", "EPSG:32632", tiler.NewTilerOptions(
       tiler.WithEightBitColors(true),
       tiler.WithWorkerNumber(2),
       tiler.WithMaxDepth(5),
       tiler.WithMutators(mut),
    ), ctx)
  4. Use gocesiumtiler as a Go library

    v2

    To integrate the tiler into your own Go applications, install the v2 module using go get. You can then instantiate a tiler using tiler.NewGoCesiumTiler() and process data using either ProcessFiles or ProcessFolder.

    Note: Compilation requires cgo. Refer to DEVELOPMENT.md for build environment setup requirements.

    package main
    
    import (
    	"context"
    	"log"
    
    	"github.com/mfbonfigli/gocesiumtiler/v2/tiler"
    )
    
    func main() {
    	t, err := tiler.NewGoCesiumTiler()
    	if err != nil {
    		log.Fatal(err)
    	}
    	ctx := context.TODO()
    	// ProcessFiles(inputFiles, outputDir, epsgCode, options, context)
    	err = t.ProcessFiles([]string{"myinput.las"}, "/tmp/myoutput", "EPSG:32632", tiler.NewTilerOptions(
    		tiler.WithEightBitColors(true),
    		tiler.WithWorkerNumber(2),
    		tiler.WithMaxDepth(5),
    	), ctx)
    	if err != nil {
    		log.Fatal(err)
    	}
    }
  5. Install gocesiumtiler

    v2

    To install the tool, follow these steps:

    1. Download the latest executable for your platform (Windows x64 or Linux x64) from the GitHub Releases page and unzip it.
    2. Ensure the executable is placed in the same directory as the share folder.
    3. Optional: If you require specific coordinate transformations (e.g., EGM to WGS84 elevation conversion), download the required Proj Data grids from the Proj CDN and unpack them into the share folder.

    Note: The tool stores data completely in memory for speed. If processing very large LAS files, you may need to split them into smaller chunks to avoid running out of RAM.

  6. Setup a Linux development environment

    v2
    For Linux development, the project recommends using the provided Dockerfile to ensure a consistent environment. The Dockerfile contains the detailed steps used to set up an Ubuntu-based environment for building and testing. Refer to the Dockerfile in the repository root for the specific command sequences used to prepare the Linux build environment.
  7. Build gocesiumtiler using Docker (Recommended)

    v2

    For reproducible builds, use the provided Docker-based workflow. This method handles the complex cgo compilation and static linking of the Proj library automatically for both Linux and Windows targets.

    • Linux/macOS: Run ./build.sh.
    • Windows: Run ./build.ps1 in a PowerShell console.

    It is recommended to use this method before creating any Pull Requests to ensure the build is reproducible.

    # For Windows (PowerShell)
    ./build.ps1
    
    # For Linux/macOS
    ./build.sh
  8. Setup a Windows development environment without Docker

    v2

    Building locally on Windows requires setting up a C/C++ toolchain to handle cgo and the static linking of the Proj library. Follow these steps in a PowerShell console:

    1. Install Go and MSYS2.
    2. Configure MSYS2 Path: Add C:\msys64\usr\bin to your system Path.
    3. Install Build Tools via MSYS2 pacman:
      pacman -S --noconfirm mingw-w64-x86_64-pkgconf
      pacman -S --noconfirm mingw-w64-x86_64-gcc
      pacman -S --noconfirm mingw-w64-x86_64-cmake
      pacman -S --noconfirm mingw-w64-x86_64-sqlite3
    4. Configure Mingw64 Path: Add C:\msys64\mingw64\bin to your system Path.
    5. Install Vcpkg: Clone and bootstrap the package manager:
      git clone https://github.com/Microsoft/vcpkg.git "C:\vcpkg"
      cd C:\vcpkg
      ./bootstrap-vcpkg.bat -disableMetrics
    6. Set Vcpkg Triplets: Set the following environment variables:
      • VCPKG_DEFAULT_TRIPLET=x64-mingw-static
      • VCPKG_DEFAULT_HOST_TRIPLET=x64-mingw-static
    7. Install Proj Dependencies: Use vcpkg to install sqlite3 and zlib:
      vcpkg.exe install sqlite3[core,tool] zlib --triplet=x64-mingw-static
    8. Build PROJ with Static Linking: Clone the PROJ repository (e.g., to C:\proj) and build it using CMake with the Vcpkg toolchain and Mingw64 compilers. Ensure -DBUILD_SHARED_LIBS=OFF is set to achieve static linking.
  9. Example: Convert a single LAS file with autodetected CRS

    v2

    To convert a single file using default settings and letting the tool detect the CRS from the LAS metadata:

    gocesiumtiler file -out C:\out C:\las\file.las
    gocesiumtiler file -out C:\out C:\las\file.las
  10. Example: Convert a folder of LAS files into a single merged tileset

    v2

    To convert all LAS files in a folder into one single 3D Tileset with specific sampling parameters:

    gocesiumtiler folder -out C:\out -crs EPSG:32633+3855 -resolution 10 -min-points-per-tile 1000 -depth 12 -join C:\las
    gocesiumtiler folder -out C:\out -crs EPSG:32633+3855 -resolution 10 -min-points-per-tile 1000 -depth 12 -join C:\las
  11. Build and test gocesiumtiler on Windows (Local)

    v2

    Once the environment (Go, MSYS2, Vcpkg, and PROJ) is configured, you can build or test the project by setting the necessary cgo and pkg-config environment variables.

    Note: You may need to adjust the paths in PKG_CONFIG_PATH and CGO_LDFLAGS to match your specific installation directories.

    # Build the executable
    $env:PKG_CONFIG_PATH="C:\usr\local\lib\pkgconfig;C:\vcpkg\installed\x64-mingw-static\lib\pkgconfig"; `
    $env:CC="x86_64-w64-mingw32-gcc"; `
    $env:CGO_ENABLED=1; `
    $env:CGO_LDFLAGS="-L/vcpkg/installed/x64-mingw-static/lib -g -O2 -static -lstdc++ -lsqlite3 -ltiff -lzlib -ljpeg -llzma -lm"; `
    go build -o ./bin/gocesiumtiler.exe ./cmd/main.go
    
    # Run tests
    $env:PKG_CONFIG_PATH="C:\usr\local\lib\pkgconfig;C:\vcpkg\installed\x64-mingw-static\lib\pkgconfig"; `
    $env:CC="x86_64-w64-mingw32-gcc"; `
    $env:CGO_ENABLED=1; `
    $env:CGO_LDFLAGS="-L/vcpkg/installed/x64-mingw-static/lib -g -O2 -static -lstdc++ -lsqlite3 -ltiff -lzlib -ljpeg -llzma -lm"; `
    go test -v ./...
  12. Implement a custom mutator

    v2

    To create a custom mutator, implement the Mutate method from the mutator.Mutator interface.

    Signature: Mutate(p model.Point, t model.Transform) (model.Point, bool)

    • Input: The original model.Point and a model.Transform object (useful for converting between local CRS and global EPSG 4978 CRS).
    • Output: The (potentially modified) model.Point and a bool. Return true to keep the point in the final output, or false to discard it.
    type ClassBasedColorMutator struct {}
    
    func (m *ClassBasedColorMutator) Mutate(pt model.Point, localToGlobal model.Transform) (model.Point, bool) {
    	switch pt.Classification {
    	case 2: // ground
    		pt.R, pt.G, pt.B = 128, 0, 0
    	case 3: // low vegetation
    		pt.R, pt.G, pt.B = 0, 60, 0
    	// ... other cases
    	default:
    		pt.R, pt.G, pt.B = 30, 30, 30
    	}
    	return pt, true
    }