DeepFactors Dense Monocular SLAM

To install DeepFactors, follow these steps to clone the repository, install system dependencies, compile third-party libraries, and build the project.

1. Get the code

Clone the repository including all submodules:

git clone --recursive <url>

If you already cloned without submodules, run:

git submodule update --init --recursive

2. Install system dependencies

Ubuntu 18.04:

sudo apt install cmake libboost-all-dev libglew-dev libgoogle-glog-dev \
      libjsoncpp-dev libopencv-dev libopenni2-dev unzip wget

Note: You must also install the TensorFlow C API. If using CUDA 10.0 and cuDNN 7.5, use pre-built binaries. For other versions, you must compile it from source.

Arch Linux:

sudo pacman -S boost cuda gflags glew google-glog jsoncpp opencv tensorflow-cuda

Note: You also need the openni2 AUR package.

3. Compile third-party dependencies

Build the vendored dependencies using the provided script:

./thirdparty/makedeps.sh

You can use the --threads option to speed up compilation (e.g., ./thirdparty/makedeps.sh --threads 5), but be mindful of system memory.

4. Build DeepFactors

Use CMake to configure and make to build:

mkdir build
cd build
cmake ..
make

You can speed up the final build using make -j5.

DeepFactors can be run on live devices using the df_demo binary.

OpenNI Devices (e.g., Asus Xtion)

Use the openni:// source URL. Replace <camera_id> with your camera index (use 0 for the first camera).

Odometry configuration:

build/bin/df_demo --flagfile=data/flags/live_odom.flags --source_url=openni://<camera_id>

Local refinement mode:

build/bin/df_demo --flagfile=data/flags/live_refine.flags --source_url=openni://<camera_id>

PointGrey Cameras

Support for flycap is available if DF_WITH_FLYCAP=ON was set during compilation.

Interactive Controls

While the system is running, use these keys:

• r: Reset the SLAM system
• space: Initialize the system; later used to add new views for refinement
• p: Pause camera input (allows rotating the reconstructed model)
• n: Spawn a new keyframe

build/bin/df_demo --flagfile=data/flags/live_odom.flags --source_url=openni://0

Additional tools can be built by enabling the DF_BUILD_TOOLS CMake option. The binaries are located in <build_dir>/bin.

• decode_image: Loads an image and displays decoded zero-code and explicitly predicted code to test the network and depth prediction. Provides timing information.
• kernel_benchmark: Performs a grid search to tune CUDA kernel parameters. Options: --sfm_step_blocks=, --sfm_step_threads=, --sfm_eval_blocks=, --sfm_eval_threads=.
• result_viewer: Displays reprojected ground-truth depth and trajectories (ground-truth vs. estimated) for qualitative evaluation. Can convert trajectories to TUM format.
• test_matching: Runs the feature matching algorithm on two images to test and tune parameters.
• voc_builder: Builds a BRISK feature vocabulary for DBoW2 using selected TUM sequences.
• voc_test: Tests a vocabulary by calculating similarity among images and generating a confusion matrix.

When programmatically using the demo system (rather than via CLI), you must populate a df::LiveDemoOptions object. This object contains two main sections:

1. Demo-specific settings: Such as source_url, calib_path, and log_dir.
2. df_opts (DeepFactorsOptions): Contains the core SLAM parameters including tracking, mapping, loop closure, and error types.

Note that string-based flags from the CLI (like init_type or keyframe_mode) must be converted using the provided translator methods (e.g., df::LiveDemoOptions::InitTypeTranslator) before being assigned to the options object.

df::LiveDemoOptions opts;
opts.source_url = "flycap://0";
opts.df_opts.network_path = "data/frozen_graph.pb";
// Use translators for enum-like string values
opts.init_type = df::LiveDemoOptions::InitTypeTranslator("ONEFRAME");
opts.df_opts.keyframe_mode = df::DeepFactorsOptions::KeyframeModeTranslator("AUTO");

// Initialize and run
df::LiveDemo<DF_CODE_SIZE> demo(opts);
demo.Run();

The following command-line flags are available for configuring the DeepFactors demo system. Note that many flags are grouped by their functional area (Demo, SLAM, Loop Closure, Tracking, Mapping, etc.).

### Demo Options
--source_url: Image source URL (default: "flycap://0")
--calib_path: Path to OpenCV yaml file containing camera calibration (default: "data/flea.yml")
--network_path: Path to protobuf file containing network graph with weights (default: "data/frozen_graph.pb")
--vocab_path: Path to ORB vocabulary for DBoW2 (default: "data/ORBvoc.yml.gz")
--gpu: Which gpu to use for SLAM (default: 0)
--init_on_start: Initialize the system on the first captured image (default: false)
--quit_on_finish: Close the program after finishing all frames (default: false)
--init_type: How to initialize the slam system (ONEFRAME, TWOFRAME) (default: "ONEFRAME")
--record_input: Path where to save input images
--pause_step: Pause after each frame
--run_log_dir: Directory where the run logs will be saved
--run_dir_name: Force a specific run directory name
--enable_timing: Enable profiling of certain parts of the algorithm
--frame_limit: Limit processing to first <frame_limit> frames
--skip_frames: Skip first <skip_frames> frames
--demo_mode: Hide GUI elements and show only reconstruction

### SLAM Options
--keyframe_mode: New keyframe initialization criteria (AUTO, NEVER)
--connection_mode: How new keyframes will be connected to the others (FULL, LASTN, FIRST, LAST)
--tracking_mode: How to select which keyframe to track against (CLOSEST, LAST, FIRST)
--interleave_mapping: Interleave tracking with single mapping steps
--inlier_threshold: Inlier threshold used to initialize new keyframes
--dist_threshold: Distance threshold used to initialize new keyframes
--frame_dist_threshold: Distance threshold used to initialize new keyframes
--max_back_connections: How far to connect back new keyframes
--debug: Display debug images in SLAM

### Loop Closure Options
--loop_closure: Enable loop closure
--loop_active_window: Active window for local/global loop detection
--loop_sigma: Noise stddev of the reprojection factors added for loop closure
--loop_max_dist: Maximum distance to potential loop closure candidates
--loop_min_similarity: Minimum similarity for loop closure candidates
--loop_max_candidates: Maximum number of candidates to get from DBoW2

### Tracking Options
--tracking_iters: Comma separated list of number of iterations per pyramid level (e.g. 3,3,5,10)
--tracking_huber_delta: Huber norm delta used in tracking
--tracking_error_threshold: When to consider tracking as lost
--tracking_dist_threshold: When to consider tracking as lost

### Mapping Options
--pose_prior: Noise of the prior on the first pose
--code_prior: Noise of the prior on the codes
--relinearize_skip: ISAM2 relinearize skip
--relinearize_threshold: ISAM2 relinearize threshold
--partial_relin_check: ISAM2 partial relinearization check
--huber_delta: Huber norm delta used in mapping
--predict_code: Initialize keyframes with a predicted code

### Photometric Error Options
--use_photometric: Use photometric error in keyframe-keyframe links
--pho_iters: Comma separated list of number of iterations per pyramid level
--sfm_step_blocks: Number of blocks to use in mapping run optim. step
--sfm_step_threads: Number of threads per block to use in mapping
--sfm_eval_blocks: Number of blocks to use in mapping to evaluate error
--sfm_eval_threads: Number of threads per block to use in mapping to evaluate error
--normalize_image: Normalize the image before feeding it to the algorithm

### Geometric Error Options
--use_geometric: Use sparse geometric error for mapping
--geo_npoints: Number of sparse random points to optimize in geometric factors
--geo_stochastic: Random sparse pattern at every linearization for geometric error
--geo_huber: Huber norm used in geometric error
--geo_iters: Number of iterations for geometric error

### Reprojection Error Options
--use_reprojection: Use sparse reprojection error for mapping
--rep_nfeatures: Number of keypoints to detect in a keyframe
--rep_scale_factor: ORB's scale factor
--rep_nlevels: ORB's number of levels
--rep_max_dist: Max distance between feature descriptors in a match
--rep_huber: Huber norm used in reprojection error factors
--rep_iters: Number of iterations for reprojection error
--rep_sigma: Noise stddev of the reprojection factors added to the graph
--rep_ransac_maxiters: Maximum iterations for the RANSAC outlier rejection
--rep_ransac_threshold: Threshold for detecting outliers in RANSAC outlier rejection

### Visualization Options
--vis_mode: Which keyframes to draw (FIRST, LAST, ALL)