Neural Circuit Policies (NCPs)

The ncps.wirings module provides classes to define the structural connectivity (wirings) of Neural Circuit Policies (NCPs). These classes allow you to specify how neurons are connected, which is a core component of building NCP models in both PyTorch and TensorFlow.

Available wiring classes include:

• Wiring: The base class for defining connectivity.
• AutoNCP: Automatically generates a wiring based on specific parameters.
• NCP: A standard wiring implementation.
• FullyConnected: Defines a wiring where every neuron in one layer connects to every neuron in the next.
• Random: Defines a wiring with random connectivity patterns.

This repository contains the code and materials used for the experiments in the paper Neural Circuit Policies Enabling Auditable Autonomy.

Important Caveats:

• The code contains significant legacy code that is no longer used.
• Documentation is sparse.
• The code is written in TensorFlow 1.X (tested with 1.14).

For a modern, user-friendly TensorFlow 2.x implementation, use the main project page.

The ncps.torch module provides PyTorch implementations of Neural Circuit Policies. You can use high-level sequence models that handle entire sequences of data, or low-level RNN cells for single time-step processing.

Sequence Models

Use these classes when you want to pass an entire sequence (e.g., a tensor of shape [batch, time, features]) to the model at once:

• CfC: Continuous-time Feedback Control sequence model.
• LTC: Liquid Time-Constant sequence model.

Single time-step models (RNN cells)

Use these classes when you are implementing a custom loop and want to process data one time-step at a time (e.g., for reinforcement learning or custom recurrent architectures):

• CfCCell: Continuous-time Feedback Control RNN cell.
• LTCCell: Liquid Time-Constant RNN cell.

The ncps package for TensorFlow consists of two primary components:

1. LTC Model: A tf.keras.layers.Layer that implements a Liquid Time-Constant (LTC) RNN. Because LTC is expressed as a system of ordinary differential equations, it is inherently recurrent.
2. Wiring Architecture: A structure that defines how the LTC neurons are connected.

You can use wirings.AutoNCP to automatically generate a wiring diagram by specifying the total number of neurons and the number of outputs.

Neural Circuit Policies (NCPs) use recurrent neural network models where neurons are modeled by ordinary differential equations. The package provides two primary models:

1. LTC (Liquid Time-Constant): Based on differential equations with sigmoidal synapses. They are universal approximators and implement causal dynamical models, but require a numerical differential equation solver, which makes training and inference slower.
2. CfC (Closed-form Continuous-time): An approximation of the closed-form solution of the LTC differential equation. CfC resolves the performance bottleneck of LTC by avoiding the need for a numerical solver, making it faster for training and inference.

Note: Both models are recurrent neural networks with a temporal state and are only applicable to sequential or time-series data.

To plot the neural state activity of RNN compartments on the road where the car was driven, use the MATLAB script Neural_activity_analysis/plot_data_on_road.m. For NCP networks, this includes time-constant (coupling sensitivity) dynamics.

Requirement: The necessary data must be located in the analysis_data directory.

To run the reproducibility code, you must use the following environment:

• OS: Ubuntu 16.04
• Training: Python 3.5 with TensorFlow 1.14.0
• Data Analysis: Python 2.7, Python 3.6, or Matlab 2019a

To compute the sorted maximum Lipschitz constant for NCPs, CNNs, CT-RNNs, and LSTMs, use the MATLAB script Lipschitz_analysis/lipschitz_plots.m.

Requirement: The necessary data must be located in the analysis_data directory.

The active_test_analysis directory contains tools to process rosbag logs from active testing:

• driving_record.py: Counts crashes and computes secondary metrics like lateral discomfort.
• export_images.py: Extracts camera images, GPS readings, and auto mode switches into numpy or csv files.

Once images and data are exported, you can use training_scripts/replay_internal_states.py to re-compute the internal states of the RNNs. These traces are used for interpretability analysis and video generation.

To run the partially observable reinforcement learning examples, install ncps, tensorflow, ray[rllib], and gymnasium[mujoco] using pip.

pip3 install ncps tensorflow "ray[rllib]" "gymnasium[mujoco]"