DOCSEARCHbyAlgolia
Submit Docs

Waymo Open Dataset Documentation

repository·master·Indexed 25 days ago

https://github.com/waymo-research/waymo-open-dataset

A collection of high-resolution sensor data, object trajectories, and 3D maps for research in machine perception, motion forecasting, and autonomous driving. Includes the Perception, Motion, and End-To-End Driving datasets, along with evaluation code, TensorFlow helper functions, 3D labeling specifications for vehicles, pedestrians, and cyclists, and guidelines for real-time detection challenge submissions via Docker.

Tokens
34K
Snippets
75
Records
183
Agent score
82%

What's inside Waymo Open Dataset

  1. Overview of the Waymo Open Dataset

    master

    The Waymo Open Dataset provides datasets and evaluation code to support research in machine perception and autonomous driving. The repository includes three primary datasets:

    • Perception dataset: High-resolution sensor data and labels for various perception tasks.
    • Motion dataset: Object trajectories and corresponding 3D maps for 103,354 scenes.
    • End-To-End Driving dataset: Camera data and high-level commands.

    The codebase provides dataset format definitions, evaluation metrics, and TensorFlow helper functions for model building.

  2. Overview of the Waymo Open Dataset Library

    master

    The Waymo Open Dataset Library provides a collection of Python and C++ utilities designed for working with the Waymo Open Dataset. Key capabilities include:

    • Computing quality metrics.
    • Reading dataset files.
    • Pre-processing data from the dataset.
    • Utilizing custom TensorFlow Ops (C++).

    For general information about the dataset and past challenges, visit https://waymo.com/open. For detailed usage instructions and technical documentation, refer to the main repository.

  3. Build a pip package locally using Docker

    master

    You can build the waymo_open_dataset wheel package using a Docker container. This process executes build.sh inside the container, which runs all tests and builds the package. The resulting .whl files will be located in the /tmp/wod/ directory on your host machine.

    mkdir /tmp/wod
cd src
docker build \
    --tag=open_dataset_pip\
    -f waymo_open_dataset/waymo_open_dataset/pip_pkg_scripts/build.Dockerfile\
    --build-arg USERNAME=$USER\
    --build-arg USER_UID=$`(id -u `$USER) .
docker run --mount type=bind,source=/tmp/wod,target=/tmp/wod open_dataset_pip

  4. Use the End-To-End Driving Dataset (v1.0.0)

    master

    The End-To-End Driving Dataset (v1.0.0) includes an expanded training set (2,037 runs), a testing set (1,505 runs), and a validation set (479 runs) featuring rater feedback labels and scenario tags. The dataset provides camera images and high-level commands for the ego-vehicle.

    To use this dataset, refer to the proto format definitions in the repository and use the updated Colab tutorials to calculate the Rater Feedback Score.

  7. Submit Docker images to Google Cloud Storage for latency benchmarks

    master

    To participate in latency benchmarking, you can upload your Docker image to a Google Cloud Storage (GCS) bucket. You must provide the full path starting with the gs:// protocol in the docker_image_source field of your submission proto.

    Requirements:

    • Permissions: You must grant Waymo's service account 213834518535-compute@developer.gserviceaccount.com read permissions to your bucket.
    • Recommended Settings: To minimize costs, create a bucket with Location: Region and Region: us-west1 (Oregon).
    • Best Practice: Use a unique name for every submitted image to avoid issues with evaluation server fetch delays.

    Workflow:

    1. Save the image: docker save --output="example_docker_image.tar" ID_OF_THE_IMAGE
    2. (Optional) Compress the image: gzip example_docker_image.tar
    3. Upload to GCS: gsutil cp example_docker_image.tar.gz gs://example_bucket_name/example_folder/
    4. Set docker_image_source in your proto to: "gs://example_bucket_name/example_folder/example_docker_image.tar.gz"
    docker save --output="example_docker_image.tar" ID_OF_THE_IMAGE
gzip example_docker_image.tar
gsutil cp example_docker_image.tar.gz gs://example_bucket_name/example_folder/

  8. Use the MeanErrorMatcher for object matching

    master

    To evaluate performance, the evaluation service uses a matching algorithm to find correspondence between predicted (PR) and ground truth (GT) objects. The official algorithm for the challenge is MeanErrorMatcher, located in src/waymo_open_dataset/metrics/python/keypoint_metrics.py.

    Matching Logic

    1. Distinguish GT types:
      • $GT_v$: GT boxes with at least one visible keypoint.
      • $GT_i$: GT objects without any visible keypoints (unlabeled or heavily occluded).
    2. Stage 1 (Filtering): Select pairs of GT and PR objects where at least one PR keypoint is within a distance threshold $C$ (0.25m) from the GT box. Pairs falling into $GT_i$ objects are excluded without penalty.
    3. Stage 2 (Hungarian Matching): For remaining $GT_v$ and PR pairs, perform Hungarian matching to minimize the PEM metric.

    Implementation Details

    • The matcher outputs three sets: True Positives (TP), False Positives (FP), and False Negatives (FN).
    • Only matches between $GT_v$ objects and PR objects are considered for PEM computation.
    • Each ground-truth box ($GT_v$ and $GT_i$) can be associated with a maximum of 1 detection. To maximize PEM scores, you must remove duplicate detections.

  9. Run a Jupyter notebook container with waymo_open_dataset

    master

    Start a container from the open_dataset image and map port 8888 to access a Jupyter notebook environment. This environment includes waymo_open_dataset as a dependency.

    After running the container, open http://0.0.0.0:8888 in your browser and select tutorial_local.ipynb to begin the tutorial.

    docker run -p 8888:8888 open_dataset

  10. Understand Lane Neighbors and Boundary Types

    master

    Lane neighbors are stored as segments of a lane, where each segment is associated with a corresponding segment of a neighboring lane.

    To determine the boundary type that would be crossed during a lane change from any specific lane point, use the following structure:

    1. Identify the segment of the lane that has a neighbor.
    2. Locate the associated segment of the neighboring lane.
    3. Within that neighbor segment, the boundary (or multiple boundaries) between the lane and its neighbor is stored as segments of the lane.

  11. Implement a model submission for Real-time Detection Challenges

    master

    To participate in the real-time 3D and 2D detection challenges, you must submit a Docker image containing a Python module named wod_latency_submission. This module must be available in the image's PYTHONPATH or installed via pip.

    The module must implement the following interface:

    1. initialize_model(): A function with no arguments to load and initialize the model. It is called once before data is passed.
    2. run_model(**kwargs): A function that accepts numpy ndarrays (passed as keyword arguments based on your DATA_FIELDS specification) and returns a dictionary containing:
      • boxes:
        • For 3D detection: $N imes 7$ float32 array [center_x, center_y, center_z, length, width, height, heading].
        • For 2D detection: $N imes 4$ float32 array [center_x, center_y, length, width].
      • scores: $N$ length float32 array of confidence scores in [0, 1].
      • classes: $N$ length uint8 array of type IDs.
    3. DATA_FIELDS: A list of strings specifying which data formats the model requires (e.g., ['TOP_RANGE_IMAGE_FIRST_RETURN', 'CAM_FRONT_IMAGE']).
    4. DATA_FORMATS: A list of strings indicating the required data formats.