Created by Martin Hahner at the Computer Vision Lab of ETH Zurich.

Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather

by Martin Hahner, Christos Sakaridis, Dengxin Dai, and Luc van Gool

Accepted at ICCV 2021. 
Please visit our paper website for more details.

pointcloud_viewer

Overview

.
├── file_lists                          # contains file lists for pointcloud_viewer.py
│   └── ...
├── integral_lookup_tables              # contains lookup tables to speed up the fog simulation
│   └── ... 
├── extract_fog.py                      # to extract real fog noise* from the SeeingThroughFog dataset
├── fog_simulation.py                   # to augment a clear weather pointcloud with artificial fog (used during training)
├── generate_integral_lookup_table.py   # to precompute the integral inside the fog equation
├── pointcloud_viewer.py                # to visualize entire point clouds of different datasets with the option to augment fog into their scenes
├── README.md
└── theory.py                           # to visualize the theory behind a single LiDAR beam in foggy conditions

* Contains returns not only from fog, but also from physical objects that are closeby.

Datasets supported by pointcloud_viewer.py:

H3D
A2D2
KITTI
LyftL5
Pandaset
nuScenes
Argoverse
ApolloScape
SeeingThroughFog :arrow_left: works best
WaymoOpenDataset (via waymo_kitti_converter)

License

This software is made available for non-commercial use under a Creative Commons License. 
A summary of the license can be found here.

Acknowledgments

This work is supported by Toyota via the TRACE project.

Furthermore, we would like to thank the authors of SeeingThroughFog for their great work. 
In this repository, we use a fork of their original repository to visualize annotations and compare to their fog simulation. Their code is licensed via the MIT License.

Citation(s)

If you find this work useful, please consider citing our paper.

bibtex
@inproceedings{HahnerICCV21,
  author = {Hahner, Martin and Sakaridis, Christos and Dai, Dengxin and Van Gool, Luc},
  title = {{Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather}},
  booktitle = {IEEE International Conference on Computer Vision (ICCV)},
  year = {2021},
}

You may also want to check out our latest work (Oral at CVPR 2022) 
LiDAR Snowfall Simulation for Robust 3D Object Detection.

bibtex
@inproceedings{HahnerCVPR22,
  author = {Hahner, Martin and Sakaridis, Christos and Bijelic, Mario and Heide, Felix and Yu, Fisher and Dai, Dengxin and Van Gool, Luc},
  title = {{LiDAR Snowfall Simulation for Robust 3D Object Detection}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2022},
}

Getting Started

Setup

Install anaconda.
Create a new conda environment.

bash
conda create --name foggy_lidar python=3.9 -y

Activate the newly created conda environment.

bash
conda activate foggy_lidar

Install all necessary packages.

bash
conda install matplotlib numpy opencv pandas plyfile pyopengl pyqt pyqtgraph quaternion scipy tqdm -c conda-forge -y
pip install pyquaternion

Clone this repository (including submodules).

bash
git clone git@github.com:MartinHahner/LiDAR_fog_sim.git --recursive
cd LiDAR_fog_sim

Usage

How to run the script that visualizes the theory behind a single LiDAR beam in foggy conditions:

bash
python theory.py

theory

How to run the script that visualizes entire point clouds of different datasets:

bash
python pointcloud_viewer.py -d <path_to_where_you_store_your_datasets>

Note:

You may also have to adjust the relative paths in pointcloud_viewer.py (right at the beginning of the file) to be compatible with your datasets relative folder structure.