E2PNet: Event to Point Cloud Registration with Spatio-Temporal Representation Learning

@article{Lin2023E2PNetET,
  title={E2PNet: Event to Point Cloud Registration with Spatio-Temporal Representation Learning},
  author={Xiuhong Lin and Changjie Qiu and Zhipeng Cai and Siqi Shen and Yu Zang and Weiquan Liu and Xuesheng Bian and Matthias M{\"u}ller and Cheng Wang},
  journal={ArXiv},
  year={2023},
  volume={abs/2311.18433},
  url={https://api.semanticscholar.org/CorpusID:265506792}
}
E2PNet is the first learning-based method for event-to-point cloud registration and is more robust to extreme illumination or fast motion due to the use of event data, and shows the potential of EP2T for other vision tasks such as flow estimation, event- to-image reconstruction and object recognition.
View PDF on arXiv
1 Citation
Background Citations
1

Figures and Tables from this paper

One Citation

RELI11D: A Comprehensive Multimodal Human Motion Dataset and Method

This work proposes LEIR, a multimodal baseline that effectively utilizes LiDAR Point Cloud, Event stream, and RGB through the authors' cross-attention fusion strategy and shows that LEIR exhibits promising results for rapid motions and daily motions and that utilizing the characteristics of multiple modalities can indeed improve HPE performance.

53 References

ECSNet: Spatio-Temporal Feature Learning for Event Camera

This work introduces a compact event representation, namely 2D-1T event cloud sequence (2D- 1T ECS), and proposes a novel light-weight spatio-temporal learning framework (ECSNet) that accommodates both object classification and action recognition tasks.

DeepI2P: Image-to-Point Cloud Registration via Deep Classification

DeepI2P is presented, a novel approach for cross-modality registration between an image and a point cloud that estimates the relative rigid transformation between the co-ordinate frames of the camera and Lidar.

End-to-End Learning of Representations for Asynchronous Event-Based Data

This work introduces a general framework to convert event streams into grid-based representations by means of strictly differentiable operations and lays out a taxonomy that unifies the majority of extant event representations in the literature and identifies novel ones.

Back to Event Basics: Self-Supervised Learning of Image Reconstruction for Event Cameras via Photometric Constancy

This work approaches, for the first time, the intensity reconstruction problem from a self-supervised learning perspective, and proposes a novel, lightweight neural network for optical flow estimation that achieves high speed inference with only a minor drop in performance.

Learning an Event Sequence Embedding for Dense Event-Based Deep Stereo

A new module for event sequence embedding is introduced, which is the first learning-based stereo method for an event-based camera and the only method that produces dense results on the Multi Vehicle Stereo Event Camera Dataset (MVSEC).

P2-Net: Joint Description and Detection of Local Features for Pixel and Point Matching

A dual fully-convolutional framework is presented that maps 2D and 3D inputs into a shared latent representation space to simultaneously describe and detect keypoints and achieves state-of-the-art results for the task of indoor visual localization.

EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras

Event-based cameras have shown great promise in a variety of situations where frame based cameras suffer, such as high speed motions and high dynamic range scenes. However, developing algorithms for…

Event-Based Vision Meets Deep Learning on Steering Prediction for Self-Driving Cars

A deep neural network approach is presented that unlocks the potential of event cameras on a challenging motion-estimation task: prediction of a vehicle's steering angle, and outperforms state-of-the-art algorithms based on standard cameras.

EV-IMO: Motion Segmentation Dataset and Learning Pipeline for Event Cameras

We present the first event-based learning approach for motion segmentation in indoor scenes and the first event-based dataset – EV-IMO– which includes accurate pixel-wise motion masks, egomotion and…

Learning Dense and Continuous Optical Flow From an Event Camera

This paper proposes a novel deep learning-based dense and continuous optical flow estimation framework from a single image with event streams, which facilitates the accurate perception of high-speed motion and outperforms existing event-based state-of-the-art methods.
...

Related Papers

Showing 1 through 3 of 0 Related Papers