Researchers have proposed EventEgoHands++, a framework for reconstructing 3D hand meshes from an egocentric viewpoint using event cameras. The work targets applications such as human-robot interaction and AR/VR, where conventional cameras often struggle in low-light conditions and under severe motion blur.
Event cameras offer high dynamic range and high temporal resolution, but egocentric use is complicated by the camera wearer's own motion, which generates dense background events that obscure hand-specific signals. An earlier egocentric event-based approach used hand segmentation to reduce this noise, yet its binary mask did not distinguish left from right hands. According to the paper, that limitation caused the model to predict both hands even when only one or neither was present, harming inter-hand relationships and reconstruction accuracy.
EventEgoHands++ addresses this with a Hand Detector that estimates instance-level bounding boxes and masks for both the left and right hands. It also introduces Adaptive Attention, which dynamically gates attention based on those detection results to better learn the spatial relationship and mutual interactions between hands.
To train and evaluate the framework, the authors extended the synthetic N-HOT3D dataset and built EEH-R, described as the largest real-world event-based egocentric hand dataset to date, comprising approximately 1 million annotated frames captured in environments that include low-light conditions.
Extensive experiments on both synthetic and real datasets reportedly show the method consistently outperforming baselines. The paper is listed on Hugging Face Papers.