All research

Egocentric Motion Tracking for 3D Scene Interactions

Detecting hand-object interactions of varying degree from egocentric video.

May 2022 Multimodal
PDF
Two degrees of interaction in a kitchen.
Two degrees of interaction in a kitchen.

A hybrid pipeline that recognises hand-object interactions in egocentric video and grounds them in 3D.

  • Designed a pipeline to identify hand-object interactions of varying degree in a video stream.
  • Explored pseudo-labelling for contact-point identification on 3D object meshes.
  • Formulated a hybrid model combining semantic segmentation, depth estimation and 3D scene-flow tracking.
BackAll research