Publications
ISMAR 2011 - Full paper
Gravity-Aware Handheld Augmented Reality
Kurz, D., and BenHimane S.
In Proc. IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR2011), pp. 111-120, Basel, Switzerland, 2011. (Best Paper Award Nominee)
Abstract
This paper investigates how different stages in handheld Augmented Reality (AR) applications can benefit from knowing the direction of the gravity measured with inertial sensors. It presents approaches to improve the description and matching of feature points, detection and tracking of planar templates, and the visual quality of the rendering of virtual 3D objects by incorporating the gravity vector. In handheld AR, both the camera and the display are located in the user’s hand and therefore can be freely moved. The pose of the camera is generally determined with respect to piecewise planar objects that have a known static orientation with respect to gravity. In the presence of (close to) vertical surfaces, we show how gravity-aligned feature descriptors (GAFD) improve the initialization of tracking algorithms relying on feature point descriptor-based approaches in terms of quality and performance.
For (close to) horizontal surfaces, we propose to use the gravity vector to rectify the camera image and detect and describe features in the rectified image. The resulting gravity-rectified feature descriptors (GREFD) provide an improved precision-recall characteristic and enable faster initialization, in particular under steep viewing angles. Gravity-rectified camera images also allow for real-time 6 DoF pose estimation using an edge-based object detection algorithm handling only 4 DoF similarity transforms. Finally, the rendering of virtual 3D objects can be made more realistic and plausible by taking into account the orientation of the gravitational force in addition to the relative pose between the handheld device and a real object.
[BibTex (bib)] [Preprint (pdf)] [@ acm Portal]
Video
Related publications
Kurz, D., and BenHimane S.
Handheld Augmented Reality involving gravity measurements
In Computers & Graphics, Special Section on Augmented Reality, pp. 866–883, November 2012 (Volume 36, Issue 7).
Kurz, D., and BenHimane S.
Inertial sensor-aligned visual feature descriptors
In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR2011), pp. 161-166,
Colorado Springs, USA, 2011.
Copyright © 2008—2021 Daniel Kurz. All rights reserved.