SmartAR on HMD
18 Oct 2016This is a re-post of my project for EN.600.661 Computer Vision at Johns Hopkins University. Team members include me and Felix Jonathan.
Demo
The video is taken by a phone camera placed behind the smart glasses.
What is it?
SmartAR is an augmented reality application on smart glass Moverio BT-200. It is able to recognize the context in front of the glass via Bag-of-Words algorithm, track fiducial marker inside the scene, and display augmentation based on its understanding.
Hardware
- A PC with Internet
- Moverio BT-200 head-mounted display
Software
- Robot Operation System installed on PC
- Aruco and OpenCV
- Python
- Android SDK and ROS Android
System architecture of SmartAR looks like this:
ROS manages the messages within the network between PC and Moverio BT-200.
Object Recognition
The object recognition algorithm here is very basic bag-of-words algorithm. A database of object images is needed on PC.
Thanks to the simplicity of Bag-of-Words algorithm, the recognition of known objects in this application is real-time.
Marker Tracking
Aruco is used to track markers in the scene. In order to correctly overlay the graphics on top of the marker seen by the user, an optical see-through display calibration is needed. Here, the traditional Single Point Active Alignment Method is applied. Rendering is performed using OpenGL ES in Android OS. The result looks like this:
Both geometric and context augmentation are provided at the same time. The performance is real-time as well.
Thanks!