ARToolKit on HoloLens

HoloLens is fascinating in terms of holographic display and in-room localization, but the capability of the front-facing camera (locatable camera) is not fully utilized for potential augmented reality applications.

ARToolKit

Since ARToolKit has served AR/VR for more than 10 years, and it is open-source, it would be nice to have it running on HoloLens. The obstacles towards this goal are quite clear as well:

  • Universal Windows has a whole different set of tools and run-time libraries. The dependencies of existing ARToolKit does not exist on UWP, such as pthreads, video libraries, opencv.
  • ARToolKit is written in native C/C++ code, and an additional wrapper is needed for development on Unity, which is recommended for HoloLens. (If you are not good at Direct3D, and Windows App)

Luckily, after spending some time on this, I successfully get some pieces of code written, and actually make ARToolKit runs on HoloLens.

The repository is called HoloLensARToolKit on Github.

Demos

The following videos and screenshots are taken with HoloLensARToolKit v0.1.

Samples of HoloLensARToolKit

There are three samples of HoloLensARToolKit, each representing a certain kind of marker that is supported: single marker, cube marker and multi marker.

Minion on the Cube

This video is taken on HoloLens Device Portal, also known as Mixed Reality Capture. You might have noticed that there is still some lag between the video that is captured and update in the rendering.

More Screenshots

  • Single Marker

  • Cube Marker

  • Multi Marker

Finally

You can access more articles describing the implementation details of HoloLensARToolKit in my blog, simply clicking on the tag: hololens-artoolkit.

With ARToolKit integrated with HoloLens, many more augmented reality applications are possible. If you are AR/VR developer, you must see what I mean here.

Thanks for reading.


Midas Touch Problem in Eye Tracking

This is an interesting metaphor in the eye tracking literature.

King Midas is able to turn everything he touches into gold. It does not actually work out, since he cannot use his hands as normal hands, to do those daily simple tasks.

When eye gaze is used an interaction media interfacing with computer systems, the basic function of eye, to look and to perceive visual information, should be distinguished from those deliberate interaction with the computer systems. Otherwise, it becomes Midas Touch.

Thanks!