Eigen in Unity

Unity3d has very basic linear algebra support, for example, in Matrix4x4, there are simple multiplication functions with vectors and matrices, however, functions like inverse and eigen value are not included. Eigen is the perfect candidate of performing complicated linear algebra algorithms, for example, Singluar Value Decomposition, Sparse Matrix, Linear Eqaution Solver.

Unity Native Programming

The key idea of integrating Eigen with Unity3d is using the native programming interface of .NET. Here is another example: Native Programming in Unity. Basically, the native programming interface of unity provides a way for .NET environment to call C functions defined in built dynamic libraries.

What we will do is:

  • Build a dynamic library that contains the functions that use Eigen (C++)
  • Initiates the call to the functions in Unity scripts (C#)

A Minimum Example

Here is a minimum example showing how to get the trace of a matrix in Eigen.

Dynamic Library using Eigen

  • Download Eigen and decompress it.
  • Create a Visual Studio dynamic library project.
  • Add the root path of Eigen package to Additional Include Directories
  • Add a CPP file to the project
  • Copy the following to the CPP file:
#define EXPORT_API __declspec(dllexport)
#include <Eigen/Dense>
using namespace Eigen;
extern "C" {
  EXPORT_API float getTrace() {
    MatrixXf m = MatrixXf::Random(4,4);
    return m.trace();
  }
}
  • Configure the building so that it is compatible with targeted platform.
  • Build
  • Copy to Unity3D project

Call the Function in Unity Script

  • Create a script EigenTest.cs.
  • Copy the following to the CSharp file:
using System.Collections;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using UnityEngine;

public class EigenTest : MonoBehaviour {
  [DllImport("YOURDLLNAME")]
  public static extern float getTrace();
  void Start () {
    Debug.Log("The trace value is: " + getTrace());
  }
}
  • Attach the script to any GameObject.
  • Deploy to the targeted platform.

Advanced

The above example is the minimal to make things work. If you want to parse a Matrix4x4 to Eigen environment, please refer to Marshaling.

Eigen is perfect for native programming because it is header-only and is potentially platform independent. If you want to understand more about this workflow, please refer to Unity Native Programming.

Thanks for reading!


ARUWPMarker Options in HoloLensARToolKit v0.1

This post is part of documentation of HoloLensARToolKit, version v0.1.

ARUWPMarker

ARUWPMarker.cs is one of the main scripts used in HoloLensARToolKit. In this post, the options of this script are listed and discussed, along with common usecases.

In Unity project using HoloLensARToolKit package, each actual marker must have one corresponding ARUWPMarker script.

ARUWPMarker is very similar to ARMarker in ARToolKit. However, because currently HoloLensARToolKit does not support NFT (Natural Feature Marker) of ARToolKit, that part of attributes are not included in ARUWPMarker.

When ARUWPMarker script is attached to some Unity GameObject, its inspector window looks like this:

and the inspector behavior of ARUWPMarker.cs is controlled by ARUWPMarkerEditor.cs located at Assets/Editor/.

Type

This field configures the type of the current marker. Available options are: single, single_barcode, single_buffer, and multi.

  • single: single pattern marker, e.g. Hiro marker and Kanji marker. If single is selected, then the field File Name will appear. Root path of file name is Assets/StreamingAssets/.
  • single_barcode: single barcode marker (as known as single matrix marker). If it is selected, then the field Barcode ID will appear.
  • single_buffer: single pattern marker, but directly specified by a byte buffer. It should be set at runtime using function ARUWPMarker::setSingleBufferBuffer.
  • multi: multi marker consists of multiple barcode marker. If it is selected, then the field File Name will appear to ask for the configuration file. Root path of the configuration file is Assets/StreamingAssets/ as well.

Size in mm

You need to specify the size of the marker (outer border), when single or single_barcode marker is used.

Show Options

If it is checked, filtering options of marker tracking can be configured, including: Continuous Pose Estimation and Confidence Cutoff. Please refer to ARToolKit documentation for details about filtering.

Visualization Target

Marker pose provided by the algorithm contains position and orientation. This information should be applied to some object in the scene so that augmented reality experience is created. This field controls the target GameObject that receives the pose of marker.

Anchored to World

HoloLens is constantly localizing itself in the room.

The default augmented reality experience of HoloLens is creating an AR scenario within the room, and let the user explore it. Therefore, HoloLensARToolKit package takes advantage of its localization algorithm, to make the virtual object appear in the world coordinate system. The benefit of doing it is that, when the marker is occluded, it stays at the same position in the room. It is desired when the line-of-sight of the HoloLens camera is blocked, but the tracked object is not moving in the environment.

If this field is not checked, then all the pose update is achieved in the camera coordinate system. The virtual object stays at the same pose with respect to the camera, when the tracking is lost.

Apply Rotation

Whether rotation of tracked marker should be applied to the Visualization Target.

Apply Translation

Whether translation (position) of tracked marker should be applied to the Visualization Target.

Confidence Textbox and Confidence Colorbox

If the marker type is single, single_barcode or single_buffer, then ARToolKitUWP provides a score representing the confidence of tracking, ranging from 0 to 1. The score could be visualized in a textbox (Unity::UI::Text object) and in a colorbox (Unity::UI::Image object).

Examples

Hiro Marker

Matrix Marker (Number 0)

Cube Marker

Finally

You can access more articles describing the implementation details of HoloLensARToolKit in my blog, simply clicking on the tag: hololens-artoolkit.

Thanks for reading!