HoloLens Video Streaming

There have been lots of discussion about streaming on HoloLens. Here is a demo of my implementation.

Demo

This is the soccer match between Barcelona and Atlético Madrid today. The screen on the right is the real monitor, and the left one is a virtual screen showing streamed video. The application runs at a reasonably frame rate and resolution (20fps with 896x594 frame resolution). Video is taken via HoloLens Device Portal.

Implementation

This application is implemented using the following toolset:

On PC

  • Epiphan DVI2USB 3.0 and its SDK
  • Python for socket, queue, and interfacing Epiphan SDK via ctypes
  • Network connection

On HoloLens

  • Made with Unity 5.5.0f3
  • Windows.Networking.Sockets.StreamSocket
  • HoloToolkit
  • Network connection

Thanks for reading!


Jekyll with Three.js

Here is an example of embedding three.js and dat.gui on a Jekyll blog. It is working after a few attempts.

Demo

You can CLICK on the canvas to explode the teapot. Number of teapots and the tessellation level of them can be modifed via the GUI elements on the top right corner.

How?

Essentially, Jekyll is a tool to parse pre-defined styles, scripts and texts, and build static website out of these pieces. Three.js and dat.gui are perfectly static, and parsable for Jekyll. So, why not?

Canvas holder

In order for the canvas to be located on a specific position of the blog, like other blog components, a canvas holder is needed to hold the canvas. Therefore, all elements in the blog are displayed in a correct order. This is a little bit different from other demos of three.js, because in most cases, or examples, the three.js canvas is the whole screen.

Another role of the canvas holder is to place the dat.gui interface as well. We don’t want the default location of dat.gui (right top corner of the whole page).

Dat.gui holder

This div element is used for holding the dat.gui, in order for manipulate the location of the GUI easier.

The combination of canvas holder and dat.gui holder can be specified like this:

<style>
  canvas { width: inherit; position: relative; top: 0;}
</style>
<div id='canvas-holder' style="position: relative; width: inherit;">
  <div id="dat-gui-holder" style="position: absolute; top: 0em; right: 0em; z-index: 1;"></div>
</div>

The dat.gui holder has two interesting style definitions:

  • position:absolute places the GUI at fixed location relative to the parent.
  • z-index:1 places the GUI on top of the canvas, aka overlay.

You might noticed the additional style definition for canvas. Basically, it forces the actual rendered canvas to be the same size of the canvas holder.

Scripts

The auto placement of dat.gui should be turned off when it is initialized. It is highly inspired by this example.

var gui = new dat.GUI( { autoPlace: false } );
// Definition of GUI elements goes here
document.getElementById('dat-gui-holder').appendChild(gui.domElement);

In addition, the canvas is put inside canvas holder :

var canvasHolder = document.getElementById('canvas-holder');
// Apply your desired aspect ratio
var width = canvasHolder.clientWidth;
var height = width * 0.8;
canvasHolder.clientHeight = height;
renderer = new THREE.WebGLRenderer();
renderer.setSize( width, height );
canvasHolder.appendChild( renderer.domElement );

Finally

Blogs should be COOL.

Thanks!