Controlling a 3D LED Matrix with Unity

This isn’t much of a how-to. I don’t know much about LED cubes, so I can’t offer help with whatever technology you may be working on. It’s more of an outline of the steps I took to get Unity to work with one of these things.

I recently spent a weekend building a 3D version of “Pong” that runs on a 25x25x8 array of LEDs. The game runs on a bit of Unity code that I hacked together to map a Unity camera and the area in space that it films onto the display. The display itself was built by a friend of a friend, who wrote some controller software for it that would display a special video file format, reading in a sequence of configurations for the LEDs, and playing that sequence back frame by frame.

First I modified the source of the controller software so that it would display real-time graphics instead of video files. These modifications weren’t particularly sophisticated, and they weren’t pretty. I just took out anything that had anything to do with playback and managing playlists, until only the parts remained that connected to the display and sent it data. There was a place in the source at which the data representing the LEDs’ state was just one long string of 0s and 1s. I edited that part, supplying my own strings, until I figured out the format that the display required. Then I grabbed some code from a simple client-server tutorial and added the server parts to the controller. The idea was that it would listen for connections on localhost, and accept streams of “0″ and “1″s from any client on the same machine, and it would update the display accordingly. That way it would be easy to control the display from all kinds of different sources, such as Unity, Processing, or even Flash.

On Unity’s side of things, I needed to slice an area in space into 8 layers, to cover the 25x25x8 pixels that are available on the LED matrix. The approach I took was to write a component for an isometric camera that would render the scene 8 times per frame. For each of those times, I’d set the near and far clipping planes of the camera to different values so as to only render small slices of the scene, and I’d have the camera render the result into a RenderTexture.

After rendering each slice of the scene, I would copy the contents of that RenderTexture into a Texture2D. Then I’d access that texture’s pixel data and go over each pixel: if it was fully transparent, then that’d mean the corresponding LED should be off. Otherwise, it’d be on.
What’s nice about the setup is that it allowed me to preview the data that would be sent to the display, by creating 8 planes in Unity that would each have a texture representing one of the 8 layers. This way I could actually develop without the display, preview right in Unity, then just hook everything up when I was done.

I’m adding the source code for the renderer here, to better illustrate the process. It was slapped together in about an hour, and there’s much you could clean up and optimize, but this setup is supposed to only ever run on one machine, and for that, it does its job. Feel free to use it as you want, but obviously, don’t expect that you can drop this into your own project and get any sort of useable result. :)

using UnityEngine;
using System.Collections;
using System.Collections.Generic;

public class Modul3DCamera : MonoBehaviour {

  public float nearClip = 4;
  public float farClip = 6;

  public Transform previewPlanesParent;

  private const int numLayers = 8;
  private int layerCounter = 0;

  private RenderTexture renderTex;
  private Texture2D readableTex;
  private Texture2D[] previewTextures;

  private int[] imageData;
  public int[] ImageData {
    get {
      return imageData;
    }
  }

  void Awake () {
    imageData = new int[25*25*numLayers];
    renderTex = new RenderTexture(25, 25, 16, RenderTextureFormat.ARGB32, RenderTextureReadWrite.Default);
    readableTex = new Texture2D(25, 25, TextureFormat.ARGB32, false);
    // Create Preview Planes
    GameObject previewPlanePrefab = Resources.Load ("Modul3D/PreviewPlane") as GameObject;
    previewTextures = new Texture2D[numLayers];
    for (int i=0; i<numLayers; i++) {
      Texture2D tex = new Texture2D(25, 25, TextureFormat.ARGB32, false);
      tex.filterMode = FilterMode.Point;
      previewTextures[i] = tex;
      GameObject pp = Instantiate(previewPlanePrefab) as GameObject;
      pp.layer = 20; // Preview Planes layer
      pp.transform.parent = previewPlanesParent;
      Utils.SetTransformToIdentity(pp);
      pp.renderer.material.mainTexture = tex;
      pp.transform.localPosition = new Vector3(0, 0, -i*0.2f);
    }
  }
  
  void Update () {
    // render all slices
    int imageDataIndex = 0;
    camera.targetTexture = renderTex;
    for (int i=0; i<numLayers; i++) {
      PrepareCameraForLayer(i);
      camera.Render();
      Color[] pixels = readableTex.GetPixels();
      for (int j=0; j<pixels.Length; j++) {
        if (pixels[j].a != 0) {
          pixels[j] = Color.white;
          imageData[imageDataIndex] = 1;
        } else {
          pixels[j] = Color.black;
          pixels[j].a = 0;
          imageData[imageDataIndex] = 0;
        }
        imageDataIndex++;

      }
      previewTextures[i].SetPixels( pixels );
      previewTextures[i].Apply();
    }

    camera.targetTexture = null;
    camera.nearClipPlane = nearClip;
    camera.farClipPlane = farClip;
  }

  void OnPostRender() {
    readableTex.ReadPixels(new Rect(0,0,25,25),0,0);
  }

  /// <summary>
  /// Prepares the camera for layer. Goes from 0 to numLayers-1
  /// </summary>
  /// <param name="layer">Layer.</param>
  void PrepareCameraForLayer(int layer) {
    float step = (farClip-nearClip)/numLayers;
    camera.nearClipPlane = nearClip+layer*step;
    camera.farClipPlane = nearClip+(layer+1)*step;

  }
}