HeiRes – MS Hackathon – PartyCrasher

On 23rd May 2014 the company HeiRes organized together with Microsoft Germany a hackathon in my hometown Dresden. The goal was to build an app in about 8 hours through the night.
I joined a team with Christian and our idea was to build a ‘PartyCrasher’-App. When you leave your kids alone at home over the weekend they may get the idea to have a party at your house which you may not want because they made a mess the last time. So why don’t you set up the PartyCrasher App in you living room that consists of a Kinect for Windows controller that is hooked to a little PC. The Kinect watches the scene in the living room and starts to take pictures as soon as there are more than a specified number of people, for example more than 3. At this time the Kinect starts to take pictures in an interval that you can configure (maybe every minute).

So we’ve started the hackathon with this little architecture in our mind.
PartyCrasher

In this blog post I want to share how easy it was to program this little Kinect service (console application) that is hooked to the Kinect hardware. We’ve made use of the Kinect for Windows SDK V2 Preview, Kinect.ReactiveV2 and Rx to implement the photo shooting every specified amount of time as soon as there a more than the specified number of people in the scene.

using Kinect.ReactiveV2;
using Microsoft.Kinect;
using System;
using System.Linq;
using System.Reactive.Linq;

static void Main(string[] args)
{
  var kinect = KinectSensor.Default;
  kinect.Open();

  var frameDescription = kinect.ColorFrameSource.CreateFrameDescription(ColorImageFormat.Rgba);
  var bytes = new byte[frameDescription.Width * frameDescription.Height * 4];

  var moreThanPeople = 3;
  var intervalInSeconds = 60;

  var reader = kinect.ColorFrameSource.OpenReader();
  var bodies = new Body[6];

  var subscription = kinect.BodyFrameArrivedObservable()
                           .SelectBodies(bodies)
                           .SelectTracked()
                           .Where(_ => _.Count() > moreThanPeople)
                           .Sample(TimeSpan.FromSeconds(intervalInSeconds))
                           .Subscribe(bs =>
                           {
                             using(var frame = reader.AcquireLatestFrame())
                             {
                               if(frame == null) return;
                               frame.CopyConvertedFrameDataToArray(bytes, ColorImageFormat.Rgba);
                             }

                             SaveInBlobStorage(frameDescription, bytes);
                           });

  Console.WriteLine("[ENTER] to stop");
  Console.ReadLine();

  subscription.Dispose();
  kinect.Close();
}

We’ve continued the hackathon with implementing the bits that saved the pictures in Azure BlobStorage. The file references to the BlobStorage were saved in a ravenDB on ravenHQ. Later on we’ve implemented an ASP.NET MVC service on Azure websites that served the pictures taken to a Windows Store App. While implementing the picture download in the Windows Store App we unfortunately ran out of time. This was the App when we had to stop.
PartyCrasherApp

Anyways the whole hackathon was really good fun. Big thanks to HeiRes and Microsoft and maybe some time in the future we’ll finish the PartyCrasher-App.

Advertisements

Azure (WebSite-) deployment additional dependencies

If you want to publish your (Website-) project from within Visual Studio via WebDeploy you need to watch out that all files that you actually need on the server are included in the WebDeploy package. You’ll get problems if you dynamically load Assemblies via Assembly.LoadFile or Assembly.LoadFrom. If you haven’t referenced those Dlls or VS projects directly these assemblies won’t be included in the WebDeploy package and the WebSite won’t work in Azure.

To include files that are not directly referenced in your project edit the (WebProject) *.csproj-File in the following way:

<!-- To modify your build process, add your task inside one of the targets below and uncomment it.
       Other similar extension points exist, see Microsoft.Common.targets. -->
  <Target Name="BeforeBuild">
  </Target>
  <Target Name="AfterBuild">
    <ItemGroup>
      <Content Include="[path to your dll]" /> <!-- the dll that should be included in the WebDeploy-package -->
    </ItemGroup>
  </Target>

Framework soup

Over the last months I’ve spent quite some time checking out the Kinect for Windows SDK and it’s really fun to work with this awesome little piece of hard and software.

While playing around with the Kinect for Windows SDK it soon became obvious that I’ll need some sort of framework that handles all the tough event handling stuff for me. Thats were the ReactiveExtensions framework came into play. Since I’ve first read about it I wanted to use it somewhere but didn’t get the chance so far. Now that the Kinect came around the corner with its event driven API Rx fitted perfectly to my requirements.

Another framework that impressed me a lot is SignalR, an abstraction layer for persistent connections over http.

So how do they all come together? In an application I called ‘KickerNotifier‘!
We have a ‘Kicker’ (thats german, in english foosball/tabletop football/tabletop soccer) to play with in the cellar of our office.

Problem is that the table stands in the cellar and you never know whether it is busy.

So the idea is the following: Set up a Kinect that is able to track the number of people in the room and push the amount to a website so that all the colleagues are able to see how many people are playing.

The technology stack:

The SignalR hub class
public class KickerNotifyHub : Hub
{
    private Int32 playerCount = -1;
    public void SetPlayerCount(Int32 count)
    {
        if (this.playerCount != count)
        {
            this.playerCount = count;
            Clients.setCurrentPlayerCount(this.playerCount);
        }
    }
}
Webpage hub connection
<script type="text/javascript" src="Scripts/jquery-1.6.4.js" />
<script type="text/javascript" src="Scripts/jquery.signalR-0.5.3.js" />
<script src="signalr/hubs" type="text/javascript" />
$(function () {
    var hub = $.connection.kickerNotifyHub;

    hub.setCurrentPlayerCount = function (count) {
        $('#currentPlayerCount').text(count); // updates <div id="currentPlayerCount" />
    };

    $.connection.hub.start();
});
Client console application
using System.Reactive.Disposables;
using System.Reactive.Linq;
using SignalR.Client;
using SignalR.Client.Hubs;

// some method

var personNotification = new PersonNotification();
var connection = new HubConnection("http://???????");
var hub = connection.CreateProxy("KickerNotifyHub");
var setPlayerCountSubscription = Disposable.Empty;
connection.Start().ContinueWith(task =>
{
   if (task.IsFaulted)
   {
      Console.WriteLine("Failed to start: {0}", task.Exception.GetBaseException());
   }
   else
   {
      Console.WriteLine("Success! Connected with client connection id {0}", connection.ConnectionId);

      setPlayerCountSubscription = Observable.Interval(TimeSpan.FromSeconds(1))
                                             .Subscribe(l => hub.Invoke("SetPlayerCount", personNotification.PersonCount));
   }
});

Console.ReadLine();
setPlayerCountSubscription.Dispose();
personNotification.Dispose();
PersonNotification.cs
public class PersonNotification
{
   private readonly KinectSensor kinect;
   private readonly IDisposable newSkeletonDataEvent;

   public PersonNotification() : IDisposable
   {
      this.kinect = KinectSensor.KinectSensors
                                .FirstOrDefault(s => s.Status == KinectStatus.Connected);
      if (this.kinect == null) throw new InvalidOperationException("No Kinect connected.");

      this.newSkeletonDataEvent = Observable.FromEventPattern(this.kinect, "SkeletonFrameReady")
                                            .Select(e => e.EventArgs)
                                            .Subscribe(NewSkeletonData);

      this.kinect.SkeletonStream.TrackingMode = SkeletonTrackingMode.Seated;
      this.kinect.SkeletonStream.Enable();
      this.kinect.Start();
   }

   private void NewSkeletonData(SkeletonFrameReadyEventArgs skeletonDataFrame)
   {
      using (var frame = skeletonDataFrame.OpenSkeletonFrame())
      {
         if (frame == null) return;

         var skeletons = new Skeleton[frame.SkeletonArrayLength];
         frame.CopySkeletonDataTo(skeletons);

         var personCount = skeletons.Count(s => s.TrackingState == SkeletonTrackingState.PositionOnly ||
                                                s.TrackingState == SkeletonTrackingState.Tracked);
         if (this.PersonCount != personCount)
         {
            this.PersonCount = personCount;
         }
      }
   }

   public Int32 PersonCount { get; private set; }

   public void Dispose()
   {
      this.newSkeletonDataEvent.Dispose();
   }
}