Custom IKinectController – Drag and Drop

Drag and Drop sample with IKinectManipulateableController

This sample shows how you can implement a custom KinectControl, for example to move things around in a Canvas.

To hook into the whole KinectRegion magic you can implement your own UserControl that implements IKinectControl.
When you move your hand KinectRegion keeps track of the movements and will constantly check if there is some Kinect enabled control at the current hand pointer’s position.
To determine such a control the KinectRegion looks for IKinectControls.

The IKinectControl interface forces you to implement the method IKinectController CreateController(IInput inputModel, KinectRegion kinectRegion).
The IKinectControl is also required to implement IsManipulatable and IsPressable.
This way you are specifying on what gestures your control will react to.
Because this sample shows how to move controls around the Draggable control’s property IsManipulatable returns true.

UserControl Draggable.cs implements IKinectControl

public sealed partial class Draggable : UserControl, IKinectControl
{
    public Draggable()
    {
        this.InitializeComponent();
    }

    public IKinectController CreateController(IInputModel inputModel, KinectRegion kinectRegion)
    {
        // Only one controller is instantiated for one Control
        var model = new ManipulatableModel(inputModel.GestureRecognizer.GestureSettings, this);
        return new DragAndDropController(this, model, kinectRegion);
    }

    public bool IsManipulatable { get { return true; } }

    public bool IsPressable { get { return false; } }
}

You’ll find two interfaces in the SDK *.Controls namespace that implement IKinectController and of course they are closely related to the other properties you have to implement.

Those interfaces are:
– IKinectPressableController
– IKinectManipulateableController

Since IsManipulatable returns true CreateController should return an instance of IKinectManipulateableController.
For an example how to implement IKinectPressableController see here.

Controller class DragAndDropController

public class DragAndDropController : IKinectManipulatableController, IDisposable
{
}

The DragAndDropController‘s constructor gets a reference to the manipulatable control, an IInputModel and a reference to the KinectRegion.

public DragAndDropController(FrameworkElement element, ManipulatableModel model, KinectRegion kinectRegion)
{
    this.element = new WeakReference(element);
    this.kinectRegion = kinectRegion;
    this.inputModel = model;

    if (this.inputModel == null)
        return;
...

The ManipulatableModel provides four events you can subscribe to in order to react to user input.
This sample uses the nuget package Kinect.ReactiveV2.Input that provides specific Rx-extension methods to subscribe to these events.

...
    this.eventSubscriptions = new CompositeDisposable 
    {
        this.inputModel.ManipulationStartedObservable()
                       .Subscribe(_ => VisualStateManager.GoToState(this.Control, "Focused", true)),

        this.inputModel.ManipulationInertiaStartingObservable()
                       .Subscribe(_ => Debug.WriteLine(string.Format("ManipulationInertiaStarting: {0}, ", DateTime.Now))),

        this.inputModel.ManipulationUpdatedObservable()
                       .Subscribe(_ => OnManipulationUpdated(_)),

        this.inputModel.ManipulationCompletedObservable()
                       .Subscribe(_ => VisualStateManager.GoToState(this.Control, "Unfocused", true)),
    };
}

All subscriptions are composed into one CompositeDisposable that is disposed in the controllers’s Dispose() method.

• ManipulationStartedObservable –> is fired when the user closes it’s hand
• ManipulationInertiaStartingObservable –> ???
• ManipulationUpdatedObservable –> is fired when the user moves it’s hand while keeping it closed
• ManipulationCompletedObservable –> is fired when the user releases it’s hand

For this sample the most interesting observable is ManipulationUpdatedObservable. Everytime this event is fired the method OnManipulationUpdated is called.

private void OnManipulationUpdated(KinectManipulationUpdatedEventArgs args)
{
    var dragableElement = this.Element;
    if (!(dragableElement.Parent is Canvas)) return;

    var delta = args.Delta.Translation;
    var translationPoint = new Point(delta.X, delta.Y);
    var translatedPoint = InputPointerManager.TransformInputPointerCoordinatesToWindowCoordinates(translationPoint, this.kinectRegion.Bounds);

    var offsetY = Canvas.GetTop(dragableElement);
    var offsetX = Canvas.GetLeft(dragableElement);

    if (double.IsNaN(offsetY)) offsetY = 0;
    if (double.IsNaN(offsetX)) offsetX = 0;

    Canvas.SetTop(dragableElement, offsetY + translatedPoint.Y);
    Canvas.SetLeft(dragableElement, offsetX + translatedPoint.X);
}

If this IKinectControl is placed inside a Canvas it’s position is updated relative to the users hand.

The complete code is found here.
This article in markdown is found here.