Nuitrack vs Gestoos for hand gesture recognition?

Technical Support & Q&A
1

Hi,

Any one having experience and knowledge about what sdk is better for hand gesture recognition using Astra Pro?

Thanks,
Nitish

2

FWIW - the gestoos C++ sdk is stuck using a 5 year old version of visual studio and limited to 320x240 resolution.
We looked at it early on - but had a very limited and poor experiance with it - would be interested if anyone has had any better luck

Westa

3

Thanks Westa, as per my findings from other sources as well I have come to conclusion that Nuitrack would be a better choice.

4

The only weakness of nuitrack right now is the lack of ability to train new gestures

Westa

5

I haven’t seen a way in Unity with Nuitrack to create a color mask. I see some silhouette masks, but nothing where the mask is filled in with the video. Any ideas?

Steve

6

Hi Steve,

I am facing the same issue. Westa told me to see color mask in Nuitrack’s pointcloud scene we need to set the id of the usb camera in the nuitrack.config - then open the color stream from nuitrack. But I haven’t had any luck locating my Astra Pro camera ID. In case you find out how we can locate Camera ID for Astra Pro in unity, let us know!

Nitish

7

Gestoos supports hand tracking with overhead-placed cameras whereas Nuitrack out-of-the-box definitely doesn’t seem to. There also doesn’t seem to be any documentation for the nuitrack.config file (at least that I can find) so maybe there’s a hidden setting in there somewhere.

8

Nuitrack is largely based on a user recognition system and a subsquent skeletal tracking system - as such it ideally needs to be able to identify a person to work optimally by default - not a full skeleton as such but at least recognise a body shape, this is how nuitrack handles avoidance of interuptions from more than one hand in the view area - its a very different kind of system to gestoos in that sense.

HOWEVER. there is a setting in the nuitrack.config that does effectively decouple the skeleton system in some way = though not clearly documented

"HandTracker": {
    "SkeletonSupport": true,

You might try setting this to false.

Westa

wESTA

9

Setting HandTracker.SkeletonSupport to false makes all of the programs I’ve tested immediately crash with a Segmentation fault (core dumped). Is there something else that has to be done for this to work?

10

Hi Steve,

If you are using unity3D you can use Unity’s build in class to get texture with the help of below code. Make sure to use Raw Image as UI element.

public class WebCamTextureOnUI : MonoBehaviour
{
public RawImage rawimage;
void Start ()
{
WebCamTexture webcamTexture = new WebCamTexture();
rawimage.texture = webcamTexture;
rawimage.material.mainTexture = webcamTexture;
webcamTexture.Play();
}
}

Thanks,
Nitish

11

Hi Nitish,
That will display the video image, but I also want to mask it so that it only shows the person as a cutout, like a green screen so that I can put the person over a different background. The Orrbec skeletal tracker has this available as the “MaskedColorMapViewer”. I don’t see anything like this in Nuitrack. I guess I could use the Nuitrack single color silhouette/mask as a cutout mask on top of the video stream.

Thanks,
Steve

12

Hi Steve,

In case you are using Orbbec Astra PRO and Windows platform you should enable color stream manually:

  1. open %NUITRACK_HOME%/data/nuitrack.config in a text editor;
  2. set “AstraProPerseeDepthProvider.Windows.CameraID” value to your camera ID (it’s 0 usually, if it doesn’t work try 1,2,… and so on).

I hope it helps!

Thanks,
Nitish

13

I have the same problem, The effect of Hand Tracking is very poor and the system can not recognize gesture.