At my company (Pixformance) we are trying to use Orbbec with body tracking to replace Kinect, but we are struggling with several problems. Currently the biggest one is strange performance problems which manifest itself when we try to use all 3 streams depth/color/body at the same time by stopping detecting bodies. I would like to make proper performance analyse but I would need to have debugging symbols for OrbbecBodyTracking.dll and orbbec.dll.
What exactly do you mean by the term try to use all 3 streams?
Technically the orbbec cameras deliver Depth,RGB and IR streams. They don’t deliver a body stream as such. To obtain body data, the OrbbecBodyTracking system works by processing information from the Depth stream.
For starters - in your test system - are you able to visualise the Depth stream as say an grayscale image at the same time as you are visualising the RGB stream.
Westa
In our software we are displaying color stream and we are using depth and body for exercise detection. When I hooked those 3 streams to our software the depth was available the color stream was displayed but the bodies were not detected. As soon as I turned off the color stream bodies were detected as usual. I was able to reproduce the same behavior in the BodyViewer example from the bodytracking SDK just by stressing the main while loop with some sorting. The body frames contains the floor pane but no bodies. I will try to prepare a minimal example to post it here.
I will make conjectures that the body tracking algorithm is not stateless it other words it doesn’t work per frame as the Kinect algorithm but it is prone to depth stream frame drop. Can you correct me if I’m wrong?
There have been discussions in the past about the depth and rgb streams conflicting with each other - but in our testing we have been using the astra pro which feeds us a separate color display using the universal video usb system instead of openNI - so its not likely we would see the issue you are seeing.
Are you by chance running different resolutions for your streams or are they all 640x480?
Also are you running async or sync frame capture - it is possible that the processor is simply running out of grunt to get you frames. We have seen some situations where we drive our test system down to around 15fps - and are still seeing reliable body frame data.
Westa