Floor Plane calculation

[johanromuald]

Hi,

if (bodyFrame.floor_detected())
{
const auto& p = bodyFrame.floor_plane();
str << "Floor plane: ["
<< p.a() << ", " << p.b() << ", " << p.c() << ", " << p.d()
<< "]" << std::endl;
}

How reliable these values ? p.d() , the height of the sensor is not correct.

What is BodyMask bodymask = bodyFrame.body_mask(), and FloorMask floorMask = bodyFrame.floor_mask();

How can I use them in frame?

Please help.

Thank you

[wtatters]

Body mask is a array of integers the same size as the depth frame ... Each point holds a value representing the id of any current body detected at that location in the depth frame.

Floor mask is similar ... But only has a value representing the floor being detected at a point in the depth frame.

As to how accurate or reliable the math is. ... sadly It's not very good IMHO

Westa

[johanromuald]

Hi , Thank you for that.

I have used Kinect in my application to detect Floor plane and it was very reliable. I need to have that info for my application. Have you Tried NUITRACK. Is it better than Orbbec body Tracking SDK?

Can I convert Body Mask frame to OpenCV Mat frame and change the color of the area where the body is detected?

Thanks once again for taking time to reply.
J

[wtatters]

I would have to say that nuitrack SDK produces better results for the most part ... But ... Some of the limiting factors seem to be more related to technical issues with the current orbbec sensor hardware/ firmwwre ... And in accurate depth reporting.

The masks are not colour as in RGB ... They are single byte per depth location memory blocks ... How you use these for visualisation is up to you... The samples in the SDK demonstrate a couple of techniques for displaying the masks.

Westa

[johanromuald]

Thank you for that. I will have look at the samples .
Best regards
J

[johanromuald]

Hi Westa,

include "orbbec/bodytracking/bodytracking.hpp"

include "OpenNI.h"

include "astra/astra.hpp"

include "cstdio"

BodyMask is showing with error "undefined".

Please help.

Thank you.
J

[johanromuald]

Hi,

I tried the following;

//created int array same size of depth frame
int Atable[(640 * 480)];

// Create mat container gray scale image
Mat imgr(480, 640, CV_8UC1, Scalar(0, 0, 0));

const auto bodymask = bodyFrame.body_mask();
auto bb = bodymask.data();

		  int h = bodymask.height();
		  int w = bodymask.width();

		for (int i = 0; i <(h*w); i++)
		{

			if (bb[i] > 0)
			{
				Atable[i] = 255;
			}
			else {
				Atable[i] = 0;
			}

		}

		imgr = Mat(480, 640, CV_8UC1, Atable);

		
		cv::imshow("video", imgr);

But I can only see the head of person .

Can you please advice what is wrong the above method.

Thanks
J

[wtatters]

Using the latest 2.0.8 beta 2 orbbec sdk - you do not need to explicitly include the bodytracking.hpp if you have your include paths setup correctly.

#include <astra/astra.hpp> will handle the mapping of all other required includes.

Westa

[wtatters]

Are you using polling or event based processing of the frame?

I assume you have setup the frame pump and are getting a valid body track?

What is the value of: bodyFrame.bodies().size():

//////////////////////////////////////
here is some sample code

// get depth frame metrics
int width = depthFrame.width();
int height = depthFrame.height();
const int depthMapLength = width * height;

// get depthData
const int16_t* depthData = depthFrame.data();

// get bodyMask
const auto& bodyMask = bodyFrame.body_mask();
const auto* bodyData = bodyMask.data();

bodyCount = bodyFrame.bodies().size();

std::uint8_t value = 0;

// we are mapping directly to a bitmap
// first 12 bytes are allocated for bitmap image type and sizing
int rgbaOffset = 12;

for (int i = 0; i < depthMapLength; i++)
{
	const auto bodyId = bodyData[i];
	const auto depth = depthData[i];

	// quick modulo of depth mapping to put gradient of depth mask on screen
    value = (depth % (255*4) / 2 );

    colorRGBA thisColor = { value , value , value , 255 };

    if ( bodyCount > 0 )
    {
        // lazy map a couple of colors for bodies  -  use a lookup table normally
        if ( bodyId == 1 )
        {
            thisColor = { 255 , 0 , 0 , 255 };

        }
        else if ( bodyId == 2 )
        {
            thisColor = { 0 , 255 , 0 , 255 };
        }

    }

    // then map color 

		memBlkPtr[rgbaOffset] = thisColor.r;
		memBlkPtr[rgbaOffset + 1] = thisColor.g;
		memBlkPtr[rgbaOffset + 2] = thisColor.b;
		memBlkPtr[rgbaOffset + 3] = thisColor.a;

		// increment offset ready for next value
		rgbaOffset = rgbaOffset + 4;

}

/// then we map bitmap to texture etc

[johanromuald]

Hi,

Thank you for that.

I got it working. The mistake I was making was converting to OpenCV Mat format.

Here is the working code;

Mat imgr(480, 640, CV_8UC1, Scalar(255, 255, 255));
const auto bodymask = bodyFrame.body_mask();
auto bb = bodymask.data();

		  int h = bodymask.height();
		  int w = bodymask.width();



		for (int x = 0; x < w; x++)
		{
			for (int y = 0; y < h; y++)
			{
				
				if (bb[(x + (y * w))] > 0)
				{
					imgr.at<uchar>(y, x) = 0;
				}
				else
				{
					imgr.at<uchar>(y, x) = 255;
				}

			}
		}

//////////////////////////////////

Thank you for your help.
J

[petitbas]

Did you figure out the floor plane calculation for Nuitrack?

[wtatters]

Nuitrack provides a lot of different data - what particular info were you having trouble with >

Westa

[petitbas]

I'm looking for the joint data relative to the floor. Right now the skeleton data appears halfway through the floor.

[petitbas]

Anyone have a solution?

[shmlex]

The solution is a basic 3D geometry calculation, the original coordinate system located on a camera (world coordinates), the destination coordinate system located on a floor, you need to calculate transformation from original to destination and apply it on each joint per each frame

[cquigley]

I found that it was easier to counter-rotate the "JointRoot" object to account for the rotation of the floor plane. The JointRoot is the parent of all generated skeleton joints/nodes. This was much easier than trying to apply a translation/rotation matrix to each joint, individually.

Keep in mind, the Astra's floor plane also includes the height of the camera (or, rather, the distance to the floor from the camera). So, you can set the JointRoot's y-position, and then counter-rotate it to account for the slope of the floor plane. When doing so, you must remember that the Astra is giving you joints with the x-position data that is mirrored, so it looks good in the demo...the floor plane data is also "x-position inverted".