Is it possible to use OpenNI with the Astra Pro?

Technical Support & Q&A
1

Is it possible to use OpenNI with the AstraPro? I’d like to start with OpenNI and just the proximity sensor.

Jim

2

Regarding OpenNI 2 - Astra (for both depth and color) and Astra Pro (just depth, no color) have drivers for OpenNI 2.2+

The proximity sensor, however, is a safety feature that is (at the moment) not directly user controllable that turns off the IR projector when something is detected to be too close to the front of the device.

The lack of support for color streams in OpenNI2 with the Astra Pro is due to the non-trivial changes required to OpenNI 2 to enable support of the higher resolution UVC-based camera CMOS. We have been considering making the necessary changes, but are prioritizing new Astra SDK features over retrofitting OpenNI 2. Of course, you can use the Astra SDK to retrieve all stream types from all of our devices: Astra, Astra Pro, and Persee.

3

Just confirmed by our FW team that they will provide code to turn on/off the proximity sensor :smile: . Sounds sweet?

4

Hi! I’m using Linux and OpenNI to connect to the Astra. Will this functionality to switch on/off the proximity sensor be available in that framework?

5

More to this question. I’ve found that unplugging the sensor gets around this problem for the meantime. Can you guarantee that there won’t be any changes to the firmware that will disable the projector if the safety sensor isn’t powered?

6

@gordon [quote=“gordon, post:2, topic:50”]
Of course, you can use the Astra SDK to retrieve all stream types from all of our devices: Astra, Astra Pro, and Persee.
[/quote]

Personally, only the depth stream is accessible with Astra SDK.
As an example:

astra::Astra::initialize();
astra::StreamSet streamSet;
astra::StreamReader reader = streamSet.create_reader();
reader.stream<astra::InfraredStream>().start();

astra::Frame frameobjct = reader.get_latest_frame();
auto test = frameobjct.get<astra::InfraredFrame16>();

std::cout << “Press any key to continue…”;
std::cin.get();

reader.stream<astra::InfraredStream>().stop();
astra::Astra::terminate();

Will be stuck at:

2016-02-10 12:01:04,355 INFO [PointProcessor] created point stream

Maybe i am doing it wrong but at this point either Color and Infrared data are not available.

7

This issue is fixed by adding:

astra::ImageStreamMode irMode;

irMode.set_width(640);
irMode.set_height(480);
irMode.set_pixelFormat(astra_pixel_formats::ASTRA_PIXEL_FORMAT_RGB888);
irMode.set_fps(30);

irStream.set_mode(irMode);

before starting the stream :slightly_smiling:

1 Like
8

Hi Victor,

i am trying the snippet you posted

astra::StreamSet streamSet;
astra::StreamSet streamSetC;
astra::StreamSet streamSetIr;

astra::StreamReader readerD = streamSet.create_reader();
astra::StreamReader readerC = streamSetC.create_reader();




readerD.stream<astra::DepthStream>().start();
readerC.stream<astra::ColorStream>().start();

astra::ImageStreamMode irMode;

irMode.set_width(640);
irMode.set_height(480);

irMode.set_pixel_format(astra_pixel_formats::ASTRA_PIXEL_FORMAT_RGB888);
irMode.set_fps(30);


astra::StreamReader readerIr = streamSetIr.create_reader();
readerIr.set_mode(irMode);

But i cant get it work.

Kind regards.

9

My mistake: Replace

ASTRA_PIXEL_FORMAT_RGB888

by

ASTRA_PIXEL_FORMAT_GRAY16

1 Like
10

Hi Victor,

it seems to work for me.

my Main code is like this:

int main(int argc, char** argv)
{

astra::initialize();
Mat depthimage;
Mat color;

astra::StreamSet streamSet;
astra::StreamSet streamSetC;
astra::StreamSet streamSetIr;




astra::ImageStreamMode irMode;

irMode.set_width(640);
irMode.set_height(480);
irMode.set_pixel_format(astra_pixel_formats::ASTRA_PIXEL_FORMAT_GRAY16);
irMode.set_fps(30);



astra::StreamReader readerD = streamSet.create_reader();
//astra::StreamReader readerC = streamSetC.create_reader();
astra::StreamReader readerIr = streamSetIr.create_reader();    

readerIr.stream<astra::InfraredStream>().set_mode(irMode);



readerD.stream<astra::DepthStream>().start();
//readerC.stream<astra::ColorStream>().start();
readerIr.stream<astra::InfraredStream>().start();


do
{
    
    //astra::Frame cFrame = readerC.get_latest_frame();
    //const auto colorFrame = cFrame.get<astra::ColorFrame>();

    astra::Frame frame = readerD.get_latest_frame();
    const auto depthFrame = frame.get<astra::DepthFrame>();
    
    //Mat rgbMat = colorframe2mat(colorFrame);
    depthimage = depthframe2mat(depthFrame);

    //astra::InfraredFrame16 irFrame = readerIr.get_latest_frame();
    astra::Frame irFrame = readerIr.get_latest_frame();
    const auto irframe = irFrame.get<astra::InfraredFrame16>();

    //imshow("RGB", rgbMat);
    imshow("DEPTH", depthimage);
    

    if (waitKey(30) >= 0) break;

} while (1);


astra::terminate();

return 0;

}

I am converting the Depth image and the color image to cv::Mat. But i am not sure how to convert the Ir image to Mat.

EDIT:
Ok I have it

Mat irframe2mat(astra::InfraredFrame16 irFrame)
{
int width = irFrame.width();
int height = irFrame.height();
Mat image(height, width, CV_8UC1);

int grayval = 0;

for (int y = 0; y < height; y++)
{
    for (int x = 0; x < width; x++)
    {
        grayval = irFrame.data()[x + y*width] / (3500 / 256);
        image.at<uchar>(y, x) = grayval;
    }
}
return image;

}

2 Likes
11

Smoli, I’ve been looking at trying to get the depth from astra converted to mat, is there an example on how you did your conversion (or your actual conversion, if you don’t mind sharing)?