AstraSDK 2.07 Astra Pro overlaying depth stream over HD stream

Technical Support & Q&A Advanced
1

I’m using Astra Pro sensor. I was trying to modify the SimpleStreamViewer-SFML to able to overlay the depth stream over HD color stream, but I don’t see an easy way to do it when the depth and color stream have different resolutions, is there any way to do it through SDK ?

2

Hello @pdondziak,

This is just my guess (I didn’t try it):

  • we can assume that principal point of RGB camera is relatively near to center of image in both modes (VGA and HD)

  • also, we can assume that vertical field of view is the same in VGA and HD modes

  • then we can calculate coordinates in VGA mode from coordinates from HD mode and vice versa:

    Yvga = 240 + (Yhd - 360) * 480 / 720;
    Xvga = 320 + (Xhd - 640) * 640 / 960; // 960 = 720 * 4 / 3

    Yhd = 360 + (Yvga - 240) * 720 / 480;
    Xhd = 640 + (Xvga - 320) * 960 / 640;

Use Xvga and Yvga in mapping. Then calculate corresponding Xhd and Yhd.

But actually, principal point can be slightly shifted from image center, vertical field of view may be different in VGA and HD modes, also distortion can be significant. In this case we have to calibrate (determine intrinsic parameters including principal point position, focal length and distortion coefficients) RGB camera in both modes (VGA and HD) and use these parameters to fit pixels in both modes.

3

Hey Pawel,
did you solve this issue? I would like to know how you implemented this.

could you please share some sample code?

Thanks,
Prasanna