Astra Pro - Depth to color registration problem

#1

Hello,

I bought a “Astra Pro” to replace a Asus Xtion. I’m using openni 2 for de depth stream and the IR stream. The function “Register Depth to color” is not working properly. I also tried a stereo calibration between the IR and RGB camera (UVC) but it doesn’t work. Yet, i think that my stereo calibration is good! It seems that the depth image does not correspond perfectly to the IR image. Is there any cropping (I have tried to disable it, but it doesn’t work) ?

Perhaps do you have a default calibration (Intrinsics of RGB camera + IR + Relative motion between the two cameras) ?

Could you help me because, actually this sensor is unusable for me.

#2

If the error you see is a vertical offset, yes there is some cropping magic going on so the depth does not correspond 1:1 with IR. I believe the offset is somewhere between 8-12 pixels in Y at VGA resolution, which you have to compensate for before feeding the data into your registration.

Someone from Orbbec may want to comment on this with more detailed information.

#3

Hello Rafael,

First of all thanks for your reply. I find that the astra pro is a handiwork compared to the xtion which is very easy to calibrate and compatible with openni, I am a little disappointed with the product. To get a good calibration with the astra pro is very difficult.

Are you Rafael Spring from DotProduct?

#4

Hi Alexandre,

yes that’s me. Small world :slightly_smiling:
We have mainly worked with the regular Astra so far so I can’t say anything about the Pro but I think it has the same IR / depth components.

One thing that helped us was to use the OpenNI 2 interface with the Astra instead of the “native” drivers.

#5

Hi Raphael,

The problem with Astro pro is that they have changed the RGB camera but not updated the automatic depth to RGB registration. I have not found the solution for now. I have done a calibration from IR to RGB but the depth map does not correspond. If you working on it and you will find a solution, I’m interested… :wink:

I’m not surprised to find you here with the end of the xtion…

PS: I’m working with OPENNI2 too.

#6

The built-in (hardware) registration also has not worked for us. Our solution is to shift the depth image in Y by some amount before doing the registration (in software). We’re calibrating the Astra by hand today and with the added shift we get good results.
We use an Astra (not Pro) though so it may be different to the Pro.

#7

Hello Rafael,.

Thank you for your time. I like to understand technically why I do things. I find it unfortunate that Orbbec have not better documented the Astra pro or provided a basic calibration or a tutorial. How do they technically justify this shift ? Why the depth map is not aligned with the IR? This is not very serious…

Good continuation and good success with your scanner ! I saw the latest developments, it is really very good.
See you soon.

Alexandre

#8

I agree, more documentation would be extremely helpful. I don’t know the technical justification for the shift but I know that Primesense / Xtion cameras have a similar shift that’s due to a sliding window that’s used by the speckle pattern correlation algorithm. I can’t say whether that’s the same for the Astra though.

#9

Registration does seem to work when you use 640x480 for the color stream, only when using 1280x720 for color there seem to be shifts.
At least using the native (non-OpenNI) SDK that is.

Also it appears that the cropping on 640x480 is narrower than 1280x720 as expected but also a bit taller.
So simply scaling things in our software implementation most probably won’t work, or at best it would be difficult to find working parameters.

Looks like this is something Orbbec should solve in their drivers/SDK.