Here's a nicer example, using an actual screenshot. Let's calculate the (X,Y, Z) coordinate of my fingertip, from this screenshot:
Ignore the white text at the bottom; it's not used in this example
First, let's go with what we currently know:
Z: the distance from the camera plane to my finger. This is the pixel value in the depth frame, and for this example let's say it's 1400. This means the Z coordinate is 1400mm.
x: the horizontal distance from the center of the frame to my finger in the depth frame, measured in pixels. In this example it's 206 pixels.
y: the vertical distance from the center of the frame to my finger in the depth frame, measured in pixels. In this example it's 198 pixels.
FoV_w: the horizontal field of view of the camera. This is provided by Orbbec, and is 60 degrees.
FoV_v: the vertical field of view of the camera. This is provided by Orbbec, and is 49.5 degrees.
W: the width of the frame, 1280
H: the height of the frame, 960
O: the origin for our calculations. It's the center of the frame, and in this example is (640, 480), because the frame is 1280x960.
We have all the information we need, so let's first calculate X, the real-world horizontal distance from the center of the camera's field of view to my finger, in mm.
The horizontal angular distance from my finger to the center of the camera's field of view, theta_w, is calculated by:
theta_w = FoV_w / W * x = 9.656º
Using some simple trigonometry we know that tan(theta_w) = X/Z. Therefore:
X = Z * tan(theta_w) = 238.206mm
Using the exact same process, but substituting for the vertical measurements we get:
theta_v = FoV_v / H * y = 10.209º
Y = Z * tan(theta_v) = 252.136mm
Rounding to the nearest mm, this gives us
(X, Y, Z) = (238, 252, 1400)
Converting to meters this means my finger is:
- 1.4m away from the camera
- 0.25m above the camera
- 0.24m to the right of the camera (left/right may vary depending on whether you have mirroring on or not)
@pkr97, in your case, where did you get phiX, phiY, thetaX, and thetaY from? When you say you get "random values" how random are you getting?
Remember that the Astra's depth stream may have holes, so you should ignore pixels where the Z value is 0. Similarly, you should expect some shimmer, as the depth values change from frame-to-frame within a few mm.
As for the CoordinateMapper class, where did you read that this class should exist? Can you post a link to the relevant documentation?