Intrinsic camera parameters from Python + Openni2

Technical Support & Q&A Advanced
1

I came to know the new Orbbec cameras are factory calibrated and that there is no need to do the calibration. I want to access the camera parameters. Is it an OpenNI2 Python to access camera parameter and Point cloud data? PLEASE give a code snippet that shows how to access camera parameter and Point cloud in Python.


def main():
    try:
        #openni2.initialize()
        dev = openni2.Device.open_any()
        depth_stream = dev.create_depth_stream()
        depth_stream.start()
        depth_stream.set_video_mode(c_api.OniVideoMode(pixelFormat = c_api.OniPixelFormat.ONI_PIXEL_FORMAT_DEPTH_1_MM, resolutionX = width, resolutionY = height, fps = frameRate))         
        color_stream = dev.create_color_stream()  
        color_stream.set_video_mode(c_api.OniVideoMode(pixelFormat = c_api.OniPixelFormat.ONI_PIXEL_FORMAT_RGB888, resolutionX = width, resolutionY = height, fps = frameRate))
        
        
        color_stream.start()
        # Register both streams so they're aligned
        dev.set_image_registration_mode(c_api.OniImageRegistrationMode.ONI_IMAGE_REGISTRATION_DEPTH_TO_COLOR)
        
        while True:
            depth_framedata = depth_stream.read_frame()
            color_frame = color_stream.read_frame()
        
            # Display depth and color images
            depth_frame = depth_framedata.get_buffer_as_uint16()
            depth_image = np.frombuffer(depth_frame, dtype=np.uint16)
            depth_image.shape = (1, height, width)
           
            depth_image_display = np.concatenate((depth_image, depth_image, depth_image), axis=0)
            depth_image_display = np.swapaxes(depth_image_display, 0, 2)
            depth_image_display = np.swapaxes(depth_image_display, 0, 1)
            depth_image_display = depth_image_display.astype(np.uint8)
    
            

           
            color_image = np.frombuffer(color_frame.get_buffer_as_uint8(),dtype=np.uint8)
            color_image.shape = (height, width, 3)

            cv2.imshow("Depth Image", depth_image_display)
            color_image_BGR = cv2.cvtColor(color_image, cv2.COLOR_RGB2BGR)
            cv2.imshow("Color Image", color_image_BGR)
            cv2.waitKey(1)
2

I hope it still helps and you sometimes look into this forum.
I can’t give you a snippet as I’m not working with OpenNI. But I’m using Python with the Orbbec SDK so maybe there is a similar way. Many functions within the SDK I just found out using the dir() function of Python. If you don’t know it: It shows you all properties and methods of an object. In your case I’d use it on your device: dir(dev). With the Orbbec SDK I get a list of methods I can use on “dev” and one of them is “getCalibrationCameraParamList”. So I call sth. like param_list=dev.getCalibrationCameraParamList(). Then you use dir() again: dir(param_list) and you get a method called getCameraParam. This you can call with param=param_list.getCameraParam(0). You have to give a number to chose from the list. This gives you a dictionary with all parameters from your sensor (e.g. depthIntrinsic as another dictionary).

check methods with dir(dev)
param_list = dev.getCalibrationCameraParamList()

check methods with dir(param_list)
param = param_list.getCameraParam(0)

For point cloud generation I use a self-written function with Numpy. First I have to bring the intrinsic depth parameter into a camera matrix form and then straight forward:

K = np.zeros((3,3))
K[0,0] = param[‘depthIntrinsic’][‘fx’]
K[1,1] = param[‘depthIntrinsic’][‘fy’]
K[0,2] = param[‘depthIntrinsic’][‘cx’]
K[1,2] = param[‘depthIntrinsic’][‘cy’]
K[2,2] = 1
PCL = np.nan*np.zeros((height, width, 3)) # pixels without depth value I consider NaN in the point cloud
for i in range(height):
for j in range(width):
if depth_image[i,j]==0:
continue
else:
PCL[i,j,2] = depth_image[i,j] # z-value
PCL[i,j,0] = depth_image[i,j] * (j - K[0,2]) / K[0,0] # x-value
PCL[i,j,1] = depth_image[i,j] * (i - K[1,2]) / K[1,1] # y-value

Alternatively you can use an OpenCV (cv2) function which is probably faster:
PCL = cv2.rgbd.depthTo3d(depth_image,K)

I hope that helps and is similar enough to OpenNI. (And that all indizes are correct esp. regarding your depth_image.)

Edit: Sorry, I don’t know how to correctly indent the code snippets.