How to use Pipeline() and Context() simultaneously without destroying Context?

I am trying to get all the sensors for a certain project:
OBSensorType.COLOR_SENSOR
OBSensorType.DEPTH_SENSOR
OBSensorType.ACCEL_SENSOR
OBSensorType.GYRO_SENSOR
For getting the depth images and colour images, I need to get them from the pipeline:

config = Config()

pipeline = Pipeline()

and then

profile_list = pipeline.get_stream_profile_list(OBSensorType.DEPTH_SENSOR)
assert profile_list is not None
try:
    depth_profile = profile_list.get_video_stream_profile(640, 0, OBFormat.Y16, 30)
except OBError as e:
    print("Error: ", e)
    depth_profile = profile_list.get_default_video_stream_profile()
assert depth_profile is not None

whereas for getting the gyro and accel readings, I have to get them from the sensor list:

ctx= Context()

device_list= ctx.query_devices()

if device_list.get_count() == 0:

print("No device connected")

device= device_list.get_device_by_index(0)

sensor_list= device.get_sensor_list()

and then

gyro_senor = sensor_list.get_sensor_by_type(OBSensorType.GYRO_SENSOR)
if gyro_senor is None:
print("No gyro sensor")
gyro_profile_list = gyro_senor.get_stream_profile_list()
gyro_profile = gyro_profile_list.get_stream_profile_by_index(0)
assert gyro_profile is not None

How can I get both the readings in one way, I am asking this because I am getting this error when I try to use both Context() and Pipeline() which says that I have already created the the device by creating the Context and when I try to create the pipeline, the Context gets destroyed.

[2023-08-25 12:10:17.591266][info][9314][Context.cpp:66] Context created with config: default config!
[2023-08-25 12:10:17.713379][info][9314][DeviceManager.cpp:562] Found 1 device(s):
[2023-08-25 12:10:17.713429][info][9314][DeviceManager.cpp:564]   - Name: Gemini2, PID: 0x0670, SN/ID: AY3A131006B
[2023-08-25 12:10:17.713578][info][9314][LinuxPal.cpp:109] Create PollingDeviceWatcher!
[2023-08-25 12:10:17.724635][info][9314][Gemini2Device.cpp:417] Device.OrbbecGemini2.FirmwareUpgradeFoolproof device firmware upgrade foolproof status true
[2023-08-25 12:10:17.726088][info][9314][Gemini2Device.cpp:116] Gemini2 device created! PID:1648, SN:AY3A131006B, depthMode: {name: Unbinned Dense Default, checksum(hex): 6bb28ca9a0dff5553497f9eecb59f667, optionCode: NORMAL}
[2023-08-25 12:10:17.726112][info][9314][DeviceManager.cpp:481] Device created successfully! Name: Gemini2, PID: 0x0670, SN/ID: AY3A131006B
[2023-08-25 12:10:17.726165][info][9314][Pipeline.cpp:15] Try to create pipeline with default device.
[2023-08-25 12:10:17.726194][warning][9314][ObException.cpp:5] Trying to create a device that you've already created! SN/ID: AY3A131006B, uid: 3-5-9
Traceback (most recent call last):
  File "/home/npd/Dhruv/drone_implementation_tangentbug-dev_stereo/rs_demo_orbbec.py", line 178, in <module>
    pipeline = Pipeline()
               ^^^^^^^^^^
RuntimeError: Caught an unknown exception!
[2023-08-25 12:10:17.770449][info][9314][Gemini2Device.cpp:121] Gemini2 device destroyed! PID:1648, SN:AY3A131006B, depthMode: {name: Unbinned Dense Default, checksum(hex): 6bb28ca9a0dff5553497f9eecb59f667, optionCode: NORMAL}
[2023-08-25 12:10:17.770963][info][9318][DeviceManager.cpp:117] task finish.
[2023-08-25 12:10:17.771053][info][9314][Context.cpp:82] Context destroyed

I want a way to use the readings from all the sensors: depth, gyro and accel. How can I do this?

Try to use pipe to get sensor
Like:
dev = pipe.getDevice();
accelSensor = dev->getSensor(OB_SENSOR_ACCEL);
gyroSensor = dev->getSensor(OB_SENSOR_GYRO);

1 Like

Is this for pyorbbecsdk?
How would the exact code look like?

I used the following code, here chatgpt suggested me to use pipeline = Pipeline(device) and I am now not getting any error related to pipeline and context. But now I am getting different error in the logs. Here is my code:

ctx= Context()
device_list= ctx.query_devices()
if device_list.get_count() == 0:
    print("No device connected")
device= device_list.get_device_by_index(0)
sensor_list= device.get_sensor_list()


config = Config()
pipeline = Pipeline(device)


# Start depth streaming
profile_list = pipeline.get_stream_profile_list(OBSensorType.DEPTH_SENSOR)
assert profile_list is not None
try:
    depth_profile = profile_list.get_video_stream_profile(640, 0, OBFormat.Y16, 30)
except OBError as e:
    print("Error: ", e)
    depth_profile = profile_list.get_default_video_stream_profile()
assert depth_profile is not None

profile_list = pipeline.get_stream_profile_list(OBSensorType.COLOR_SENSOR)
try:
    color_profile = profile_list.get_video_stream_profile(640, 0, OBFormat.RGB, 30)
except OBError as e:
    print(e)
    color_profile = profile_list.get_default_video_stream_profile()


gyro_senor = sensor_list.get_sensor_by_type(OBSensorType.GYRO_SENSOR)
if gyro_senor is None:
    print("No gyro sensor")
gyro_profile_list = gyro_senor.get_stream_profile_list()
gyro_profile = gyro_profile_list.get_stream_profile_by_index(0)
assert gyro_profile is not None


accel_sensor = sensor_list.get_sensor_by_type(OBSensorType.ACCEL_SENSOR)
if accel_sensor is None:
    print("No accel sensor")
accel_profile_list = accel_sensor.get_stream_profile_list()
accel_profile = accel_profile_list.get_stream_profile_by_index(0)
assert accel_profile is not None


config.enable_stream(color_profile)
config.enable_stream(depth_profile)

pipeline.start(config)

and here is the error log:

[2023-08-31 14:30:50.137291][info][116062][Context.cpp:66] Context created with config: default config!
[2023-08-31 14:30:50.616724][info][116062][DeviceManager.cpp:562] Found 1 device(s):
[2023-08-31 14:30:50.616744][info][116062][DeviceManager.cpp:564]   - Name: Gemini2, PID: 0x0670, SN/ID: AY3A131006B
[2023-08-31 14:30:50.616788][info][116062][LinuxPal.cpp:109] Create PollingDeviceWatcher!
[2023-08-31 14:30:50.626001][info][116062][Gemini2Device.cpp:417] Device.OrbbecGemini2.FirmwareUpgradeFoolproof device firmware upgrade foolproof status true
[2023-08-31 14:30:50.627453][info][116062][Gemini2Device.cpp:116] Gemini2 device created! PID:1648, SN:AY3A131006B, depthMode: {name: Unbinned Dense Default, checksum(hex): 6bb28ca9a0dff5553497f9eecb59f667, optionCode: NORMAL}
[2023-08-31 14:30:50.627817][info][116062][DeviceManager.cpp:481] Device created successfully! Name: Gemini2, PID: 0x0670, SN/ID: AY3A131006B
[2023-08-31 14:30:50.628120][info][116062][Pipeline.cpp:42] Pipeline created with device: {name: Orbbec Gemini 2, sn: AY3A131006B}, @0x1B462A0
[2023-08-31 14:30:50.630542][info][116062][Gemini2Device.cpp:614] Depth sensor has been created!
[2023-08-31 14:30:50.635722][info][116062][Gemini2Device.cpp:662] Color sensor has been created!
[2023-08-31 14:30:50.636791][info][116062][GyroSensor.cpp:18] GyroSensor created!
[2023-08-31 14:30:50.636808][info][116062][Gemini2Device.cpp:846] Gyro sensor has been created!
[2023-08-31 14:30:50.639091][info][116062][AccelSensor.cpp:18] AccelSensor created
[2023-08-31 14:30:50.639105][info][116062][Gemini2Device.cpp:804] Accel sensor has been created!
[2023-08-31 14:30:50.645520][info][116062][Pipeline.cpp:236] Try to start streams!
[2023-08-31 14:30:50.645621][info][116062][VideoSensor.cpp:554] start OB_SENSOR_COLOR stream with profile: {type: OB_STREAM_COLOR, format: OB_FORMAT_RGB, width: 640, height: 480, fps: 30}
[2023-08-31 14:30:50.653975][info][116062][VideoSensor.cpp:554] start OB_SENSOR_DEPTH stream with profile: {type: OB_STREAM_DEPTH, format: OB_FORMAT_Y16, width: 640, height: 400, fps: 30}
[2023-08-31 14:30:50.654327][warning][116083][VideoSensor.cpp:402] This frame will be dropped because jpg format verification failure! @OB_SENSOR_COLOR
[2023-08-31 14:30:50.674873][info][116062][Pipeline.cpp:249] Start streams done!
[2023-08-31 14:30:50.674942][info][116062][Pipeline.cpp:232] Pipeline start done!
[2023-08-31 14:30:50.698892][warning][116084][VideoSensor.cpp:407] This frame will be dropped because because the data size is larger than expected! size=480756, expected=448000 @OB_SENSOR_DEPTH
[2023-08-31 14:30:50.761088][warning][116087][FormatConverter.cpp:84] Failed to decompress color frame
[2023-08-31 14:30:50.775094][warning][116062][Pipeline.cpp:310] Wait for frame timeout, you can try to increase the wait time! current timeout=100
The grid map is  200 x 200 .
[2023-08-31 14:30:51.154532][warning][116087][Pipeline.cpp:289] Pipeline source frameset queue fulled, drop the oldest frame!
Testing!!
[2023-08-31 14:30:52.694283][warning][116094][FrameProcessingBlock.cpp:89] Source frameset queue fulled, drop the oldest frame! N11libobsensor19IMUFrameTransformerE@0x 1fd0360
[2023-08-31 14:30:54.155538][warning][116087][Pipeline.cpp:289] Pipeline source frameset queue fulled, drop the oldest frame! [**89 logs in 3001ms**]
[2023-08-31 14:30:55.698651][warning][116094][FrameProcessingBlock.cpp:89] Source frameset queue fulled, drop the oldest frame! N11libobsensor19IMUFrameTransformerE@0x 1fd0360 [**603 logs in 3004ms**]
[2023-08-31 14:31:00.183420][warning][116087][Pipeline.cpp:289] Pipeline source frameset queue fulled, drop the oldest frame! [**182 logs in 6027ms**]
[2023-08-31 14:31:01.698979][warning][116123][FrameProcessingBlock.cpp:89] Source frameset queue fulled, drop the oldest frame! N11libobsensor19IMUFrameTransformerE@0x 1fd0360 [**1207 logs in 6000ms, last: 14:31:01.698135**]
^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C[2023-08-31 14:31:12.186463][warning][116087][Pipeline.cpp:289] Pipeline source frameset queue fulled, drop the oldest frame! [**360 logs in 12003ms**]
[2023-08-31 14:31:13.707141][warning][116094][FrameProcessingBlock.cpp:89] Source frameset queue fulled, drop the oldest frame! N11libobsensor19IMUFrameTransformerE@0x 1fd0360 [**2413 logs in 12008ms**]
[2023-08-31 14:31:36.211020][warning][116087][Pipeline.cpp:289] Pipeline source frameset queue fulled, drop the oldest frame! [**721 logs in 24024ms**]
[2023-08-31 14:31:37.707557][warning][116202][FrameProcessingBlock.cpp:89] Source frameset queue fulled, drop the oldest frame! N11libobsensor19IMUFrameTransformerE@0x 1fd0360 [**4825 logs in 24000ms, last: 14:31:37.705348**]
[2023-08-31 14:32:24.222210][warning][116087][Pipeline.cpp:289] Pipeline source frameset queue fulled, drop the oldest frame! [**1442 logs in 48011ms**]
[2023-08-31 14:32:25.713461][warning][116094][FrameProcessingBlock.cpp:89] Source frameset queue fulled, drop the oldest frame! N11libobsensor19IMUFrameTransformerE@0x 1fd0360 [**9649 logs in 48005ms**]

I just see warnings; you can test different RGB format.

Hey,
I am new to this. Can you provide me some insights on how I can transform a 2d point to 3d using pyorbbecsdk for femto mega?