Astra SDK Samples on Ubuntu 16.04 - Blank SFML screen

[luketaverne]

Good afternoon everyone!

We are having some trouble getting the SDK samples working properly. We've compiled the samples (from the glone git repository, develop branch: https://github.com/orbbec/astra/tree/develop).

It seems to be the same problem as this thread: https://3dclub.orbbec3d.com/t/astra-sdk-black-screen-for-all-viewer-programs/356/4 but the solutions there don't seem to work (will cover what I've tried below).

I'll try to outline everything I've done in detail, so hopefully this can be a reference for other people debugging the compilation of the samples.

Compilation process for the samples:

$ mkdir build & cd build
$ cmake ..
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test COMPILER_SUPPORTS_CXX11
-- Performing Test COMPILER_SUPPORTS_CXX11 - Success
-- Performing Test COMPILER_SUPPORTS_CXX14
-- Performing Test COMPILER_SUPPORTS_CXX14 - Success
flags  -Wall -fPIC -std=c++14
-- Found clisp: /usr/bin/clisp  
Setting rpath to $ORIGIN/../lib
-- Found Astra includes in /home/luke/master-thesis/thirdparty/astra/include
-- Found Astra lib dir: 
-- Found Astra libraries: astra_core;astra_core_api;astra
-- Found Astra TOML: 
-- Found Astra plugins: 
-- Found Astra plugin TOML: 
Setting rpath to $ORIGIN/../lib
-- Found SFML .. in /usr/include
-- Found SFML .. in /usr/include
-- Found SFML .. in /usr/include
-- Found SFML .. in /usr/include
-- Found SFML .. in /usr/include
-- Found SFML .. in /usr/include
Setting rpath to $ORIGIN/../lib
CMAKE_INSTALL_PREFIX : /home/luke/master-thesis/thirdparty/astra/build/sdk
CMAKE_SYSTEM_NAME: Linux
-- Configuring done
-- Generating done
-- Build files have been written to: /home/luke/master-thesis/thirdparty/astra/build

Which seems to be fine as far as I can tell. Then making:

$ make -j2

(rest of the output on pastebin: https://pastebin.com/4WUFfCwX

Nothing fails, there are some warnings about unsigned int comparisons.

I then changed ./build/lib/astra.toml to be the following, as suggested in another thread (this is to get logging output from the samples, for anyone reading this with a similar problem):

[logging]
# trace, debug, info, warn, error, fatal
level = "trace"
console_output = true
file_output = true
[plugins]
#path = "Plugins"

Then to execute the samples:

$ cd bin
$ ./SimpleDepthViewer-SFML
$ 2017-10-29 14:49:02,303 WARN  [context] Hold on to yer butts
2017-10-29 14:49:02,303 INFO  [context] configuration path: /home/luke/master-thesis/thirdparty/astra/build/bin/../lib/astra.toml
2017-10-29 14:49:02,303 INFO  [context] log file path: astra.log
2017-10-29 14:49:02,303 INFO  [context] plugin path: /home/luke/master-thesis/thirdparty/astra/build/bin/../lib/Plugins/
2017-10-29 14:49:02,303 TRACE [plugin_manager] [void astra::plugin_manager::load_plugins(std::__cxx11::string)] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_plugin_manager.cpp:35] load plugins
2017-10-29 14:49:02,303 TRACE [plugin_manager] [void astra::plugin_manager::try_load_plugin(const string&)] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_plugin_manager.cpp:96] try_load_plugin /home/luke/master-thesis/thirdparty/astra/build/bin/../lib/Plugins/liborbbec_hand.so
2017-10-29 14:49:02,306 TRACE [plugin_manager] [void astra::plugin_manager::try_load_plugin(const string&)] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_plugin_manager.cpp:111] try_load_plugin valid plugin
2017-10-29 14:49:02,306 TRACE [plugin_manager] [void astra::plugin_manager::try_load_plugin(const string&)] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_plugin_manager.cpp:113] try_load_plugin initialized plugin
2017-10-29 14:49:02,306 TRACE [plugin_manager] [void astra::plugin_manager::try_load_plugin(const string&)] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_plugin_manager.cpp:96] try_load_plugin /home/luke/master-thesis/thirdparty/astra/build/bin/../lib/Plugins/libopenni_sensor.so
2017-10-29 14:49:02,307 TRACE [plugin_manager] [void astra::plugin_manager::try_load_plugin(const string&)] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_plugin_manager.cpp:111] try_load_plugin valid plugin
2017-10-29 14:49:02,307 INFO  [orbbec.ni.oni_adapter_plugin] Initializing OpenNI v2.3.0.15
2017-10-29 14:49:02,310 INFO  [orbbec.ni.oni_adapter_plugin] Initialized OpenNI v2.3.0.15
2017-10-29 14:49:02,310 TRACE [plugin_manager] [void astra::plugin_manager::try_load_plugin(const string&)] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_plugin_manager.cpp:113] try_load_plugin initialized plugin
2017-10-29 14:49:02,310 TRACE [plugin_manager] [void astra::plugin_manager::try_load_plugin(const string&)] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_plugin_manager.cpp:96] try_load_plugin /home/luke/master-thesis/thirdparty/astra/build/bin/../lib/Plugins/liborbbec_xs.so
2017-10-29 14:49:02,311 TRACE [plugin_manager] [void astra::plugin_manager::try_load_plugin(const string&)] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_plugin_manager.cpp:111] try_load_plugin valid plugin
2017-10-29 14:49:02,312 INFO  [astra.xs.plugin] Initializing xs plugin
2017-10-29 14:49:02,312 TRACE [plugin_manager] [void astra::plugin_manager::try_load_plugin(const string&)] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_plugin_manager.cpp:113] try_load_plugin initialized plugin
2017-10-29 14:49:02,637 INFO  [context] client opening streamset: device/default
2017-10-29 14:49:02,638 INFO  [streamset_catalog] default uri provided.
2017-10-29 14:49:02,638 TRACE [astra.streamset] [astra::streamset_connection* astra::streamset::add_new_connection()] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_streamset.cpp:118] new connection to device/sensor0
2017-10-29 14:49:02,639 INFO  [astra.streamset] connecting to (7,0) on device/sensor0
2017-10-29 14:49:02,639 TRACE [astra.streamset] [astra::stream_connection* astra::streamset::create_stream_connection(const astra_stream_desc_t&)] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_streamset.cpp:40] registering orphan stream of type (7,0) on device/sensor0
2017-10-29 14:49:02,639 DEBUG [astra.streamset] registering stream (7, 0) on device/sensor0
2017-10-29 14:49:02,639 DEBUG [astra.stream_connection] 0x2bd7670 starting (7, 0)
2017-10-29 14:49:02,639 INFO  [astra.streamset] connecting to (1,0) on device/sensor0
2017-10-29 14:49:02,640 TRACE [astra.streamset] [astra::stream_connection* astra::streamset::create_stream_connection(const astra_stream_desc_t&)] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_streamset.cpp:40] registering orphan stream of type (1,0) on device/sensor0
2017-10-29 14:49:02,640 DEBUG [astra.streamset] registering stream (1, 0) on device/sensor0
2017-10-29 14:49:02,640 DEBUG [astra.stream_connection] 0x2bd8890 starting (1, 0)
Failed to load font "Inconsolata.otf" (failed to create the font face)

Then there is a blank screen for the sample. I close the window and see

2017-10-29 14:49:08,671 INFO  [orbbec.ni.oni_adapter_plugin] shutting down openni
2017-10-29 14:49:08,821 INFO  [astra.xs.plugin] Terminated xs plugin
2017-10-29 14:49:08,821 TRACE [streamset_connection] [virtual astra::streamset_connection::~streamset_connection()] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_streamset_connection.hpp:40] destroying streamset_connection: 0x2bd39c0
2017-10-29 14:49:08,822 TRACE [astra.stream_reader] [virtual astra::stream_reader::~stream_reader()] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_stream_reader.cpp:36] destroying reader: 0x2bd76f0
2017-10-29 14:49:08,822 TRACE [astra.streamset] [bool astra::streamset::destroy_stream_connection(astra::stream_connection*)] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_streamset.cpp:59] destroying 0x2bd8890 on device/sensor0
2017-10-29 14:49:08,822 TRACE [astra.streamset] [bool astra::streamset::destroy_stream_connection(astra::stream_connection*)] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_streamset.cpp:67] removing unused/unavailable stream 0x2bdac80 on device/sensor0
2017-10-29 14:49:08,822 TRACE [astra.streamset] [bool astra::streamset::destroy_stream_connection(astra::stream_connection*)] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_streamset.cpp:59] destroying 0x2bd7670 on device/sensor0
2017-10-29 14:49:08,822 TRACE [astra.streamset] [bool astra::streamset::destroy_stream_connection(astra::stream_connection*)] [/home/luke/master-thesis/thirdparty/astra/src/astra_core/astra_streamset.cpp:67] removing unused/unavailable stream 0x2bd7560 on device/sensor0
2017-10-29 14:49:08,822 INFO  [context] Astra terminated.
2017-10-29 14:49:08,822 WARN  [context] attempt to destroy a non-existent reader: 0x2bd76f0

So as the other thread suggested, I checked the udev rules. I have an Astra Mini S (idVendor=2bc5, idProduct=0407), so I created /etc/udev/rules/orbbec-usb.rules as the following (where the last row should be for my device):

# Make orbbec devices mount with writing permissions (default is read only for unknown devices)
SUBSYSTEM=="usb", ATTR{idProduct}=="0401", ATTR{idVendor}=="2bc5", MODE:="0666", OWNER:="root", GROUP:="video"
SUBSYSTEM=="usb", ATTR{idProduct}=="0402", ATTR{idVendor}=="2bc5", MODE:="0666", OWNER:="root", GROUP:="video"
SUBSYSTEM=="usb", ATTR{idProduct}=="0403", ATTR{idVendor}=="2bc5", MODE:="0666", OWNER:="root", GROUP:="video"
SUBSYSTEM=="usb", ATTR{idProduct}=="0404", ATTR{idVendor}=="2bc5", MODE:="0666", OWNER:="root", GROUP:="video"
SUBSYSTEM=="usb", ATTR{idProduct}=="0405", ATTR{idVendor}=="2bc5", MODE:="0666", OWNER:="root", GROUP:="video"
SUBSYSTEM=="usb", ATTR{idProduct}=="0407", ATTR{idVendor}=="2bc5", MODE:="0666", OWNER:="root", GROUP:="video"

Checking the output of lsusb gives:

Bus 002 Device 018: ID 2bc5:0407

And ls -l /dev/bus/usb/002/0018 gives:

crw-rw-rw- 1 root video 189, 145 Oct 29 15:03 /dev/bus/usb/002/018

which seems to be in order. I've also rebooted and also used sudo udevadm control --reload-rules in between reboots when testing different rule configurations.

Checking dmesg results in this, although I'm not sure what the desired output is:

[168168.558349] usb 2-1.7.7: new high-speed USB device number 18 using ehci-pci
[168170.307211] usb 2-1.7.7: New USB device found, idVendor=2bc5, idProduct=0407
[168170.307215] usb 2-1.7.7: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[168170.307217] usb 2-1.7.7: Product: ASTRA
[168170.307220] usb 2-1.7.7: Manufacturer: ORBBEC

I've noticed for other devices I see something like [167717.387537] usbcore: registered new interface driver cdc_acm, but I'm not sure if anything like that should come up.

I can confirm that the camera is working properly, as I'm able to access and display the depth stream using OpenNI2 and primesense in python (there is another thread detailing this). The OpenNI2 samples also work properly, when I compile the zip file for download from http://orbbec.com/develop.

Does anyone have any further suggestions or anything I can try? I'm hoping to be able to use the Astra SDK and not rely on purely the OpenNI2 libraries. Thanks in advance!

[luketaverne]

Quick update: I've compiled the OpenNI2 git repo https://github.com/orbbec/OpenNI2 (develop branch), and the samples report:

$ ./SimpleViewer
After initialization:

SimpleViewer: Device open failed:
	DeviceOpen using default: no devices found

The primesense and python method still works for retrieving video though.

Longer lsusb -v:

Bus 002 Device 033: ID 2bc5:0407  
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  idVendor           0x2bc5 
  idProduct          0x0407 
  bcdDevice            0.01
  iManufacturer           1 ORBBEC
  iProduct                2 ASTRA
  iSerial                 0 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           32
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0x80
      (Bus Powered)
    MaxPower              500mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass      0 
      bInterfaceProtocol      0 
      iInterface              3 ORBBEC Depth Sensor
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               1
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x82  EP 2 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               1
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  bNumConfigurations      0
Device Status:     0x0000
  (Bus Powered)

I also compiled hello_world from the docs, and was able to compile and run the samples that call astra::initialize() and astra::terminate() with nothing between them, but when I add the frame reading code, then the program hangs. It seems to be likely that the camera is not found...

EDIT: I downloaded the 2-Linux.zip from the http://orbbec.com/develop site, and after extracting the OpenNI-Linux-x64-2.3.zip, I was able to run SimpleViewer with no problems. It still hasn't fixed the other problems, but at least some other program from Orbbec is able to access the camera.

EDIT2: Will also add: my user is definitely in the video group.

$ sudo groups luke
luke : luke adm cdrom sudo dip video plugdev lpadmin sambashare

[luketaverne]

Good news everyone! After contacting Orbbec Tech Support I received the following information:

1) the Astra SDK is not officially supported on Linux yet, and they suggest using the OpenNI2 SDK provided at http://orbbec3d.com/develop (this is a special version of OpenNI2 that contains the drivers for orbbec's cameras. I'm not sure if there are any other differences).

2) The Astra Mini S (and possibly also the Astra Mini, I didn't ask about Mini specifically, only the Mini S) is not supported by the Astra SDK yet. A potential workaround is to copy the drivers from the Orbbec provided OpenNI2 SDK (again, from http://orbbec3d.com/develop), to the proper folder of the SDK.

I saw multiple posts on the forum that say "I just had to put the drivers in the right folder and it worked!" and never explain exactly what they mean. Here's what I did to get the Astra SDK working for the Mini S:

1) Make sure that your /etc/udev/rules.d/55-orbbec-usb.rules file has a line for your USB device. See my first post in this thread (basically just find the camera with lsusb and then make sure there is a line in the udev rules that has the correct vendor and product id for your camera. Restart your computer after changing this.

2) The updated drivers for the Astra Mini are in the OpenNI2 SDK. Download the Orbbec provided OpenNI2 SDK from the site above, and the drivers are located in ./Redist/OpenNI2/Drivers. According to Orbbec support, the two files you need are liborbbec.so and orbbec.ini (I copied the entire contents of the Drivers folder and it doesn't seem to have harmed anything). Copy these files to the Astra SDK (I cloned the Github repository and switched to the develop branch) by placing them in ./vendor/openni2/ubuntu-x64/OpenNI2/Drivers. You can then compile as normal from the Astra SDK root directory, and run one of the samples ie:

$ cd /path/to/your/sdk
$ mkdir build 
$ cd build 
$ cmake ..
$ make
$ ./bin/SimpleDepthViewer-SFML

And now the video appears immediately.

Thanks you to @Jackson for the help and suggestions!

[chonhc]

Thanks for sharing about the Astra SDK not officially supported yet.