I’m currently working with the Astra SDK 0.4 for Mac OS X (with Astra sensor).
I found that the frame rate sometimes dips below 27 fps with the SimpleDepthViewer example.
After profiling I found that a call XnDepthProcessor::SoftFilter in liborbbec.dylil takes a big chunk of processing power (24% of runtime; another big chunk is visualisation but that is not relevant to this topic).
Since I work on a software project in which every CPU cycle available is needed for registration/SLAM and other processes, I have the following questions:
Is there a way to enable/disable the XnDepthProcessor::SoftFilter process or put it on GPU?
Is there a documentation on the parameters and their values ?( It seems there is astra_stream_set_parameter but there seems to be no doc/list of parameters and their possible values)
Does the sensor work in push or pull (or both) mode(s) ? …
… and consequently does the driver have its own thread for polling the sensor in callback/stream mode?
If I poll a frame, is this frame from a buffer (on sensor) or does the astra_frame_get_xxxxx function wait on the next available frame?
Does the conversion from depth values to PointFrame happen on sensor or client side?
Thanks in advance to anyone who may help answer any of theses questions.
Sorry for the delay in responding. We were at CES 2016 when you posted and are catching up on threads now.
To answer your questions:
We will look at further optimizing the SoftFilter method but it is necessary to produce correct results. Future revisions of the Astra camera may not require SoftFilter.
astra_stream_set_parameter is a low level call, part of the parameter system including get_parameter and invoke, used to opaquely pass data from a client to a plugin. It isn’t meant to be used directly. Wrapper methods such as enabled_mirroring() or set_mode() call set_parameter() behind the scenes. The specific data sent through set_parameter() is part of a contract between the wrapper library and the plugin. We don’t have great documentation on everything there yet, but if you’re trying to do something specific and the sample code doesn’t show how to do it, just post on the forum and we’ll help.
I’m not sure which aspect you’re asking about. The Astra SDK provides both an event and a poll API. The sensor hardware uses isochronous USB transfers to continuously stream the depth and color (or IR) data to the host system as soon as it is available.
Yes the driver uses its own thread for processing the USB data. There is no polling of the hardware though.
When you use the Astra SDK polling API, it provides you with the latest frame available on the host system. For example, if you call reader.get_latest_frame(10) (10 is 10ms timeout) then within approximately 10ms, if a new frame is available, it will return it. If not, it will return a frame where is_valid() is false. You can provide a timeout of 0, which will return immediately, or a timeout of ASTRA_TIMEOUT_FOREVER (-1) which will wait forever for the next frame. None of this directly affects the hardware, though. It is simply waiting for the driver thread to report the new frame has been received from the sensor hardware. The driver thread does that continuously (as long as the streams are enabled).
PointFrame conversion happens on the host system, within the orbbec_xs plugin. This allows that calculation to happen only once, as long as there are 1 or more clients or plugins (who also are treated as clients) that subscribe to the PointStream. If no clients are subscribed to PointStream, the calculation does not occur.
Please let me know if you need any more clarification on any of these questions, or other questions.
