EmguCV and AstraSDK for object dectection

Developer Resources Getting Started
1

Hi everyone,

Newbie here.

I’m an engineer from Germany and currently building a concept for object detection and measurement. The objects lengths, widths and heights should be then sent to the robot. So using BlobTracker and extract information from the depth frame would be my main goal.

Background:

  • I purchased the Astra Mini S as our application requires the camera to be enclosed in a case and the detection range should be short. (Sadly I just realized that Astra Pro might be the better option as it supports UVC.)
  • I do not have a strong programming knowledge. That’s why I prefer to use EmguCV as it saves me a lot of work by offering a good wrapper for programming in C# without the compiling and settings.

My thoughts:

  • using AstraSDK to get colorFrames and DepthFrames
  • convert the colorFrames into Mat (which can be accepted by EmguCV)
  • get the height of the object from DepthFrame after BlobTracker has done its job
  • pack these information into a package and send it via Socket

Questions:

  • Generally, what would you recommend? How should this concept be built?
  • I cannot find any description of ColorFrame, if I use AstraSDK, how can I get the frame and convert it into a Mat? Can I convert ColorFrame into a Bitmap first? How?
  • Should I give up the idea of using EmguCV and use OpenCV directly? I somehow saw an example with OpenCV library and they somehow got the Frame directly passed to OpenCV.

There are a lot of questions…As a beginner I need your help! Thanks in advance!

2

Hi @CKL ,

You can try the following sample of using Astra SDK with c# wraper: GitHub - bibigone/AstraDotNetDemo: Simple .Net solution to demonstrate working with Orbbec Astra (.
I have used it with EmguCV for video recording.
Yes you can convert ColorFrame to the Bitmap and later to the Mat.

1 Like
3

Hallo @shmlex,

Thanks for your help! I found your example before - sadly I could not get the sample built. I tried it and it throws System.IO.FileLoadException and said that a dependency is not found…
After searching for solution (changing the app.config) I finally get it to build and run just now!
You mentioned that ColorFrame can be converted, can you show me the direction? I know in OpenCV you can convert a Bitmap into Mat but as I said I did not find a way to convert ColorFrame to Bitmap…

Thanks again! You helped me a lot with the great example!

4

Hey @CLK,
you may contact me directly, if you like to discuss your problem in person and in german.
Andreas
tel (in leet): OI7Z 6Z 3537 8nine

5

I switched to use MediaFoundation for video recording, so the EmguCV code is not available now.

As I remember it should be like the following:

add this method to BufferOfT

public WriteableBitmap GetWritableBitMap()
{

  	return writeableBitmap;

}

in the SensorViewModel update method HandleColorFrame

private void HandleColorFrame(Astra.ReaderFrame frame)
{

  	var colorFrame = frame.GetFrame<Astra.ColorFrame>();
  	if (_colorBuffer.Update(colorFrame))
  	{
  		BitmapSource image = _colorBuffer.GetWritableBitMap();
  		Bitmap bmp = getBitmap(image);

  		Mat mat = getMat(bmp); //               use that mat for EmguCV!!!
  	}
  }

add convert methods:

private Mat getMat(Bitmap bitmap)
{

  	Image<Bgr, Byte> imageCV = new Image<Bgr, byte>(bitmap);
  	return imageCV.Mat;

}

private Bitmap getBitmap(BitmapSource source)
{

  	Bitmap bmp = new Bitmap(source.PixelWidth, source.PixelHeight, PixelFormat.Format24bppRgb);
  	BitmapData data = bmp.LockBits(new Rectangle(System.Drawing.Point.Empty, bmp.Size), ImageLockMode.WriteOnly, PixelFormat.Format24bppRgb);
  	source.CopyPixels(Int32Rect.Empty, data.Scan0, data.Height * data.Stride, data.Stride);
  	bmp.UnlockBits(data);
  	return bmp;

}

1 Like
7

HI,
How do I calculate the distance for each pixel from the camera?
Thanks

8

@CKL Hi , please can u tell me the final steps u followed
with ur project for object detection ?