How to use SoftwareBitmap to display video from EZ-B controller

SoftwareBitmap is a new class which is supported by Universal Windows platform. It’s part of Windows.Graphics.Imaging namespace and allows to represent an uncompressed bitmap. Of course, it has nothing to do with this class itself but UWP supports many different classes which allows to get SoftwareBitmap from camera, from video stream etc. Once you get SoftwareBitmap, it’s easy to apply different effects there and show the result or convert it to popular image file formats.

I missed this class before but this week I got the camera for my EZ-B controller. So, I attached it to my rover and decided to show video from there on my phone.

In the first step you need to add reference to EZ-B SDK for Universal Windows Platform which you can find here and initialize EZB and EZBv4Video objects

 EZB ezb;
EZBv4Video video;
DispatcherTimer timer;
 
protected async override void OnNavigatedTo(NavigationEventArgs e)
{
 ezb = new EZB();
 await ezb.Connect("192.168.1.1");
 
 video = new EZBv4Video();
 video.OnImageDataReady += Video_OnImageDataReady; 
 
 await video.Start(ezb, ezb.ConnectedEndPointAddress, 24);
 
 video.CameraSetting(EZBv4Video.CameraSettingsEnum.Res640x480);
}

There is nothing special but in the next step you need to implement OnImageDataReady event handler. The camera sends data frame by frame using Jpeg file format. So, the basic task there is to convert Jpeg to uncompressed bitmap, apply any effects and show it using Image control or anything else. I decided to implement the method in the following way:

 private async void Video_OnImageDataReady(byte[] imageData)
{
 await Dispatcher.RunAsync(
 Windows.UI.Core.CoreDispatcherPriority.Normal,
 async () =>
 {
 MemoryStream ms = new MemoryStream(imageData);
 
 var decoder = await BitmapDecoder.CreateAsync(
 BitmapDecoder.JpegDecoderId, ms.AsRandomAccessStream());
 
 var sbit = await decoder.GetSoftwareBitmapAsync();
 
 WriteableBitmap bm = 
 new WriteableBitmap((int)decoder.PixelWidth, 
 (int)decoder.PixelHeight);
 
 sbit.CopyToBuffer(bm.PixelBuffer);
 image.Source = bm;
 });
}

Since OnImageDataHandler handler runs in non-UI thread, you need to use Dispatcher to run your code in UI thread. There you can use BitmapDecoder class which is part of the same namespace. This class allows to convert Jpeg data to SoftwareBitmap and returns the reference to the last one. In the next step you can easy get access to uncompressed buffer and apply any effects there. I just decided to copy SoftwareBitmap to WritableBitmap in order to get object which implement ImageSource. That’s all and finally I received video from my robot. In case of EZ-B, quality is pretty good and its performance is enough to emulate video stream frame by frame.