Off-thread decoding of images on Mango, how it impacts your application ?

The part 1 of the SLMPerf Mango blog post series was posted here, where we spoke about ListBox in general. We will take it up one step further and reach out for a deeper (and common) scenario related to ListBox.

We see the following template very often for WP7 apps. 

<ListBox ItemsSource="{Binding Items}" Height="652" Canvas.Top="80">

<ListBox.ItemTemplate>

<DataTemplate>

<StackPanel Orientation="Horizontal" Margin="0,0,0,17">

<Image Height="100" Width="100" Margin="12,0,9,0" Source="{Binding ImgURL}"/>

<StackPanel Width="311">

<TextBlock Text="{Binding LineOne}" TextWrapping="Wrap" Style="{StaticResource PhoneTextExtraLargeStyle}"/>

<TextBlock Text="{Binding LineTwo}" TextWrapping="Wrap" Margin="12,-6,12,0" Style="{StaticResource PhoneTextSubtleStyle}"/>

</StackPanel>

</StackPanel>

</DataTemplate>

</ListBox.ItemTemplate>

</ListBox>

  There was one room for improvement on the platform side that we had identified. For this above scenario running on WP7, the decoding of the image happened on the UI thread. The decoding process time was proportional to the size of the image. More time UI thread spent on such processing, less time it had for interactivity.

 

So we decided to move the decoding off to a separate background thread. The goodness of this change can be seen using the attached sample, LargeListBoxImageDecoding (it’s a huge sample as the images are local in the project).

Here are the pretty diagrams demonstrating the change. 

  This is how it will be for WP7/Mango default:

  

Here is how it will be with BackgroundCreation

 

Opt-In property

 

Moving the image decoding to a background thread resulted in behavioral change for some existing app, so we had to introduce a new property for developers to opt into this goodness for 7.1

https://msdn.microsoft.com/en-us/library/system.windows.media.imaging.bitmapcreateoptions(v=VS.95).aspx

If you see “Do Not use” at the above link, it will be updated soon.

This is a flagwise enum, here is how the combinations should work

 

CreateOptions

Functional Behavior

None

Same has before, None 

DelayCreation

Causes a BitmapSource object to delay initialization until it is necessary. This is useful when dealing with collections of images. This is the default value in WP7(and Mango) of the BitmapImage.CreateOptions property in Silverlight.

IgnoreImageCache

Loads images without using an existing image cache. This option should only be selected when images in a cache need to be refreshed.

BackgroundCreation

Causes object to initialize as soon as declared, uses image cache for same URIs, downloads images in background and decodes them in background.

BackgroundCreation,DelayCreation

Causes object to delay initialization until it is necessary, uses image cache for same URIs, downloads images in background and decodes them in background.

This is default in Beta but we reverted the change, as it caused a lot of developer pain on upgrade to 7.1, we will talk about it below.

BackgroundCreation,DelayCreation,IgnoreImageCache

Causes object to delay initialization until it is necessary., ignores image cache for same URIs, downloads images in background and decodes them in background.

 

The usage of the property for our scenario will be as highlighted below

  

<ListBox ItemsSource="{Binding Items}" Height="652" Canvas.Top="80">

<ListBox.ItemTemplate>

<DataTemplate>

<StackPanel Orientation="Horizontal" Margin="0,0,0,17">

<Image Height="100" Width="100" Margin="12,0,9,0" >

<Image.Source>

<BitmapImage UriSource="{Binding ImgURL}" CreateOptions="BackgroundCreation"/> <!-- flip this to none or delay creation to see the impact on the app-->

</Image.Source>

</Image>

<StackPanel Width="311">

<TextBlock Text="{Binding LineOne}" TextWrapping="Wrap" Style="{StaticResource PhoneTextExtraLargeStyle}"/>

<TextBlock Text="{Binding LineTwo}" TextWrapping="Wrap" Margin="12,-6,12,0" Style="{StaticResource PhoneTextSubtleStyle}"/>

</StackPanel>

</StackPanel>

</DataTemplate>

</ListBox.ItemTemplate>

</ListBox>

  

Impact of BackgroundCreation

Defaults: We recognized offthread-decoding or asynchronous decoding as a good pattern that the developers should follow and wanted to make it a default for the new 7.1 application. Existing 7.0 applications will not be impacted by this change.

So in the mango tools we experimented and kept offthread decoding (BackgroundCreation) as a default. However, we saw several beta developers feel some pain when they upgraded their app from 7.0 to 7.1

Change#1

    

BitmapImage b = new BitmapImage();

b.CreateOptions = BitmapCreateOptions.BackgroundCreation;

b.SetSource(stream);

b.PixelHeight, b.PixelWidth

// now do something with b.PixelWidth/PixelHeight, these values are 0 till the decoding is completed. Use mitigation.

Change#2

BitmapImage b = new BitmapImage();

b.CreateOptions = BitmapCreateOptions.BackgroundCreation;

b.SetSource(stream);

WriteableBitmap wbp = new WriteableBitmap(b);

//this will throw a null ref exception as you cannot create a wbp for the bits that aren’t yet decoded. Use mitigation.

Mitigations:

For both the cases above, you can resolve by waiting for ImageOpened event, and checking if ImageFailed isn’t fired.

Change#3

This will be a little difficult to imagine. It’s a subtle behavior change that you might notice in your app.

Scenario here is, you update the URL of the image (could be part of listbox item), from URL1 to URL2.

With DelayCreation, since the decoding is happening on the UI thread, the image for URL2 won’t show up till the decoding is completed. As soon as the decoding is completed, URL1 image is replaced by ULR2 image. UI thread is bogged down, so less interactivity.

With BackgroundCreation, however, the decoding will happen on a background thread. Which means the UI thread is free to update the UI; while URL2 image is being decoded in the background, URL1 image is replaced by a momentary “blank” and as soon as the URL2 image is decoded, it appears in place of the blank. Resulting in what might looks like a momentary flicker or change in behavior. Depending on your app, this might be desirable or not.

You will notice this in the sample when you change the URL. Play with the property to see the impact yourself.

As another example, you can experiment with the panorama, create a basic panorama application using the pano template. Add the following code to it  

  <controls:Panorama.Background>

                <ImageBrush>

                    <ImageBrush.ImageSource>

                        <BitmapImage CreateOptions=" BackgroundCreation

" UriSource="PanoramaBackground.jpg"></BitmapImage> <!--or png-->

                    </ImageBrush.ImageSource>

                </ImageBrush>

            </controls:Panorama.Background>

 

  

Launch the app on device and see the change

  1.  With BackgroundCreation: the app launches and the panorama background animates in, what appears like a flicker. Or a delayed appearance of the background. This is because the image is still being downloaded in the background while everything else has started to appear.
  2.  With DelayCreation: the app launches and the panorama background animates in as expected, smoothly without any flicker. This is because, we don’t proceed until the background image decoding is completed.

 

When do you use BackgroundCreation ?

The question should be other way actually, when to NOT use BackgroundCreation ? If your application is impacted by BackgroundCreation, as described in the previous section, then you shouldn’t use it. Otherwise, in all other scenarios, having BackgoundCreation for images is nothing but goodness for your application’s UIThread.

Use background creation wherever possible as it, in a very subtle way, improves the interactivity of the application.

The goodness of this change is linearly proportional to the size of the images.

[Edit update – 8/1/2011] - removed the ‘More optimizations’ section as it is no longer relevant for Mango.