What’s new in Expression Encoder 3

Article
07/10/2009

We on the Expression Encoder team are proud to announce that Expression Encoder 3 is complete and will shortly be available! Since releasing V2 SP1 back in November of 2008 we have worked hard to improve core encoding capabilities, continued to enhance new features such as Smooth Streaming, H.264 and editing as well as broaden the applicability of the product by adding new features such as Screen Capture.

We have also focused heavily on our Silverlight output upgrading the MediaPlayer control to Silverlight 3 and the template skins that are based on it. We’ll drill into the details of what is new over the course of this post and look forward to hearing your feedback.

So how do you get Encoder 3? We’ll keep you in suspense a little while longer and direct you to the https://team.silverlight.net blog for the official announcement of Expression Studio 3.

Encoder Enhancements

We’ll start our tour of what’s new with the enhancements to the core encoding engine of the product.

Improved H.264. First up, we’ve improved the H.264 encoder that was part of V2 SP1 by adding support for custom resolutions (V2 SP1 was limited to two portable device-specific resolutions), single pass CBR, VBR Constrained, VBR unconstrained and Quality-based VBR rate control methods, Main Profile support, B-Frame support and advanced settings which include the ability to choose between the two major Entropy modes of Context-adaptive variable-length coding (CAVLC) and Context-adaptive binary arithmetic coding (CAVAC).

VBR Smooth Streaming. Microsoft is investing heavily in Smooth Streaming across its product lines, which in conjunction with Silverlight, enables everyone to watch great quality video on the web at up to full HD 1080p. One of the results, that we are very excited to be shipping, is a brand new VC-1 encoder that has been optimized specifically for the multi-bitrate, multi-resolution nature of IIS Smooth Streaming. Some of the features of the new technology include:

A new rate control that uses an analysis pass to distribute rate across GOPs
Support for variable GOP duration (the original CBR form of encoding requires fixed GOPs)
Dynamic optimization of coded resolution based on scene complexity and bitrate. Additionally, directional energy measurements are used to change resolution both horizontally and vertically
An optimized polyphase filter for resizing resulting in better speed and quality that has been optimized for the SSE4 instruction set.
Significant performance optimizations such as sharing motion information across bitrates and a faster analysis pass
Ability to cancel an encode if QPs start to spike, and restart with lower resolution

Source CODECs. In Encoder 3, we’ve broadened our support for source CODECs. We have improved the performance of our H.264 decoder as well as adding support for AVCHD format. We’ve also added some enhancements to our QuickTime stack to enable compatibility with ProRes which will help with post production workflows. We are also pleased to be shipping Dolby Digital decoding in the box. Finally, we’ve added the ability to import Smooth Streaming files so that you can review an encode outside Silverlight. In this case, you are able to open the .ism file and the largest stream will be rendered. This final feature is also available outside of Expression Encoder because we register the decoder on the system. This means that once the full version of Expression Encoder has been installed, other applications on the system that can make use of DirectShow source filters will be able to open .ismv files.

Audio Enhancements. We heard your feedback and added support for multi-channel audio output:

We have also added the ability to select between multiple audio streams, where sources have them, as well as support for the Voice codec and VBR audio encoding. Last but not least, we dynamically determine what audio encoding capabilities are available from the host OS.

Performance. Performance is something we are never done with. In Encoder 3 we’ve made great progress, particularly on multi-core, multi-processor scenarios. First off, our core encoders have been rev’d to include support for newer Intel SSE instructions as well as support for up to 8 physical cores. Secondly, we’ve improved CPU utilization on multi-core systems by parallelizing preprocessing tasks. To illustrate the improvements, here are some before and after comparison charts based on analysis of Encoder 2 and Encoder 3 performance:

Core 2 Duo Laptop; Vertical axis time in seconds – smaller is better; V2 in blue, V3 in red; results aggregated from 6 different source files

Dual proc 8 Core Xeon; Vertical axis time in seconds – smaller is better; V2 in blue, V3 in red; results aggregated from 6 different source files

We are working on more performance comparisons including i7 Nehalem systems which we’ll publish in due course.

Live Encoding. We spent a bunch of time in this release moving the Live portion of our encoder over to using the new encoding pipeline first introduced in Encoder 2. This results in some immediate benefits such as better performance and de-interlacing support as well as some potential future benefits. We’ve also added support for WMA Pro audio in Live, a new object model for Live (more details on this in the API section) and support for a broader range of capture devices.

Expression Screen Capture

In recent years, screen-casting has become an extremely popular way of sharing information you are looking at on your system with other users; encompassing eLearning, product support, user-generated content and many other domains. Included with Expression Encoder 3 is a new application called Expression Encoder Screen Capture available in the start menu or from within the main Expression Encoder UI.

You simply decide if you want to record webcam, and mic as well as the screen and system sound, pick a screen region with the easy to use Window picker (or by free-form selection):

.. and start capturing. One of the benefits of our new implementation is that we capture to a light weight intermediate CODEC, developed by Microsoft Research, rather than attempting to capture and encode directly to VC-1 or H.264. This leaves more of the CPU available for the application you are capturing. After capture, it is easy to import the sources into Expression Encoder for final encoding and Silverlight templating if required.

UI Enhancements

Improved profile palette. As Expression Encoder has evolved between versions, the old Encoding settings palette had became increasingly unwieldy.. so for Encoder 3 we rebuilt it from the ground up.

The most common encoding settings are now easier to find and set. The advanced encoding settings are hidden by default so you only see the options you need. And the ability to address different output CODECs is now explicitly designed in.

Inside audio and video sections, there are many enhancements that make day to day tasks easier such as changing the aspect ratio or configuring multiple streams for Smooth Streaming.

Preset Palette. Encoder 3 now has a more extensive library of encoding presets including an expanded set that target devices such as Xbox, iPod, Zune and the forthcoming Zune HD, online video services such as Facebook and YouTube and an expanded and improved set that target IIS Smooth Streaming. As well as the expanded library, we have also added a new searchable preset palette where both system and user presets are stored as well as the ability to easily apply presets to all items via right-click.

Dockable palettes. In common with the rest of the Expression 3 suite, Encoder 3 now uses the common palette docking framework which enables you to lay out your interface however you choose by simply dragging palettes and either popping them out into floating windows or grouping them into docked locations. The Hide Palettes function has also been enhanced allowing the hidden palettes to pop open on hover.

Summary Palette. Summary information has been moved into a separate palette which can be displayed via the Window menu.

Cog Menus. In order to streamline palette-specific functions, we’ve introduced cog menus:

Cog menus are present on all palettes. We’ve also got rid of the small white dot menus that were somewhat undiscoverable :-).

Editing Enhancements. In Encoder 2, we had the concept of adding a leader and trailer to a main file. In Encoder 3, we have removed leader and trailer and replaced it with an extended Clips palette which enables multiple items to be stitched together. This enables scenarios such as multiple leaders / trailers, Ad insertion and simple editing.

It is also possible to set the stretch mode per clip and choose to apply crop which was not previously possible with leader and trailer. You can append still frames (and set their duration) as well as XAML animations. Note that all of the enhanced editing capabilities are also available through the object model.

Silverlight 3 MediaPlayers

The Silverlight mediaplayers that have shipped in the box with Expression Encoder since V1 make it really easy to publish videos that target Silverlight. In Encoder 3, we have updated all of the included templates to be based upon Silverlight 3 which means they are offered in the easy to use and deploy XAP format and all share the same common MediaPlayer control at their core. We have also exposed the core MediaPlayer control in the Blend 3 toolbox which makes it easy for designers to embed and customize a full featured MediaPlayer in their projects that supports captions, chapters, smooth streaming etc.

SL3 specific features. We have added a number of Silverlight 3 specific features to all of the new players including

Cached composition: Think of cached composition as hardware acceleration for the final frame presentation step. Enabling cached composition (on by default) is as simple as a checkbox and results in much lower CPU utilization which helps general playback performance particularly on lower end systems such as NetBooks. It also means that setting Stretch Mode Uniform incurs no performance penalty.
Offline support : Offline also known as Out Of Browser (OOB) is a new feature of Silverlight 3 that enables users to install Silverlight apps locally on their Mac or Windows machine. In Encoder 3, if you choose to enable offline support when you publish a Silverlight video, the user will have the option of downloading and installing not only the media player code but also the video and audio files.

This feature also works in conjunction with IIS Smooth Streaming so you get the benefit of adaptive streaming when online and a downloaded, high quality version to watch when away from a network.

To show this in action, I created the following screencast. Note that you can actually try the offline feature on the screencast it by clicking on the cog menu and selecting Offline :-)
We’ve made some changes to the Adaptive Streaming support that is baked into our MediaPlayer control. Firstly, we’ve improved the bandwidth heuristics in a number of ways such as using the new CPU load metrics available from the Silverlight 3 runtime and constraining the playback resolution based on the size of the video window (to prevent bandwidth wasted by downloading e.g. 720p when the window size is only e.g. 320x240).

We have also moved the heuristics into a separate XAP so that the adaptive streaming module can be loaded only when necessary thus helping to reduce the core download size. Finally, we have changed the way we distribute the Media Steam Source (MSS) that implements the client portion of IIS Smooth Streaming. We have moved to a model where we ship object code for this component, not source. Source code is shipped for all other MediaPlayer functionality and is usable, redistributable and modifiable under the MS-PL.

New Player Skins. We have worked with ace design firm Archetype to create some great new player skins featuring smoothly animating controls and new Silverlight 3 features such as perspective transform.

Other Silverlight Improvements:

XAP Size reduction: we’ve taken steps to reduce the size of the MediaPlayer control.. the core player is now 109k down from 162k in Encoder 2 SP1.
Popout: We’ve added a Popout feature to the templates that allows a video to be popped out of a webpage as a webpage dialog.. you can then drag to another monitor and go fullscreen to keep watching while you work.
Scriptability: The core mediaplayer control is now fully scriptable from Javascript.
Improved Install Experience: Expression Encoder now automatically generates a Content Gate install experience for users that don’t have Silverlight Installed:
Improved accessibility on Silverlight Media Player: We’ve added tooltips to buttons on the MediaPlayer and done some additional testing with screen readers.
Edit in Expression Web: We offer the ability to edit the output of a job in Expression Web.
Preview in Browser: You can now open a Smooth Streaming job previously encoded and saved and start up the Smooth Streaming preview server and launch the default browser without having to re-encode the job.
MediaPlayer control available in Blend Asset Palette: When Expression Encoder is installed on the same system as Blend, the Expression MediaPlayer control becomes available in the Blend asset palette. This enables designers and developers to easily use and customize our base player and take advantage of all the built-in functionality such as chapter points, closed captions, play lists smooth streaming, Offline support etc.
Transcoding in Expression Web: Expression Web ships with Expression Encoder and utilizes the API to offer simplified transcoding functionality integrated right in.

API Enhancements

Expression Encoder has had a .NET API since Version 2. In Version 3, we are making some changes:

Object Model for Live Encoding. Live encoding now has an object model so you can create applications that capture and stream from live capture sources as shown here:

    1: using (LiveJob job = new LiveJob())
    2:            {
    3:                // Create a new file source from the file name we were passed in
    4:                LiveFileSource fileSource = job.AddFileSource(fileToEncode);
    5:  
    6:                fileSource.PlaybackMode = FileSourcePlaybackMode.Loop;
    7:  
    8:                //Alternatively, enumerate a capture device (not shown here)
    9:  
   10:                job.ActivateSource(fileSource);
   11:  
   12:                WindowsMediaBroadcastOutputFormat outputFormat = new WindowsMediaBroadcastOutputFormat();
   13:  
   14:                // Let's broadcast on the local machine on port 8080
   15:                outputFormat.BroadcastPort = 8080;
   16:  
   17:                // Set the output format on the job
   18:                job.OutputFormat = outputFormat;
   19:  
   20:                // Start encoding
   21:                Console.Out.Write("Press 'x' to stop encoding...");
   22:                job.StartEncoding();
   23:  
   24:                while (Console.ReadKey(true).Key != ConsoleKey.X)
   25:                {
   26:                    // We are waiting for the 'x' key
   27:                }
   28:  
   29:                // Stop our encoding
   30:                Console.Out.WriteLine("Encoding stopped.");
   31:                job.StopEncoding();
   32:            }

Refactored Encoding Profile OM. In addition to refactoring the Encoding profile palette, we have also refactored the underlying object model for configuring profiles making it simpler to use. Instead of a monolithic Profile class, settings have been broken into a class hierarchy allowing the app to more easily and logically support multiple output CODECs such as H.264 and VC-1.

SDK in the Box. The SDK is now shipped along with the application. It has also been updated to include more samples, including a Powershell cmdlet sample.

Redistributable version of Expression Encoder. We will also be offering a no fee redistributable version of Expression Encoder for developers who wish to ship applications using the SDK. Stay tuned for more information on this that we’ll share here on the blog in due course.

General

Windows 7 Superbar Progress: View at a glance how far through the encode has progressed:

First pass:

Second pass:

Removal of PS 2.0 requirement: We had feedback from users of older graphics cards that did not support Pixel Shader 2.0 that they were unhappy that they could not preview video. So in Encoder 3, we still take advantage of PS 2.0 if present, but do not require it for preview.

Support for LargeAddressAware flag: Some of the new features, in particular the new VBR Smooth Streaming Encoder, can require a lot of memory in certain scenarios. In Encoder 3, we set the /LARGEADDRESSAWARE bit in the application’s PE header which enables 3GB of addressable memory on x86 systems (requires BCDEDIT /Set IncreaseUserVa 3072) and 4GB on x64 systems. For more information on this and how to configure an x86 system to take advantage of this feature, see https://msdn.microsoft.com/en-us/library/aa906217.aspx and https://blogs.technet.com/brad_rutkowski/archive/2006/10/03/Hey-where-did-_2F00_3GB-go-in-Longhorn-and-Vista_3F00_.aspx

Removal of command-line encoder: One of the tough trade-offs we had to make for this release was what to do with the command-line. In Encoder 2 SP1 the command-line encoder was functionally behind what was possible in the UI and the Object Model. We also observed a lot of community support for specialized command-line alternatives such as WMCmd.vbs by Alex Zambelli. So, for Encoder 3 we took the difficult decision of removing the command-line encoder from the product. Instead we directed our energies into making the .NET Object Model full fidelity in order to give application developers the maximum API surface area possible, including support for Live encoding. To mitigate the decision, we are shipping working sample code in the box in the form of a Powershell cmdlet wrapper around the object model which provides basic Powershell-based command-line encoding supporting batch operations via the Powershell object pipeline:

Additionally, we are keen to work with the community to get an equivalent to WMCmd.vbs built and published on Codeplex to satisfy users who really want to automate encoder using cmd.exe rather than from Powershell or C#. We hope you agree that this was the right trade-off to make. If you have any opinions, positive or negative, please do send us your feedback.

Summary

We look forward to making the release available for download in the coming weeks and talking more about its capabilities. Stay tuned and, as ever, your feedback is welcome.

The Expression Encoder Team.