I’m currently working on this Silverlight project, that will be a growing video archive that you might see go online in the close future. Unfortunately I can’t say more at the moment. But that is not what this post is about. The post is about playing Smooth Streaming videos in Silverlight.
I’m not that familiar with media streaming and video formats and so on, but when we started talking about playing video, I insisted on it being streaming. Http progressive download is just not that attractive and great… It will use up unnecessary bandwidth as well as cause certain “issues” at the client. Such as limiting the users ability to “scrub” though the video…
We quickly decided not to build our own player. Why do that, when people have already done it for you… So I pulled out Expression Encoder and got one of its premade players. I opened up the xap, pulled out the assemblies I needed and threw the rest away. That makes for a great base to inherit from. All I needed to do was basically redo the template and I was done.
However, besides the normal issues with the templating, we ran into some issues with the playback. We had the player in the application and initially did no streaming. Instead we fed it files from disk while developing the application. And then the day came when the customer had found a streaming supplier. It happened to be iStreamPlanet. And as they got involved it was time to get started with streaming. I thought we were going to use Windows Media Services and stream over the mms protocol. I had tried this before. But that wasn’t the case. iStreamPlanet streams using a new technique called Smooth Streaming and what they gave us were a couple of test urls. They had a file “.ism” file extension. Hmm…interesting…
I started looking around to see what this was and how it worked. And of course what implications it would have for us… Well… Luckily, the implications for us were not that big. The media player that Expression Blend uses, supports Smooth Streaming natively. However, we still had streaming issues, so I decided to look under the hood and see what was going on…what is Smooth Streaming? How does it work? Why is it cool? Why haven’t I built it myself?
Well, let’s start with what Smooth Streaming is. It is a technology that supports streaming from an IIS 7 server. That’s nothing new, but the way it works is kind of cool. At least this is the way that I have figured it works…
When you get a url to a .ism file, don’t try to browse to it straight up. That won’t work. What you need to do is get the manifest. This is available by appending “/manifest” at the end of the url. This manifest is basically an Xml file containing the different feeds available. There are several different feeds involved. Each feed has its own bitrate. So when the player is playing, it can actually switch between feeds with different bitrates while playing. So the quality of the streamed video actually fluctuates while playing. Actually, it is not the player that does the switching…but I will get to that later…
The player then starts downloading the bitrate it wants (once again…not the player…more to come). If the bitrate is too high for the available connection, the selected stream will change. So it will go to one with less or higher bitrate depending on bandwidth available.
How can it change in the middle of streaming? Well, it actually streams in a cool way. It downloads a little piece at the time. It isn’t an open connection constantly streaming. Instead the stream is more like a file on disk, that the player can request chunks from. So it can download a couple of kbs, called a chunk, and then play that while it requests the next chunk. So it is more like chunk reading a large file, than streaming. This technique makes it possible to figure out how fast we are actually downloading, as well as switch between streams by just downloading the next chunk from another stream…
And why is this cool? Well…it makes it possible to stream high quality video to those who have enough bandwidth, but still fallback to lower quality if needed. We are actually streaming 720p video at good quality. And that is streaming to New Zealand…which is not really the best connected place in the world.
This technology has been used by a lot of players already, including the Olympic Games and companies like Netflix. The Olympic Games used smooth streaming as it enabled them to quickly edit video on the fly using a Silverlight based application, and then deliver more content to the viewers faster and with the best available quality. And companies like Netflix use smooth streaming as they too need the ability to offer the best quality possible for each individual viewer looking to for the best quality when they watch TV shows online.
So…technically, what do we need to do? Do we need some new fancy control for Silverlight to play this? No…the MediaElement can actually play this. But not out of the box. The way that you get this going is by using a new type of source. This source can be extracted from the Encoder player, which is what I did. It is called AdaptiveStreamingSource and in the AdaptiveStreaming.dll assembly.
AdaptiveStreaming.dll is available inside the xap that Expression Encoder produces when asked to add a player to the encoded output. Just take that xap. Rename the .xap to .zip. Open and pull out the assemblies needed…
So what you need to do, is to reference the AdaptiveStreamingSource assembly. Then in code you create a new AdaptiveStreamingSource passing in the MediaElement and the Uri to the .ism file. However, you need to add the “/manifest” at the end of the Uri. Then you use this source to set the source of the MediaElement. After that, the MediaElement will play just as normal. The AdaptiveStreamingSource object also gives you access to a few new events that might be interesting. You can get notified of each chunk download, bitrate change, buffer start and so on…
public void Play(string url)
{
AdaptiveStreamingSource src = new AdaptiveStreamingSource(myMediaElement,new Uri(url + "/manifest"));
myMediaElement.SetSource(src);
myMediaElement.Play();
}
If you want to extend the Encoder player, I recommend you inherit from the ExpressionMediaPlayer.MediaPlayer class in the MediaPlayer.dll assembly. The MediaPlayer.dll assembly is available in the Encoder xap just like the AdaptiveStreaming.dll. If you chose to inherit from this class, you do not handle the setting of the MediaElement’s Source on your own. Instead you have to create a class implementing the IMediaStreamSourceFactory interface. This class is then used to set the base class’ MediaStreamSourceFactory property. In this class you are responsible for instantiating the AdaptiveStreamSource to use for the playing. The factory can look like this
private class AdaptiveStreamingSourceFactory : IMediaStreamSourceFactory
{
public virtual MediaStreamSource Create(MediaElement mediaElement, Uri uri)
{
return new AdaptiveStreamingSource(mediaElement, uri);
}
}
This factory can then be set on the base class in the constructor…
public Player()
{
this.SourceProperty = DependencyProperty.Register("Source", typeof(IPlayerSource), typeof(Player), new PropertyMetadata(null, new PropertyChangedCallback(Player.OnSourceChanged)));
base.MediaStreamSourceFactory = new AdaptiveStreamingSourceFactory();
}
As you see in the constructor I have a DependencyProperty called Source. When the source is changed, the following is code is executed
if (url.Contains(".ism"))
{
PlaylistItem item = new PlaylistItem();
item.MediaUrl = new Uri(url + "/manifest");
item.IsAdaptiveStreaming = true;
base.Playlist.Add(item);
}
else
{
PlaylistItem item = new PlaylistItem();
item.MediaUrl = new Uri(url);
item.IsAdaptiveStreaming = false;
base.Playlist.Add(item);
}
Setting the IsAdaptiveStreaming to true, causes the player to use the MediaStreamSourceFactory to get the source instead of just using the url.
I hope this helped…I haven’t been able to find that much information about this on the almighty Google… If you have any questions, you can just send a message or add a comment as usual…