This is a big week for the iPlayer team - we're launching an all new BBC iPlayer with a host of new features designed to make it simpler to use, personalised and social.

The new site is in public beta right now - you can try it out at https://beta.bbc.co.uk/iplayer - where it will dual-run alongside the existing iPlayer site while we get your feedback, fix bugs, and add the remaining features that didn't quite make it into the first beta release.

But before looking at the new site in detail, I'd like to take a step back and try to explain the design challenges we were trying to solve and the solution we came up with (or, if you'd like to skip the background story and head straight to the list of new features, scroll down to features in detail below).

BBC iPlayer: the story so far

designing a platform capable of handling our rapidly growing traffic

ensuring that content became available in iPlayer as soon as possible after it aired on TV

providing the best possible video quality

improving the reliability of video delivery, including failover between content delivery networks, adaptive bitrate for people on lower bandwidth connections

dealing with massive peak loads - the so-called "Top Gear effect" when 100,000+ people descend on the iPlayer site directly after programmes like Top Gear and Doctor Who finish on TV

The current version of iPlayer, known internally as iPlayer V2, was launched almost two years ago in July 2008. Back then the main problems we had to solve were largely technical things like:The iPlayer V2 hosting platform was also designed to scale across multiple platforms - mobile, TV sets, set top boxes, games consoles, PCs, iPhone, etc.

One issue that we needed to solve when delivering content across so many devices and platforms was that in some cases we only had the right to make certain programmes available on, say, PC platforms but not mobile or TV platforms. Additionally, the media files for each platform take different amounts of time to encode, which means that we need to deal with situations where a programme is available on some platforms but not others.

This meant that we couldn't make the same version of the iPlayer site available on each platform. That would mean people on mobile devices might get offered links to programmes that are not available on that device, giving an error when you clicked to play the programme. So we introduced a concept that we called Actual Availability, which allows the iPlayer publishing system to offer independent content sets - we call them Media Sets - to different devices.

BBC iPlayer V2 has proven a trusty workhorse, successfully scaling to 1.5 million users, 15 million page views delivering over 1.1 billion(!) minutes of video each month across more than 40 different devices and platforms. You can see the list at https://www.bbc.co.uk/iplayer/where_to_get_iplayer/.

A new iPlayer: the design challenge

Two years ago when we launched iPlayer our goals and challenges were largely technical - scalability, reliability, video encoding, etc. But as we began work on the next-generation iPlayer, it became apparent that our next set of challenges was not so much technical as social - and that turned out to be a much bigger problem to solve.

At the highest level, the fundamental problem that the iPlayer design team was trying to solve was "As people begin moving from television to the web, what happens to the role of the linear TV scheduler as the tastemaker?" Let's be clear: we are not there yet. Although iPlayer traffic is doubling each year, it still only accounts for 2-3% of linear TV viewing. But something seems to be stirring. Let me explain:

When you turn on your TV in the evening peak viewing hours and idly flip through the available channels, the programmes that you see are carefully chosen by each channel's scheduler. He/she picks the programmes that you can see, and since most of the country watches TV each evening, the scheduler is the leading tastemaker. By scheduling a particular programme at peak viewing time, the scheduler is both creating and satisfying the desire to view that programme.

Now, today iPlayer does a fine job of satisfying the time-shifted desires created by the scheduler: the BBC schedulers create the desire to watch a programme; iPlayer lets you see it at a time that's convenient to you.

But what if you no longer watched linear TV? Who becomes the tastemaker then? Right now this is a largely theoretical problem as very few people watch no live TV at all. However, for a small but growing number of people this is indeed the case, and the fundamental problem that I sought to address was "who becomes the tastemaker for such people in a world without schedules?"

Now I use Twitter periodically during the day to monitor what iPlayer users are saying about the service - "thank god for iPlayer", "waiting for Doctor Who to arrive", "iPlayer slow today", etc. - Twitter is a great early warning tool for spotting problems. But increasingly I began seeing Tweets from people saying "Watching ", "Loving Charlie Brooker on iPlayer", etc. I began clicking on those links, and found myself watching more programmes in iPlayer than I would have by browsing. In other words, for me, the Twitterverse is becoming the tastemaker.

Looking at developments across the industry in this space, it's clear that I was not alone. Particularly in the world of YouTube where there is no master scheduler who can shape demand. The tastemaker is rapidly becoming your friends.

Separately, our iPlayer stats told us that, while our users really liked the service, most only came back every week or two when they had missed a programme on TV - clear evidence that linear TV created the demand while iPlayer satisfied it. In order to get more users to iPlayer, we needed to make iPlayer something more than TV catch-up alone - we wanted it to become a driver of demand, so that you returned to iPlayer daily to see what new programmes were there just for you.

The question then is, in a world which cannot be driven by schedulers, who or what will play the role of tastemaker? Well, we think that's going to be a mix of things that your friends recommend, things that our servers recommend based on what you've watched, things that you tell us you like, as well as the linear scheduler, whose selections continue to matter to an important part of the online audience.

So, if schedulers are going to be augmented by your friends as drivers of consumption in the future, the challenge for the team was to integrate friends and social into the iPlayer site, and to do so in a way that doesn't alienate people who aren't interested in this kind of thing. Sure, it's easy enough to sprinkle Share, Recommend, Digg, Follow, etc. buttons across the site, but social shouldn't be a prerequisite to participation or add complication or clutter for those who just want to get going. It's about giving more choice and control.

The iPlayer design team thus found itself with a major challenge: Take a popular and mainstream product, and reinvent it so that it becomes not just a place you go to catch up on programmes that you know you missed, but to become the place where demand is both created and satisfied. Oh, and to do that in a way that doesn't make the site more complex, and in a way that delights both early adopters and the mainstream audience. Make it personal, make it social, and keep it simple.

Challenge #1: Making the site personal without creating a separate 'you' site

Here's an example of the type of challenge we needed to address: If iPlayer was to become your personal viewing portal, then there needed to be an area of the site that you could call "yours" - i.e. a place where you could assemble all your favourite programmes, and only your favourite programmes.

Initially we decided to create a new area of the site called My iPlayer, which would be your personalised place to find all your favourite programmes. But it became apparent that creating a My iPlayer page, separate from the rest of the site, would mean separate user journeys and duplicated content between the main site and your personal site. In the end we dropped the concept of a separate personal site and instead folded your personal experience into the fabric of the main site - something that will become apparent as we look at the features in more detail below.

In short, we sought to add a large range of personalisation and social features to iPlayer, making every feature part of a coherent whole, and avoiding adding anything that didn't have a clear purpose.

Challenge #2: Integrating social connectivity

press the Recommend button on your site >> activity posted on Facebook or Twitter on Facebook or Twitter, see activity, click link >> go back to the site hosting the recommended content

Another challenge was how to integrate with Facebook and other social networks. Pretty much every site these days has a Share button which posts your activity to Facebook, Twitter or other social networking sites. The usual interaction model is like this:



That's great if you want to use social networks to increase incoming traffic to your site - which is nice - but that detour via an external 3rd-party site seemed to us to create a disconnect between the user pressing a Share or Recommend button and getting the reward for that action. And we want it, above all, to be seamless and simple.

Our thinking was that in order to create a lively social recommendation scene, we needed to make the recommendations and social graph visible within iPlayer, in addition of course to any external activity.

Additionally, we wanted to allow a single click of a Recommend button to post activity to both Facebook or Twitter (or any other network or micro-blogging site for that matter) where appropriate, in addition to posting that activity into our own activity streams.

The solution we came up with was to create a BBC login - known as BBC iD - which users can then connect with Facebook, Twitter or any other social network that we choose to partner with in the future, allowing us to create the following social recommendation ecosystem:



press the Recommend button in iPlayer >> activity appears on your friends' iPlayer home pages >> AND activity posted on Facebook AND/OR Twitter on Facebook / Twitter, see activity, click link >> watch it in iPlayer



Basically, we use your external social graph to connect you with your friends within the iPlayer site, and make it scalable for other BBC Online services in the fullness of time. To do this, we connect Facebook, Twitter and where appropriate other social networking sites to your BBC iD login account, providing you with a single gateway to multiple social graphs and a single Recommend button that can post activity to multiple social networks.

The new iPlayer: features in detail

And so, without further ado, let's take a look at all the new features in the new iPlayer, with commentary explaining the rationale for each feature.

Some of the features listed below didn't make it into the first beta release and will be added over the coming weeks (features that are 'coming soon' are noted in italics below).

1. Favourites

The single feature request that we are asked for most often is "favourites" - and, as explained above, integrating favourites into the site was one of our early dilemmas. The solution that we came up with was to add an expandable Favourites zone to the top of every page of the site:In response to our user research that told us that people only came back to the site every week or two to catch up on programmes they'd missed, we designed Favourites to be like your mail Inbox, showing the total number of items, how many are newly arrived, etc.

The great thing about Favourites in the context of iPlayer is that by simply adding, say, QI as a favourite, every time a new episode of QI goes to air, your favourites list will update and reorder itself to show the new episode. Adding items to your favourites takes only a single click, so I found myself spending the first 5 minutes on the new iPlayer site simply looking for any series that I liked and clicking to add it to my favourites. I now have - as can be seen in the image above - 48 items to watch right now, with the iPlayer servers constantly scanning for new episodes becoming available, adding them to my favourites, and very shortly sending me an email notification.

Now that I have so much stuff to view, I can almost ignore the rest of the iPlayer site and simply rely on Favourites to give me a constant stream of things to watch.

2. Roaming

You know how on many sites every time you do something simple like adding a programme to your favourites or trying to personalise the site, you get prompted to register - very annoying. So we carefully designed the new iPlayer site so that you could use most of the functionality instantly on first visit without being constantly hassled to register.

But... if you do choose to sign in, then all your favourites and other settings can roam across all the devices on which you use iPlayer. For example, we're planning on updating the mobile iPlayer site - the one you can access on a range of mobile phones - to let you access your Favourites as well - here's an image of Favourites in the new Mobile iPlayer site:





So now if I'm bored sitting in a train on the way home, I can look for new programmes to watch, add them to my Favourites, and when I get home they'll be right there on my home PC ready to watch.

Full roaming is coming to the mobile iPlayer site shortly - the initial implementation contains local favourites only.

But what will make this good service great is when we can synch up your BBC iD with your TV or mobile, so you can pick up where you left off on whatever connected device you have - the team are working on this now and we hope to say more about this soon.

3. Personalised iPlayer home page

Going back to our dilemma of how to provide a personalised iPlayer portal experience that was not isolated from the main iPlayer site, our solution was to create an iPlayer home page that can morph, under your direction, from a default view that everyone sees to something that's, well, just for you.

We wanted to create an iPlayer home page that feels almost more like an application than a traditional web site, making it a familiar place you return to frequently for your favourite comedy, drama, music and more.

Here's how it works:

When you come to the iPlayer site as a new user, you're presented with a nice simple promo zone at the top of the home page that contains the Featured and Most Popular zones that would be familiar to any existing iPlayer user:

Now, as soon as you've played a couple of programmes, our recommendations system has enough information to guess what you may like and offer personalised programme recommendations for you, and so when you next return to the iPlayer home page you'll now see two extra zones: For You and Friends:

Now here's where the personalisation comes in: you can slide open any of the drawers to turn the iPlayer homepage into the tastemaker of your choice. For example, if you'd like your viewing to be driven by programmes that the BBC editorial team has chosen, simply slide open the Featured drawer:

Or, if you're so inclined, sign up on the site, then connect iPlayer to your Facebook and/or Twitter social graphs, and you'll get a steady stream of recommendations from your friends:

These sliding drawers will remember the state that you left them in, allowing a single iPlayer home page to meet the needs of a mainstream audience looking for editorialised recommendations, through to users who look to their friends as the tastemaker.

Remembering the open/closed state of each of the drawers is being added shortly.

4. My Categories

In addition to Favourites - where you nominate your favourite shows or series - we also added a My Categories zone to the home page. To use it, simply navigate to any category that you normally like to watch (or listen to - this works for radio as well), click Add To My Categories, and then the iPlayer server will keep a lookout for any new content in your selected categories, and the iPlayer home page will show you a constantly updated list of new programmes in those categories. As you can see below, I like classical and world music, and science & nature programmes:





5. Better live TV

Although you've been able to watch live TV in iPlayer for well over a year now, this isn't that well known, but recently we've seen a big increase in live TV viewing in iPlayer - and with the upcoming World Cup being a huge driver of live online viewing, we're making the live viewing experience a little more prominent on the home page:

and we've also created a new Live Viewing page which allows you to easily view all the BBC TV channels in iPlayer:





6. New radio console

We created an all-new popup radio console that includes Favourites and other key features from the new iPlayer site, allowing you to find and listen to your favourite BBC radio programmes all within the popup console player:









7. Recommend to friends

The fuel for the Friends drawer on the iPlayer home page - and shortly on the playback page as well - is the Recommend button that appears below the playback window:

As mentioned above, you can link the Recommend button to the social network(s) of your choice via a single BBC login, and our servers will constantly check your social graph on those sites and import latest friends additions and deletions across all your networks.





8. Watch with friends

And now something that for some will be the killer feature of the new site: the ability to watch programmes with friends. If you already have a Windows Live Messenger account you can see which of your Windows Live Messenger friends (and other instant messenger services to be added in due course) are in iPlayer right now and what they're watching, and even how far into the programme they are. You can then sync your iPlayer with theirs and chat with them in real time, all within the iPlayer site.

Here's how it works:

On all TV playback pages in iPlayer you'll see a button to add the IM chat widget to your iPlayer pages. If you're a Messenger user and this is of interest to you, click the Get Started button.

After signing in to Windows Live Messenger with your Messenger credentials, you'll now see an extra panel that shows which of your Messenger contacts are online and in iPlayer right now:

Separately, while you're watching a programme, anytime you're feeling excited about that programme or even just a particular moment in the programme, you can shout about it to your Messenger friends - simply type whatever comes to mind into the text box and hit the Shout button - all your Messenger friends who are in iPlayer right now will get the message, and may then choose to sync their iPlayer to yours and join you to watch and chat together.

By the way, your shouts only go to your Messenger friends who are in iPlayer right now - they won't go to contacts who are not in iPlayer - so you don't need to worry about spamming contacts who don't live in the UK or who aren't interested in your shouts of "It's the Stig!" or whatever.

The Messenger that you can add to your iPlayer site is a JavaScript implementation of the Windows Live Messenger client - i.e. your private chat conversations travel over the same MSN network as regular Messenger IM chat.

Watch with Friends is being added to the site in the next few weeks - stay tuned!





9. Better video quality

For quite some time now iPlayer has had the ability to switch down to a lower bitrate video stream if you didn't have enough bandwidth to play the selected version. We're now rolling out the next evolution in our adaptive bitrate system which automatically adjusts the video quality up and down every few seconds, if necessary, to match your instantaneous line speed. The improved video quality will be most apparent in full-screen mode, where iPlayer will automatically switch up to our 832x468 1500Kbps high-quality SD streams as soon as you go fullscreen, seamlessly dropping to/from the 480Kbps and 800Kbps lower bitrate streams as needed.

This new adaptive bitrate system, coupled with Adobe's upcoming Flash 10.1 release with H.264 hardware acceleration, should give better quality, less jerkiness and lower CPU usage on PCs equipped with a graphics cards that support H.264 hardware acceleration - see https://labs.adobe.com/technologies/flashplayer10/releasenotes.pdf for technical details.





10. New iPlayer Desktop

In addition to the new iPlayer web site, we also have a greatly enhanced version of iPlayer Desktop, our Adobe AIR-powered iPlayer download manager, now with two great new features: Series Downloads and live radio & TV.

This means you a) don't have to wait to download programmes you want and b) you've got your favourite programmes already downloaded to your computer ready to view when you're offline.

Downloading whole series to your computer is easy - just click the Download button on iPlayer playback pages and choose the "Download future episodes" button:

iPlayer Desktop will now automatically download every future episode for you, including where possible downloading episodes before they air on TV, making them available for you to play back within minutes of the programme finishing on TV.

To avoid iPlayer Desktop maxing out your internet connection we've added the ability to tell iPlayer Desktop to schedule all series downloads between, say, midnight and 7AM:

Additionally, iPlayer Desktop will pause any automated downloads if previous episodes start expiring without being watched, avoiding end of month bandwidth surprises.

One of the features that I'm really liking is the new feature in iPlayer Desktop for live TV and radio too, which puts the BBC's 17 network and national radio stations....

and 7 TV stations...

...all right there on your desktop for instant listening and viewing.





10. And finally...

As I hope you can see, this relaunch represents many months of work for the iPlayer team and gives us a platform that will serve as a base the next wave of innovation over the coming months.

As in any beta there will be bugs, and we still have some features to add. This new site is a big step - and a big bet - for us, and we'd really welcome your feedback - look for the Site Feedback link at the bottom of each page on the site.

We really hope you like it. As some of you may know, I am moving on to become CTO of Project Canvas, and this is the last major piece of work from the team under my leadership. It's been quite a journey for me and the BBC since I got that call from Erik in late 2007 "Anthony, I could use your help with something" and I'm very proud of what we've created.

Please keep a lookout for additional blogs from my colleagues on how the new iPlayer site was built, the Java/PHP hosting platform that powers the new iPlayer site, BBC iD and how we connect to social networks, and more.



Anthony

Anthony Rose is Controller, Online Media Group and Vision, BBC FM&T.