Ncam explains how it created the augmented reality in BT Sport’s east London studio space.
BT Sport’s production base is in what was the International Broadcasting Centre for the 2012 Olympic Games in east London.
Because of this remarkable and unique space, the designers of the BT Sport service, working with its technology partner Timeline Television, elected to take an unusual approach to its host broadcasts. Where other broadcasters have moved the sports pundits into ever-smaller spaces, using virtual studios to make them look bigger, BT Sport built an enormous studio with equally impressive flexibility.
Generally recognised to be the largest television studio in the UK and probably Europe, it is an L-shaped space covering 14,500 square feet (1350 square metres) and with a lighting grid a around 15 metres high. Interestingly, they routinely use this to produce two and sometimes even three live programmes simultaneously.
Within this vast space there are many permanent dynamic elements. At the corner of the L, providing a boundary between the two main production areas, is a very large cylindrical structure wrapped in large video screens (known as the BT Tower). In the larger part of the studio two walls are also covered in vertical LED display screens, broken up with coloured light panels. This gives designers the ability to match the otherwise black box to the time of day, and to show relevant video clips and stills.
On the third wall (the fourth contains windows to two galleries) is a 4k projection screen and in front of it, covering close to half the length of the studio, is a glass floor. LED lighting arrays beneath the glass allow this to display set-ups relevant to the event being shown: a full set of football or rugby pitch markings; a football penalty box; a tennis court, and so on.
While the BT Tower provides something of a barrier between the two parts of the studio, and certainly mitigates the sound, it does not completely divide the two areas. The production style allows one stage to be seen by the other’s cameras, making it very visible that there is more than one channel and more than one game being covered at a time.
BT Sport rapidly established itself as a leading broadcaster, not least through the style of its presentation as well as the coverage of the events itself. It was time to move to the next stage of production development: the use of augmented reality.
The challenge
BT Sport’s decision to use augmented reality was carefully thought through, as a genuine augmentation to what was already a sophisticated studio production. The physical elements in the studio – player pictures on the walls via screens, the match coverage on the giant 4k projection screen, the glass floor with its LED pitch marking – were already very impressive.
The use of augmented reality had to fit into this environment. BT Sport’s designers and directors were clear about what they wanted to achieve. They wanted something that would create a sense of occasion around the most valuable property: coverage of the UEFA Champions’ League, a club football competition among the leading clubs from around Europe.
The scale of the studio meant that productions in the larger end routinely used eight or more cameras, including pedestals, rails, Steadicam and a jib. The fluid direction of the programmes could not be constrained just because augmented reality was in use.
The other part of the challenge was that the studio was essentially a black box, with any part of it available to be in shot. That meant that camera tracking using targets or markers was impractical. The augmented reality graphics engines would have to know what the cameras were seeing without any additions anywhere in the studio.
Ncam
To create convincing augmented reality, the virtual elements have to appear to be firmly anchored in the real environment. They cannot be seen to move or the illusion is broken and they become a distraction rather than an enhancement.
To do this, the graphics engines have to be fed precise information on where the camera is in three dimensional space; its angle of pan, tilt and rotation; and the lens settings (zoom, focus and aperture).
Ncam’s camera tracking technology is unrivalled in determining with remarkable precision the camera position in its six degrees of freedom, thanks to a combination of optical and mechanical sensors. In particular, it uses stereoscopic cameras to detect natural points within the scene, which it maps to create a continuously updating set of references from which the position and angle can be calculated.
The Ncam cameras and motion sensors run at very much faster than standard television frame rates – 250 frames a second – ensuring high positional precision and a degree of prediction of where a camera move is going. This also ensures very low latency in the data streamed to the graphics engine.
Ordinarily, the Ncam detector is a bar which mounts beneath the camera lens, facing forwards, and using the scene itself to create its reference map. A small data multiplexer on the camera also brings in information from the lens, and streams everything to an Ncam server (a standard PC workstation) over ethernet. From there, positional data is constantly streamed to the graphics engine.
The challenge at BT Sport is that the studio is essentially a black box, which does not provide the multiple and randomly-placed points of high contrast from which Ncam typically calculates position and movement. As part of the sales process Ncam proposed some modifications and while the solution did require some changes to the core algorithms and calibration techniques – which calculate precise position and orientation – it was quickly demonstrated to be a perfect solution.
In operation
On a Champions’ League night, augmented reality is available from three cameras: usually one on a tracking system, one on a jib and one on a pedestal, although if the director prefers it one can be on a Steadicam. Each is fitted with Ncam tracking and each has its own Ncam server to stream the positional data. In turn, each Ncam server feeds a separate graphics engine (by RT Software), but they run the same graphic modelling and are linked to ensure continuity between shots. So the director is free to control the flow of the programme, and can switch between views of the same augmented reality models.
As noted, BT Sport wanted a signature shot, a memorable moment to make its Champions’ League coverage stand out. This comes near the beginning of the presentation when the big glass floor becomes a map of Europe which the cameras move over – the jib camera taking viewers on the journey – towards three-dimensional pictures of the stadiums in the cities where the games are to take place.
During the pre-game discussions team line-ups appear as three-dimensional objects in the relevant places on the pitch, with pictures or action footage of each on the 4k projector behind them as each player is discussed. Post-match, the pitch on the glass floor can be used to illustrate specific moves with individual players appearing in place, for heat maps of action, and more.
“The danger with augmented reality is that it could be like giving a kid a machine gun,” said Jamie Hindhaugh. “For some, the temptation to go wild and have stuff whizzing in and out, flying around could to be too tempting.”
“We were determined to use it only to enhance our coverage, to support the analysis and to engage with the viewer,” he explained. “This natural approach meant that the output had to be perfect: the augmented reality graphics had to sit exactly where they were supposed to be, and to stay there.
“Achieving that – and our audiences tell us that our coverage is very highly appreciated – demanded close co-operation between us and our technology partners to achieve our vision. That could happen because of the contractual relationships we as the customer set up: we created an environment where it was in everyone’s interests to work together to achieve a great outcome, and in this case stunning sports television.”
Staff Reporter
Share this story
