StudioNeural delivered AI VFX for BBC Studios’ The Murder Detective, a docuseries about the police investigation into the murder of journalist Lyra McKee, who was shot when reporting on a riot in Derry, Northern Ireland in 2019.
The series – seven years in the making – was filmed, directed and produced by James W. Newton and executive produced by Joe Mather.
The observational documentary follows police investigators in the unfolding terror-related murder investigation, with the BBC Studio team embedded with the PSNI’s Serious Crime Branch and filming with its Intelligence Branch and MI5.
Given the highly sensitive nature of the access, story and footage, StudioNeural partnered with BBC Studios to digitally veil and anonymise over three dozen investigators to conceal their identities within a highly controlled information security environment.
“Paramilitary groups like the New IRA are known for wearing masks to conceal their identities as they carry out horrific crimes. We took that power of concealment – the mask, enhanced it with AI and stringent security protocols and applied AI VFX to protect the people that are trying to prosecute a terrorist group and bring justice for Lyra McKee,” says Barnaby Francis, founder of StudioNeural.
Due to legal and safety concerns regarding participants exposure to threats of retaliation from the New IRA, StudioNeural was approached to deliver AI VFX work whilst the series was in the offline edit. Pre-production technical planning and guidance were not possible, instead they worked on shots containing multiple subjects, with a range of lenses, cameras, complex focus pulls, blurring, extreme face angles and heavily occluded faces throughout the series.
In order to achieve this, StudioNeural had to develop and upgrade their Neural Suite face capture and performance transfer toolkit.
The brief from BBC Studios was to maintain the integrity of the principle performances of over 30 contributors featured across the series, so StudioNeural applied their AI pipeline and improved it to meet rigorous information security classifications including ISO27001 compliance and a full AI audit. Then StudioNeural provided creative to develop AI generated identities comprising of faces, hairstyles and voices for each investigator, and BBC Studios obtained sign off from each investigator for their AI identity prior to any AI VFX work commencing.
StudioNeural has experience applying face capture and performance transfer for TV productions and BBC Studios wanted the AI VFX to preserve as much original performance as possible and recreate exact facial expressions and nuanced emotions of each participant in order to maintain character-led authenticity across the series.
Behind the veiled investigators is a ‘grey stage ready’ face capture and performance transfer toolkit that can process up to 32-bit linear float images in ACES colour and .EXR format. An upgraded face capture model was developed to be lens and camera agnostic, and capable of capturing and tracking complex face angles including faces with over 50% occlusion, across both day and night-time environments.
“A large part of our work was developing our AI toolkits and automating parts of our comfyUI workflow to deliver over 60,000 frames of AI VFX” says Rahul Raj, Head of AI at StudioNeural. “Our face detection, recognition, masking, cropping, and aligning algorithms needed to automated to work at scale prior to beginning AI work within our Neural Suite toolkit. We could then also begin the parallel task of building the AI voice models to a specific regional dialect, in order to mask the voices of the investigators along with their faces”. The rendered plates could then be passed into Nuke, Blender or Mocha for 2D and 3D VFX work to change hairstyles and other identifying features.
“One scene with the lead investigators meeting with their teams was pivotal to the story,” says Francis. “It was important for this group shot to emotionally connect the viewer with the integrity of the investigators during their appearance on screen. The scene features a series of close up shots with multiple characters in frame, including focus pulls, occlusion and blur.
“We can see the original performance of the subjects translated through our AI face capture toolkit, not as a replacement for their original performance – but as a subtle recreation that preserves the impact of each character’s emotional expression and appearance on screen.”
The Murder Detective launched on Channel 4 26th August.
Images have been restricted for publication due to security concerns.
Pippa Considine
Share this story
