Skip to content
Customer Story

Automated Labels for Better Broadcasting Insights with Gen AI

Xebia helps AVROTROS automate video labeling to get more insights into viewing behavior
{body={src=https://697348.fs1.hubspotusercontent-na1.net/hubfs/697348/AVROTROS_logo_2020.svg.png}, css={}, child_css={}, id=client_logo, name=client_logo, type=image_src, order=8, smart_type=null, styles={}}

Related Services
Google Cloud Platform, Google Speech API, Gemini, Google Video Intelligence

Industry
Broadcasting

Company
Xebia Data

AVROTROS is a Dutch radio and television public broadcaster  that produces a wide range of content. It aims to make culture accessible to all Dutch people, from young to old. 

Some notable content includes: 

  • • News and consumer affairs programs such as EenVandaag and Radar
  • • Entertainment shows like Beste Zangers (Best Singers) and Op zoek naar... (Searching for...)
  • • Comedy shows such as Dit Was Het Nieuws (This Was the News)
  • • Music events like Muziekfeest op het Plein (Music Festival on the Square)
  • • Coverage of the Eurovision Song Contest and Junior Eurovision Song Contest 

AVROTROS focuses on connecting creative talent with audiences and bringing people together through shared cultural experiences.
Why
AVROTROS reaches 2 out of 3 people in the Netherlands per month​. They wanted more insight into their viewing numbers to determine what’s driving people in or away.
What
To get better insights into viewers’ interests and preferences, AVROTROS wanted to build a solution to get granular insights into their videos stored in GCP.
How
Together with AVROTROS, Xebia built a GenAI-powered solution that automatically analyzes raw video files from TV programs to extract granular insights. The solution identifies changes in topic and people in the recordings.

Before the Project 

In a fast-paced media industry, AVROTROS is responsible for broadcasting and maintaining quality TV content. Before the project with Xebia, AVROTROS already collected viewing numbers with minute-level granularity, allowing them to analyze their produced content. They had also created a data analytics platform to democratize access to said viewing data. 

However, they also wanted to segment those viewing numbers based on the topics covered on the show and the people present in the recording. This would allow them to ensure their content aligns with the audience’s needs. Since TV is a fast-changing medium, the topics covered might not be known beforehand and can, therefore, only be extracted from the recordings after broadcasting. This requires manually timestamping each episode of the show, identifying topics and people, and storing the results, which is a time-consuming process currently performed only on a selected number of shows and episodes.   
The labeling solution transforms scattered data into a clear and actionable narrative. It helps us understand the impact of specific topics or guests on viewer ratings. 
AVROTROS_logo_2020.svg
Tjitze Zijlstra Data Analyst

Automated Labeling Solution 

AVROTROS asked Xebia to help implement an automated labeling solution that would identify and timestamp the topics covered during the show as well as the guest speakers, and prepare that data for ingestion into the broadcaster’s Google Cloud Platform (GCP). The implemented solution relies on Google Speech API to detect language and transcribe videos, Gemini to extract topics, and Google Video Intelligence API for face detection and clustering to identify people. An extra computer vision model, in combination with a language model, attempts to identify the name and role of the person on screen. Once the labels are extracted and timestamped, this information is stored in a vector database to allow for efficient and fast search results. 

Additionally, to build trust in the implemented solution and to inspect or correct automatically-created labels if necessary, an intuitive web app was built for end users. 

Avrotros1

Avrotros2

The solution identifies 90% of the topics and people correctly.

The Impact 

The implemented solution can correctly identify and timestamp about 90% of the topics covered in a specific episode, as well as close to 90% of the people. In addition, an intuitive app was deployed, allowing users to correct labels detected by the algorithm if necessary.​  

With the detected timestamps for topic changes and relevant people in TV programs, AVROTROS is now able to associate viewer numbers directly with the topics discussed and the people in their TV content. This allows them to steer the content to align better with the audience’s needs in subsequent broadcasts. 

Explore related customer stories

Questions?

Get in touch with us