In the project, we’ve been exploring and researching how end users could benefit from AI-based recommendation and user profiling systems. The result: the 4u2 use case, which aims to provide consumers with quick and easy access to personalised content from broadcasters and media archives via novel publication channels. To achieve this goal, two prototype applications were developed.
Personalised, AI-driven Content
4u2 for Messengers
The first application is a chatbot for the messenger service Telegram. Subscribers are given the opportunity to receive automatically generated video summaries tailored to their needs. For this purpose, each user individually determines the main video properties (e. g. topic, frequency, time, length).
The use case described below is based on the rbb show ‘zibb’ and the corresponding ‘zibb Messenger’. ‘zibb’ is broadcasted on rbb every weekday at 18:30, reporting on daily life in Berlin. From 2017 to 2019, the editorial department offered the ‘zibb Messenger’ for WhatsApp and Telegram to send users text messages about topics based on their preferences. User engagement was high, so we decided to improve on the chatbot for ReTV by offering personalised video content.
How it works: Use Case for a ‘zibbi’ Subscriber
Linda is a 44 year old woman, who lives with her two children and her husband in Potsdam, Germany. She is interested in topics about the local region. She watches ‘zibb’ to be up to date, however she has limited time. She needs a service which provides her with the ‘zibb’ content she wants, when she wants it.
This is where ‘zibbi’, the 4u2 chatbot, comes in. After registering for Telegram and subscribing to ‘zibbi’, Linda defines various preferences: she chooses trending, guests and service as her favourite topics and can add an option for stories about her local area. Then she chooses a video summary every day in the afternoon with a maximum length of two minutes.
From now on, ‘zibbi’ will automatically send her a personalised video summary every afternoon. She is able to change her settings whenever she wants. This entire profiling process is convenient for Linda but is also necessary so that the ReTV content services can work together as intended, i. e. the cold start problem is solved.
In addition to the daily update, Linda also has the opportunity to request ad hoc videos and ‘zibbi’ will send her a video summary of the latest ‘zibb’ video from her chosen category. The presentation of videos offered by ‘zibbi’ for both regular and ad hoc receiving is the same. First Linda gets a short GIF, composed of four characteristic scenes. After watching this three second preview she can decide whether she wants to see more or not. If the answer is yes, Linda has three options to watch a summary, the full video) or read the article.
The preview GIF as well as the video summary are automatically generated by an AI. Since the creation is based on the individual settings of each user, every video adaptation is unique. So by using ‘zibbi’ Linda really gets exactly what she wants: individual ‘zibb’ videos tailored to her personal interests and needs.
4u2 for Smart Speakers
The second application is an app for the Google Assistant. Since ReTV focuses on video content, it was designed primarily for use with Google Nest Hub, i. e. a smart speaker with display. The app enables users to create their own videos from existing archive material using voice commands.
Its current version is based on the rbb children’s programme ‘Unser Sandmännchen’, which is well known in Germany. It’s a seven minute show, in three short segments. It’s broadcast daily, just before 18:00, and targeted at pre-school children, to send them to bed with an ‘Abendgruß’ (bedtime story).
How it works: Use Case for ‘Abendgruß’
It’s Tuesday, 18:30. Bedtime for four-year-old Paul. But without his daily ‘Abendgruß’ he doesn’t want to go to sleep. It’s too late to watch today’s episode on TV, however, Paul can create his own ‘Abendgruß’ today. Paul and his mom sit down in the comfortable armchair in the living room. On a small table right next to it they have their Google Nest Hub. Paul says the magic words: OK, Google, call ‘Abendgruß’. The Nest Hub then prompts Paul to choose between two options for each segment of the show and in the end Paul’s episode is shown on the display. When the sandman throws his sleeping sand, he also rubs his eyes. Now Paul is ready to sleep.
We decided to use the ‘Sandmännchen’ episodes to develop this use case because of their simple and consistent structure: there is always a frame story, composed of an intro and an outro (Sandmännchen arrives on/leaves a special setting in a special way), and a main story (an adventure with one or more friends). Therefore it is easy to train an AI on it and thus to realise the fully automated process of video adaptation and repurposing. This also includes the ingestion of new episodes as well as video fragmentation & annotation – initiated by voice commands, as described above.