Today we want to update you on our ‘Abendgruß’ application. As part of the 4u2 use case, ‘Abendgruß’ application aims to provide consumers with quick and easy access to personalised content from broadcasters and media archives via novel publication channels. In this context, we’ve developed ‘Abendgruß’ for use with smart speakers.

Focused on RBB’s well-known TV programme  ‘Unser Sandmännchen’ for pre-school children, young users (and their parents) can create their own Sandmännchen episodes. Free from editorial restrictions, users can independently select their own content. The app is supported by Artificial Intelligence-based functions for video adaptation and re-purposing.

Technical bits

Typically, the Sandmännchen episodes follow a similar structure. There is a framing story, consisting of an intro and an outro (the Sandmännchen arrives and leaves a particular setting in a particular way), and there is a main story, usually an adventure that involves  one or more of  his friends. Our video analysis framework divides an episode into these three segments, and subsequently annotates the main storyline with the main characters involved in the story. Currently, eleven  Sandmännchen characters can be annotated. These analysis techniques have been incorporated into a video analysis component, deployed as a REST service.

The Abendgruß API periodically checks whether new Sandmännchen episodes have been published, which it does by monitoring selected web sources. If there are new episodes, they are sent to the video analysis service and the results are stored in the Abendgruß database, ready for integration into future stories.

How it works

We developed an Abendgruß action for Google Assistant, focusing on the Google Nest Hub. Here’s how it works: 

• To start Abendgruß, the user says “OK, Google, mit Abendgruß sprechen (OK, Google, speak to Abendgruß).” 
• The Nest Hub answers: “In Ordnung, ich starte die Testversion von Abendgruß (OK, I’m starting the test version of Abendgruß).” 
• The application opens. 
• The user sees the start screen and a welcome, combined with a call to action: “Hallo! Um deinen eigenen Abendgruß zu sehen, sage das Wort Abendgruß (Hello! To watch your own Abendgruß, say the word “Abendgruß).” 
• After saying “Abendgruß”, the user sees two options. First, the user can choose how the Sandmännchen should arrive. For example, “Mit dem Zeppelin oder auf dem Roller? (By airship or on the scooter?)”. In other words, this step defines the framing story. 
• Secondly, the user can determine the main story by answering the question “Und welche Geschichte möchtest Du heute sehen? (And what story do you want to see today?)”. Two options are presented, e.g., “Herr Fuchs und Frau Elster oder Jan und Henry? (Mr Fox and Miss Magpie or Jan and Henry?)”.
• The Abendgruß application then shows an automatically generated Sandmännchen episode with a storyline featuring the selected elements and choices.

Take a look at the following video, where we showcase the ‘Abendgruß’ application in action:

In the future, our goal is to develop a version of the application that can be fully implemented with Amazon Alexa. We will also adjust and expand the video analysis components to enable creation of even more diverse personalised episodes using voice commands. Specifically, we plan to expand the set of identifiable characters to include all the friends of the Sandmännchen. We also plan to automate annotation of the intro and outro sections of an episode with information on the vehicle that the Sandmännchen uses in the particular episode.

Are you interested in trying out our solution? Let us know! Send us an e-mail to info(Replace this parenthesis with the @ sign)retv-project.eu or join our stakeholder forum here.