Summarizing videos on the Web

Journalists, broadcasters, as well as simple users regularly create video content to be published in established (e.g., TV) and emerging (e.g., Twitter, Facebook, YouTube, Instagram) channels. Whilst content produced for TV (e.g., an episode of a TV show) or for an amateur’s own use (e.g., vacation footage) can be quite lengthy, this is not optimal for content distribution on social media. We live in an era where social media users’ demands dictate the production of short and attractive audio-visual content that attracts the users’ attention and can be ingested quickly; studies show that audience engagement drops significantly during the playback of long videos. Therefore, for sharing on social platforms, video creators often need a trimmed-down version of their original full-length video. Also, different social platforms impose different restrictions on the duration and format of the video that they accept, e.g., on TikTok up to 60-second videos can be shared, whereas on Instagram stories the upper limit is 15 seconds. This makes the generation of tailored versions of video content for publication in multiple platforms a tedious task.

In the context of the ReTV project, and motivated by the lack of web-based video summarization tools, we built a freely accessible Web application that enables users to submit locally-stored or on-line available videos and automatically generate shorter versions of the submitted full-length video. This Web service harnesses the power of artificial intelligence – Generative Adversarial Networks, to be precise – to automatically generate video summaries. It takes as input a video and produces a video summary that encapsulates the flow of the story and the essential parts of the full-length video, adapting the length and format of the produced summary for publication on social media platforms, thus easing the creation of engaging video stories for on-line audiences. 

The framework we designed consists of a backend service that hosts the developed technologies for video summarization and an interactive user interface (shown in the next figure) that allows the user to exploit the functionality of the backend service. In particular through the user interface of this tool, the user is able to:

• Submit a video for analysis (either available on-line or locally stored in the user’s device), and select the specifications of the generated summary from a list of predefined configurations that were properly adjusted for the most common social media and video sharing platforms (Figure 1). 
• Get the created summary in a way that enables both immediate on-line inspection through the user interface of the Web service tool and the downloading of the generated summary in the user’s device (Figure 2).

Aiming to help the user in deciding the appropriate target duration and aspect ratio of the produced video summary, we have created a list of configurations for the most common social media and video sharing platforms. The user can select one using the respective option buttons found in the service’s landing page (Figure 1). This action configures the service to produce a video summary that meets the prerequisites of the target platform. The specifications for 5 widely used social media and video sharing platforms are shown in the next table. These parameters aim to maximize user engagement and experience. Nevertheless, it’s important to stress that these are best practices and they are subject to change based on various trends.

Processing of a video and generating its summary is faster than real-time (where real-time processing would have a processing time equal to the video’s duration); though of course delays may be noticed if multiple analysis requests are submitted to the service, since the latter applies a queuing strategy on the incoming analysis requests and the analysis is performed on a one-by-one basis. The summarization results, as soon as the processing of a submitted video is completed, can be viewed and downloaded in the user’s device (Figure 2).

Using the video analysis service, the team summarised a video from the archive into a 30-second video. Both videos show key shots from the programme that encourage audiences to watch the full video.

 Original video

Summarized video

Interested in creating your own videos? You can watch a 2-minute video on how to use the web service here, and try it out with your own videos or check some pre-processed example video summaries here.