geralt / Pixabay

AI/ML is real for M&E – here and now!

Artificial Intelligence (AI) and Machine Learning (ML) have gone beyond being catchphrases today. M&E industry is looking to reap real business benefits like revenue maximization and efficiency improvement from AI/ML.

Automatic cognition of video and other media content is by and far the most important contribution of AI towards M&E industry. A lot of super beneficial and path-breaking business values can be derived out of automatic content recognition using AI.

Industry is abuzz with stories like IBM Watson creating Trailer for the movie Morgan, highlights for US Open using AI or more recently the FIFA World Cup highlights.

For most M&E enterprises, the biggest potential of AI lies in automatic content recognition, which can drive several path-breaking business benefits. For instance, most content owners have thousands of video assets. Cataloging, managing, processing and re-purposing this content typically requires extensive manual effort. Advancements in AI and ML algorithms have now made it possible to drastically cut down the time taken to perform many of these tasks.

Basic tools for content recognition are provided by OEMs like Google, Microsoft, IBM and Amazon, as well as a number of start-ups. But there is still a lot of work to be done – especially as ML algorithms need to be trained using the right kind of data and solutions to achieve accurate results.

Media Companies to adopt clear AI Strategy

If your company is into media creation, production, post-production, distribution or even into building technology around media – you should have a clearly articulated AI strategy. It’s no longer a question of “if” or “when”, but rather a question of “how”.

Your strategy should include knowing which AI tools to use, who to turn to for right advice through your AI/ML journey and which software solutions will yield real business benefits. You will also need to decide what to build in-house and what to buy off-the-shelf, how many AI Engineers or data scientists to hire and for what job(s). The other important consideration would be what data to use for training your machines (computers/computer programs) and how to protect the resultant ML models from being used by competitors and other companies.

Just like your video assets, you need to take great care to protect the AI models created leveraging your content and other data.

The Economist recently floated an eye-opening theory – the world’s most valuable resource is no longer oil, but data. We couldn’t agree more. Data and the ML models built around them will soon become priceless assets for companies across the globe.

The AI/ML Triangle

The challenges and opportunities in the media cognition space can be depicted as a triangle with 3 vertices: 1) Tools 2) Consulting, and 3) Product Software

AI/MLTriangle of M&E industry


The AI tool landscape features specialized cognitive cloud services from the usual suspects like Google, Microsoft, IBM, Amazon, Adobe and the likes. These provide easy API integration to recognize different aspects of video content e.g. general objects (like human, trees, colors, theme etc.), voice transcripts, shot boundaries, on-screen text, OCR (Optical Character Recognition) celebrity faces, emotions, actions, locations, brands etc. Some of them also provide custom training engines to create new ML models using a customers’ own training data.

There are also aggregation or synthesis vendors who combine AI engines and APIs from different OEMs and increase the ability of computers to recognize a higher number of attributes from the same piece of content.

Next, open source based ML model builders like Tensorflow or Caffe are also available. One can use these to build different types of Convolution Neural Networks (CNN) that mimic the brain’s ability to recognize complex objects and actions in layers. These model builders are computation intensive and can be hosted on any cloud-based Virtual Machine (VM) that has a powerful enough Central Processing Unit (CPU) and Graphics Processing Unit (GPU).

Finally, metadata discovery tools need to be augmented by advanced search and filtering capabilities to deliver meaningful inference of content.


While the business benefits that AI/ML can deliver are vast, M&E companies should not join the AI/ML bandwagon just because other industry players are. If the endpoint is not a business imperative, the AI/ML initiatives become hobby projects for the technology shelf of the company. Such initiatives may sound cool on some executives’ CV, but do not serve any real purpose for the company.

A great starting point is working with a value consultant who can suggest practical ways to leverage AI for meeting business goals based on the company’s unique needs and challenges. A knowledgeable consultant well versed with the latest tools and techniques can help identify the pain points that can be solved by AI, and highlight the opportunities where AI can boost monetization significantly. The consultant should also be able to guide companies about the data and application software needed to tap the potential of AI most effectively.

Product Software

An AI tool becomes most effective if integrated with a software specially tailored for M&E operations. For example, integration of AI tools with a media asset management (MAM) software will eliminate the need to perform different content operations on multiple platforms. What M&E companies should look for is an end-to-end solution, like a Media ERP software, that embeds AI-assisted content discovery, and on top of it, gives additional layers of business intelligence to perform tasks like Quality Check (QC) and Subtitling more efficiently.

Such applications are hard to find in the market as these require a deep understanding of content operations, coupled with extensive experience in data science and machine learning technologies.

Tapping the potential of AI across the content lifecycle

Today, AI can be leveraged for myriad use cases across the content lifecycle – right from scripting to post-production to archival.

Use of AI/ML in different phases of content supply chain

Here are 5 strategic business use cases where AI & ML are going to have a disruptive impact:

1. Automated sub-titling and closed captioning

skeeze / Pixabay

Most production houses and broadcasters invest significant effort on Subtitling and Closed Captioning (CC) their content. This is done to improve access to the content, adhere to regulatory requirements and meet the specifications of VoD/OTT platforms. Typically, Subtitling and CC is done manually by a large number of freelancers and Localization vendors. AI-driven automation can help reduce this manual effort by as much as 60%, and cut costs by 50%.

Some of the tools that can be used in this space include automated Speech-to-Text solutions from Google, IBM, Microsoft and Amazon. Of course, these tools do have accuracy limitations (varying from 30% – 80%) depending on background noise, number of speakers, heavy accents, high context and emotion etc. However, the results are extremely promising and improvements are taking place rapidly.

2. AI Assisted promo creation for shows and movies


The creative process of promo creation can be assisted by machines that identify sections of interest within content, eliminating the need for an editor to watch the complete footage. Further, these cuts can also be assembled together by AI algorithms based on learnings from similar projects, and the application of chosen “promo grammar” (specific edit actions etc.).

Such interventions can help reduce the effort of promo creation by about 30%, thereby driving substantial cost savings while freeing up manpower to focus on other creative tasks.

3. Content compliance Check and rule bases edit


Compliance regulations for broadcast differ across countries. While many regulations are similar worldwide, often culture-specific nuances dictate different regulations for certain territories. For content suppliers, this involves identification of NSFW (Not Safe For Work) segments, nudity, violence, smoking, drug abuse, objectionable language, visible brands etc. The different edit actions needed to be taken on such content range from adding a disclaimer, to blurring a portion of the frame to inserting an audio beep.

Leveraging AI to identify content segments with compliance issues can reduce manual effort by 30%-50%, reducing operational costs significantly while providing scalability to handle peaks in volume.

4. Sports Highlights

Achieving scale and speed during live matches has been a challenge for sports producers and broadcasters for years. Today, AI and ML are throwing up promising prospects of creating sports highlight packages through automation in near real time, to help save tremendous manual effort as well as time. AI can also help identify brand imprints within a given match or series (like pin-pointing brands displayed in the clip when Messi scores a goal).

Mike Malpass/Freeimages

Several AI OEMs and smaller start-ups have already launched sports action and scene detection models. These, however, are still far from accurate. Also, each sports model has to be trained extensively with hundreds of hours of sports content that has been annotated manually. Yet, with the recent success achieved in this space in events like FIFA World Cup, advances are being made at lightning speed.

5. Contextual Ads based on Content

Today, ad engines display ads based in the context of user, content, time and location. AI can help create contextual ads effectively based on auto-cataloging of the content and by supplying relevant keywords at the opportune moment.

To achieve this, content can be cataloged with 3rd party cognition tools or in-house cataloging software. Next, ad engines and video players can leverage AI to deliver contextual ads based on the narrative of the content being displayed.. For instance, viewers watching a movie set in New York City might be presented an automated advertisement for discounted tickets to that destination.

It’s About Time to Act

AI and ML have arrived and hit the road already. M&E industry can start exploiting the technology to their advantages hear and right now! The investment required to get started is absolutely small, if not nil – thanks to the readymade tools and APIs that are available from 3rd party vendors.

If you are a content owner, review your archive of content – old and new – that could be gathering dust. Can you make them perform better for you, earn you new revenue stream and help you create a new asset called an “AI model”?

We believe the real value will come from discovering which business use case can deliver you maximum bang for your buck and having access to a tool that can seamlessly merge and AI-enable your existing media work-flows.

It’s time to start your AI/ML journey is NOW!

[This article was originally published in Prime Focus Technologies Blogs]