An AI solution that seamlessly swaps a character’s face with the user’s
This case study showcases our contribution to creating a solution that replaces a character's face with the user's face in a short, comical video.
Entertainment
Industry
25 specialists
Team size
Sydney, Australia
Location
2022 – 2023
Project duration
[ Business Context ]
How the client came to us with the concept for face animation software
The client approached us with a specific and entertaining requirement: they needed a software solution to replace a character's face in a short humorous video with the user's face.
Additionally, they desired basic animation to enhance the comedic effect.
[ Product scope ]
The Onix team was responsible for:
Gathering requirements to define objectives and project scope
Ensuring an effective and transparent development process
Creating an efficient and accurate face detection and replacement system within video streaming contexts
Developing scripts to determine the coordinates of the face oval
Creating an algorithm for determining characteristic points on a cropped face (conditional geometric center and points of the mouth corners)
Adding the ability to work with various accessories (hats, glasses, wigs, etc.)
Implementing reliable and advanced technologies to ensure high software operation and proper functioning
Have a project in mind?
Let's talk!
The Onix team developed
a dependable solution that serves
the interests of six distinct user groups:
General consumers
Any individual who wishes to create a personalized video for entertainment or to share with friends and family.
Marketers and advertisers
Those who wish to create engaging ad campaigns where the potential customer sees themselves in the ad, thus increasing the likelihood of the ad's message being remembered.
Social media influencers
Individuals looking to create engaging content for their followers to increase their reach and engagement.
Event organizers
Those organizing events (like birthdays, weddings, or corporate events) might use this software to create customized invitation videos or memorable highlights.
Video game development
Developers might use face replacement to allow players to insert their own faces onto game characters, enhancing the personalization and immersion of the gaming experience.
Educational institutions
Teachers might use face replacement to customize educational videos, making them more relatable to a specific audience or culture.
Solutions we provide
Inanimate object face replacement
Unlike human faces, animals or inanimate objects won't have the same geometric structure or landmarks. Placing a human face on them requires a dynamic approach that can adjust to various shapes and structures. Moreover, some objects or animals move unpredictably, which complicates the tracking and anchoring of the human face onto them.
The Onix team implemented a mapping algorithm that doesn't solely rely on traditional facial landmarks but also uses the broader contours and shape of the target entity, whether it's an animal's face or an object. Moreover, we used advanced tracking algorithms in OpenCV to keep the human face consistently placed on the moving object.
Accessory augmentation
As users move or change expressions, accessories should adjust accordingly. A hat, for instance, should remain on the head, not float above it if a user raises their eyebrows. Also, different users have different face sizes and shapes. The accessories must fit proportionally, looking both too large and too small.
We used facial landmarks to create "anchor points" for accessories. For instance, glasses anchor to the bridge of the nose and ears, ensuring they move with the face realistically. Also, we developed an algorithm that adjusts accessory size based on detected face dimensions. For instance, the width between the eyes can determine the size of glasses.
Performance efficiency
Regardless of where you make changes - on the mobile version or web, all modifications are automatically reflected across all platforms.
For example, if you change the color of a button, this change will be applied both on mobile devices and on the website. This ensures design consistency and ease of management, as you don't need to make separate changes on each platform.
All changes are automatically synchronized, allowing you to efficiently control the appearance and functionality of your product on different devices.
Business goals we helped our client to achieve:
Personalization
The solution empowers businesses to craft tailored content for their users, enhancing engagement.
Personalized content often results in better retention and conversion rates.
Engagement and interaction
Software fosters active participation and interaction by enabling users to feature in videos, resulting in a more immersive user experience.
Viral marketing potential
Incorporating personalized, humorous videos enhances the likelihood of them being shared on social media platforms, amplifying the reach of marketing campaigns.
Monetization
Businesses can introduce this as a premium feature or utilize it for advertising campaigns, potentially generating revenue.
Core technology stack we used
OpenCV,
Medipipe,
RetinaFace,
PyTorch,
Python,
FastAPI,
Elasticsearch,
React
Results
Thanks to the results of our collaboration, our client now has a powerful solution for replacing faces in video streams with high efficiency and accuracy.
The potential of using this AI-based solution spans various sectors, including entertainment, education, research, and forensics.
See other related projects
Kazakhstan
Mobile Communications
Centralized aggregator classifying news
into 10 predefined topics for 40K users
Optimizing news access with LSTM categorization
Centralized aggregator classifying news into 10 predefined topics for 40K users
Services provided:
Onix provided solution for streamlined news consumption, language detection, database development, intuitive UI, and services integration
Ukraine, Kyiv
Automatic Electric Drive
Image Analysis that uses unique
capabilities of the Intel Neural
Compute Stick 2
Software solution with real-time image analysis
Image Analysis that uses unique capabilities of the Intel Neural Compute Stick 2
Services provided:
We built software that offers real-time image classification and analysis, eliminating delays experienced with cloud-based solutions
USA, Florida
Beauty, Cosmetics
AI-powered iOS app that allows
consumers to easily find safe beauty
products
AI-powered iOS app for cosmetics & beauty industry
AI-powered iOS app that allows consumers to easily find safe beauty products
Services provided:
We developed an iOS skincare app with an intuitive admin panel, robust security, AI-driven recommendations, and a vast ingredient database powered by ML