Building On-Device Emotion Recognition Software for smileML
Project overview: privacy-first emotion recognition AI
smileML uses emotion recognition to help people understand each other in video calls and face-to-face meetings. Maxiom built the product and staffed the team: on-device emotion recognition software that never sends video or images to the cloud, plus SaleSpot, a sales call analytics tool that reads sentiment and body language.
About smileML
smileML uses emotion recognition to help people understand each other in video calls and face-to-face meetings. The product reads facial expression and body language so a conversation is easier to follow in the moment.
The challenge: emotion recognition that can leave the cloud
Most emotion recognition sends video or images to a cloud service, runs the model there, and sends the results back. For many of the people smileML wanted to serve, that round trip is unacceptable. Security and privacy rules out sending meeting video to someone else's servers.
The product had to read emotion well and fast, on the device the person is already using, without the video ever leaving that device.
What Maxiom built for smileML
Maxiom built the smileML product and staffed the team behind it. The work had two parts.
On-device emotion recognition
The core of the product is emotion recognition that runs on the device. Video and images are analyzed locally and are never sent to the cloud. That makes the product usable for people and organizations that cannot let meeting video leave their own hardware, and it means feedback during a live conversation does not wait on a server.
SaleSpot: sales call analysis from sentiment and body language
SaleSpot is smileML's sales analytics tool. It analyzes sentiment and body language during video calls so sales teams can see how a customer reacted, not only what was said, and improve how they communicate.
The team Maxiom staffed
Alongside the product, Maxiom staffed the engineering team. An AI startup gets people who can ship the product, and it can start computer vision and machine learning work without a long hiring cycle first.
How the on-device product is built
On a laptop or a phone, the model has to be small and fast, work with whatever camera is there, and leave the rest of the app responsive. smileML's emotion recognition is built for that:
- Edge-based: processing happens on the device, including in sensitive environments.
- Camera agnostic: built to run on any camera.
- Real-time: analysis runs live, while the conversation is happening.
- Lightweight: a small model built to run on everyday hardware.
- Secure: no video is saved or stored.
- Continuous learning: model updates are delivered straight to the device.
The engineering stack includes Python and TensorFlow for the computer vision and machine learning work.
Building privacy-first AI
If the model works in a notebook and still has to prove itself on real devices, start with an AI proof of concept on accuracy and speed. If the approach is proven and you need it in users' hands, that is AI MVP development. To add computer vision to an app you already have, see AI app development. To staff the team, you can hire computer vision engineers through Maxiom.dev.
Building emotion recognition or computer vision software?
Tell us what the model needs to do, where it has to run, and what data cannot leave the device. On a 30-minute call we will tell you whether we are the right team.
Related
Project inquiry
Tell us what you are trying to ship
Response within 1 business day
The Challenge
Our Solution
Technologies Used
Services Provided
Why Maxiom?
- Software since 2002
- $100M+ in contract value
- 221+ clients
- Top-rated on Clutch
Frequently asked questions
What is emotion recognition software?
Emotion recognition software uses computer vision and machine learning to estimate how a person is feeling from cues like facial expressions and body language. smileML uses it to help people understand each other in video calls and face-to-face meetings.
What is on-device emotion recognition?
It is emotion recognition that runs on the user's own device, such as a laptop, phone, or camera, instead of on a cloud server. The video is analyzed locally, so it never has to leave the device.
Why does smileML keep video on the device?
Many of the people and organizations smileML serves cannot send meeting video to the cloud for security and privacy reasons. On-device analysis also means live feedback does not wait on a round trip to a server.
How does SaleSpot analyze a sales call?
SaleSpot uses computer vision to read sentiment and body language from the video during a sales call. Sales teams see how customers reacted, not only what was said, so they can improve how they communicate.
What did Maxiom build for smileML?
Maxiom built the smileML product and staffed the team behind it. That includes on-device emotion recognition that never sends video or images to the cloud, and SaleSpot, smileML's sales analytics tool for video calls.
What tech stack does smileML's emotion recognition use?
The stack includes Python and TensorFlow for the computer vision and machine learning work. The models run on the user's device, so video and images are never sent off the device.
Can Maxiom build emotion recognition or computer vision software?
Yes. We build AI and computer vision products, from a first AI proof of concept to an AI MVP, and we can staff the team through Maxiom.dev.
How do engagements start?
With a 30-minute call about what the model has to do, where it has to run, and what data cannot leave the device. After that we send a written scope.