Glovatrix is ​​building artificial intelligence gloves to help speech-impaired people speak again

Since the global explosion of generative artificial intelligence in 2022, the technology has become a topic of conversation. Not only artificial intelligence based on large language models (LLM), but all different branches of the technology have seen exponential growth in development and application. Artificial intelligence has also moved out of the digital realm and entered the real world, helping people improve their lives. From language translation tools to early detection of disease, it has impacted every area of ​​society.

Glovatrix is ​​an Indian startup based in Pune that also aims to solve similar pain points for those suffering from speech and hearing impairments.according to a Report According to the World Health Organization (WHO), 1.5 billion people worldwide suffer from some degree of hearing impairment. While the organization does not keep data on such speech disorders, there are likely millions. In many cases, these individuals rely on written communication or sign language to communicate. The former can make communication very slow, while the latter requires the listener to have access to an interpreter to translate sign language into spoken language.

This situation often makes it difficult to find a job, especially jobs that are client-facing or require a lot of communication. But that’s where companies like Glovatrix come into play. The Indian startup is developing an artificial intelligence device called Fifth Sense that can convert gestures from sign language to speech in almost real time, helping people with speech disorders communicate effectively and stress-free. assist. We at Gadgets 360 spoke with the startup’s co-founders to learn more about the product and the technology behind it.

Glovatrix, an Indian artificial intelligence startup

Glovatrix was founded in 2021 by co-founders Aishwarya Karnataki and Parikshit Sohoni, who both serve as co-CEOs of the company. Karnataki met a child with hearing loss in 2009 who inspired her to learn sign language, and their interactions made a difference for those suffering a similar fate. The seeds of Fifth Sense blossomed when she met fellow co-founder Sohoni. Sohoni, a data scientist with extensive experience in predictive analytics, had experienced these struggles in his own family and could immediately relate to the cause.

The two have been working with an engineer in a shared office space in Pune’s Pana, with a team size of three. Explaining her vision, Karnataki said: “Our vision is to facilitate seamless communication between people of all abilities, giving every deaf and speech-impaired person access to their natural language (sign language) Express and be heard.”

Fifth Sense, artificial intelligence gloves
Photo credit: Glovatrix

Fifth Sense, an artificial intelligence glove designed for people with speech disabilities

From a shape perspective, the artificial intelligence device looks like a glove with a smartwatch on top. Sohoni told us that the AI ​​gloves are made of lightweight fabric and can be worn continuously for 6-8 hours without discomfort. The top of the glove has an opening that allows the fingers to extend, allowing the user to use a smartphone or any other task that requires a better grip. The fabric is equipped with a smartwatch and multiple sensors that can capture any gesture. The fabric itself is removable and can be washed separately.

When it comes to hardware, Sohoni explained that the company sources the components used in its devices from different countries and then manufactures them individually based on the company’s in-house designs. This is the standard model adopted by most wearables companies operating in India.

But Glovatrix’s innovation lies in its software. There are two parts of the system that enable seamless two-way communication. The first is the device itself, powered by artificial intelligence, and the second is the companion app. “The AI ​​architecture was developed completely in-house because there were no reference materials for us to look at,” Sohoni said. Interestingly, Glovatrix does not use generative AI, but instead uses a mix of machine learning and different types of analytics algorithms. Its gesture-to-speech interface.

When a gesture is made, the companion app converts it to audio and plays it back to the listener. It can also act as a receiver when a hearing-impaired person must listen to a speaker without sign language. The app listens to sounds and then converts them into text for the user to read. Interestingly, the app can pick up not only speech, but also other sounds, such as doorbell chimes. The company also claims that the device works in almost real-time, allowing for smooth communication.

How Glovatrix solves privacy and connectivity challenges

Common issues with such smart devices are connectivity and privacy. Most smart devices, especially those using artificial intelligence, perform computing and processing on servers. This means a fast-running internet is crucial for a lag-free experience. Likewise, smart devices need to collect large amounts of user data to provide their functionality. Storing this data on a server also raises privacy concerns in case of a breach.

Glovatrix has found solutions to both problems. The entire receiver part of the application is done on the device, meaning any user-side audio collected and converted to text never leaves the device. This part is also lag-free as it does not require an active internet connection. In terms of gesture-to-speech, Sohony said some important words and individual letters will also be added to the app itself to eliminate connectivity issues. However, since AI models do require powerful computing processing power, the rest will be done in the cloud, which requires a stable internet connection. Notably, the company’s cloud is also built locally, which should help optimize server device connectivity for Indian users.

It is important to note here that while the AI ​​has been trained in Indian Sign Language and converts text in Hindi and Marathi, the in-app text translation tool will be able to generate audio in English and most Indian regional languages.

Find product-market fit

Fifth Sense is currently a prototype, and Sohony revealed that the company will soon begin its first pilot tests. The startup is also confident it can find product-market fit within the next six months.

Although the product is not yet ready for market, Glovatrix is ​​already seeing the impact of its device. A man with a speech impediment was reportedly able to land a job after using his fifth sense to communicate with interviewers.

How expensive is this product? Sohony said the vision for the AI ​​gloves is to keep prices competitive so they can benefit the masses. While he didn’t reveal a specific price point, he said it could cost as much as a mid-range smartphone. Additionally, to lower prices, Glovatrix is ​​also working on a subscription-based revenue model, which could further reduce the price burden on end consumers.