This is how Extender is creating immersive AR experiences

When did you first hear the term Augmented Reality (AR)? If you’re new to the tech scene, your first encounter with this visual interactive technology was probably the Apple Vision Pro or Meta Quest headsets. However, gamers may have probably first interacted with the technology in the mobile game Pokemon Go, which was released in 2016. Movie buffs can also reference the 2008 Iron Man film, where Jarvis gave a glimpse of what the future of AR could hold.

Interestingly, this technology originated before all these modern milestones. The first AR device was created in 1968 when American computer scientist Ivan Sutherland invented a head-mounted display that could place two-dimensional images in a 3D environment. It was a very heavy apparatus that was suspended from the ceiling and only rudimentary wireframes of the room were visible. However, that invention became the cornerstone for AR in the decades to come.

What is Augmented Reality (AR)

One of the most interesting technologies of the modern world, AR is a conceptually straightforward mechanism. It creates a 2D or 3D overlay of digital information on top of a real-world environment. It essentially shows you information, objects, animations or an entire visual scene in the middle of your environment. For example, the Pokémon Go game can place a moving Pokémon on top of your table or on the floor as long as you point your camera at it.

Apple Vision Pro and Meta Quest have further advanced the capabilities of AR. While viewing objects or a movie with this effect is a nice novelty experience, true immersion comes when the user is not only able to see the digital information in the real world but is also able to interact with it. And mixed reality headsets allow exactly that. People using these headsets can watch a movie, play a game, or use an app on what is essentially a floating screen in the middle of your room. It can be increased or decreased depending on your preference.

The growing understanding of augmented reality and the development of hardware infrastructure to support such technology today has ensured that these experiences are not limited to exclusive apps or high-end devices, but have the potential to be experienced in platformless environments. are present all around us. Today, many Indian companies are creating AR content that can be experienced with a click of a button on a website or directly on a smartphone by scanning a QR code – without the need to purchase or download anything.

Some of the leading names in this field include Imaginet, Parallax Labs, Playshifu, Extender, and more. Here’s an example to put it in context of what they’re actually building. If you’re a Maggi fan, you might remember that the company recently released its Korean BBQ noodles, where the package had a visible QR code. Scanning it will open an AR session, allowing you to cook noodles in a 3D interactive space. The experience acted as a ‘how-to’ for those unfamiliar with the product and allowed people to play mini-games without downloading or installing anything.

This particular platform-less AR experience was created by Xtendr, and there is a lot of nuance behind creating such experiences. To understand the technology behind such AR technology and its potential, we spoke to Anurag Sachdeva, Founder and CEO of the company at Gadgets 360.

Extender enters platformless AR

The vision of building an AR-focused company is not a common one. But for Sachdeva, the path was paved by his vast experience of 23 years with tech-focused companies. Before trying his hand at entrepreneurship, he was leading Rovio Entertainment’s Angry Birds business for India, Southeast Asia and the Middle East. The company did a lot of AR engagement; However, it was all powered through apps.

Being a marketing professional, Sachdeva wanted to take advantage of the engagement that comes with an immersive AR experience, but without the limitations that an app brings. That idea led to the founding of Xtendr in May 2020. However, the path was not without challenges. Starting a company in the middle of a global pandemic meant making necessary adjustments to remain operational.

“We were the first in the field to create AR experiences in a non-Apple way. In the early days, one concern was to understand our target demographic at a time when people were also dealing with the COVID-19 pandemic,” Sachdeva said. Despite initial hiccups, the startup was successful in onboarding the two brands and the founder claims to have seen great results in terms of engagement. This development gave Xtendr the speed it needed.

Today, Xtendr has worked with over 50 brands across India, the Middle East and Southeast Asia. Brands it has worked with include Red Bull, Coca-Cola, ISRO, Unilever, Nestle Maggi, ITC Classmate and others. But how do these ‘platform-less AR experiences’ actually work? We investigated in depth.

Breaking down Xtendr’s AR experiences

Before we proceed, let’s take a moment to understand what Xtendr does. The startup works with consumer brands and other organizations and creates branded content in the AR space. These could be anything from a little animation that plays on your bed to a fully interactive cricket game that you can play in your garden. Once it is activated via URL or QR code, it is loaded onto your device without the need to download the app. These experiences, also known as Web AR, run entirely on the cloud.

However, creating such experiences requires many components to work correctly. Latency should be low so that these experiences do not take too long to load on devices with poor internet connections. Experiences should be fully AR-supported, immersive and interactive.

Sachdeva told us that Extender uses spatial computing and artificial intelligence (AI) to create augmented reality. Immersion has been a main focus of the company, which aims to “enhance human vision”.

The actual creation process involves first finding an idea and creating a narrative-based story through characters, objects, and elements. Once the storyboard is complete, the technical heavy lifting begins on the backend. The content is then converted into a program that can be executed seamlessly on the end user’s device.

The company primarily uses JavaScript to write programs using the Three.js library. Web AR and VR frameworks such as A-Frame, SLAM (Simultaneous Localization and Mapping), and React VR are also used in development. It all moves to the cloud, where the frontend comes together, and the AR experience is ready to be delivered.

role of ai

Interestingly, the startup also incorporates generative AI into some of its AI experiences. The company uses large language models (LLM) for characters in cases when they want to give users a conversational experience. AI is also used to generate art and let users experience AI-generated visual media.

These tools are used to design a gamified experience where the user can talk to the chatbot about the brand or do quirky things like drawing the sky. AI models that Extender has worked with include Inworld, MidJourney, ChatGPT, Gemini, Dell-E, and more.

solution to latency problem

An important part of creating a platform-less AR experience is its distribution. Users would love to load their content in seconds, and ensuring that happens on budget to premium devices and poor to excellent internet speeds is a big challenge. This becomes important, especially when experiences include 3D characters, complex artwork, and interactive elements.

Xtendr has created gamified engagements in the past where users can play a game and challenge another player online or on a leaderboard, where multiple other people are also entering their scores. With a single AR experience reaching two million impressions, ensuring consistency becomes critical.

The startup uses Amazon Web Services (AWS) for its cloud-based needs, which solves part of the problem because the cloud computing giant powers some of the most traffic-driven platforms, like Netflix, Twitch, LinkedIn, Facebook, and others. Handles. , The second part is solved by optimizing 3D renders to ensure that no one is too heavy for any device. Sachdeva stressed the need for a balance between art and conversation, which contribute equally to engagement.

A look toward the future of augmented reality

For a technology that first came out 56 years ago, AR remains in its infancy due to the lack of technology that can support the vision. But over the past few years, it has really gone mainstream and become viable and cost-effective. This is why companies are experimenting more and more on it. Today, AR has gone from being a party trick to a utility tool. The digital impact on human vision is opening up new ways to be productive online while staying connected to the real world instead of glued to a screen.

Sachdeva believes that the future of AR is bright as all the important things required for the technology to evolve rapidly are finally coming together. “5G is going to become mainstream in the next few years, which will solve many latency problems. AR-enabled hardware, higher processing power, the emergence of generative AI and the entry of mixed reality headsets [by Apple and Meta]Everything is coming together to create the next highway for immersive media,” he said.