Tencent has released a new artificial intelligence (AI) model called InstantMesh that can render 3D objects using still photos. The new AI model is an upgrade to the company’s older Instant3D framework and now uses a combination of multi-view diffusion models and sparse view reconstruction models based on the Large Reconstruction Model (LRM) architecture. Tencent has also open-sourced the InstantMesh model and provided a preview application for enthusiasts to test its functionality or generate and export 3D renderings.

The company releases a preprint of its study Paper on arXiv. It’s worth noting that arXiv does not conduct peer review, so it is difficult to say whether the model has been evaluated. However, the company has open sourced the AI ​​model on Hugging Face so that developers can test its efficiency.For enthusiasts, there is a application view They can also add a photo and watch it turn into a 3D rendering. We tested the platform at Gadgets 360 and found that renders were created in 10 seconds, as the company claims. However, the quality of the rendering feels rather low. An X (formerly known as X) user posted a video using the AI ​​model, and you can see the results below.

Talking about the technology behind the AI ​​model, the company uses two different architectures – multi-view diffusion model and LRM architecture. While the former helps process the input image and generate different dimensions not visible in the image, LRM builds orbital view objects that can be experienced in a 3D environment.

According to Tencent, InstantMesh solves the Janus problem in the field of 3D rendering. The Janus problem is a phenomenon in 3D rendering space where, because the model must “imagine” different sides of a reference object and create them, it creates multiple canonical views of the object rather than a cohesive 3D object. The company solved this problem by using a novel view generator fine-tuned for stable diffusion.

The research paper also shares benchmark scores compared to different existing models, including the recently launched Stability AI’s Stable Video 3D. Based on the scores, InstantMesh performs better than SV3D on Google Scanned Object (GSO) and OmniObject3D (Omni3D) orbit views. SV3D performed better in several parameters on the Omni3D benchmark that correspond to the resolution of the output, but Tencent said this was intentional. “We believe that perceptual quality is more important than fidelity, as the ‘true novelistic point of view’ should be unknown and open to multiple possibilities given a single image as a reference,” the company explains.