With a pair of new APIs and low-latency media servers, Twilio's Programmable Video platform could soon help ARKit and ARCore app developers build shared AR experiences between multiple users.
By synchronizing AR content with audio and video communications in real time, Twilio can enable architects, doctors, and other professionals to collaborate remotely while working with 3D models. Apps can generate real-time subtitles translating audio content, or sync voice data with virtual avatars for gaming and telepresence purposes.
The main use case that Twilio envisions for developers, though, is in remote technical support.
"Imagine if the next time you called your cable company, you could simply show them a video of the flashing lights on your cable box and they could instantly know what's wrong, and then guide you through the process of wiring it up correctly by overlaying the correct cabling on-screen in your physical space. The prospects for improving remote support and sales conversations is incredible," said Rob Brazier, director of product for Twilio's Programmable Video platform, in a statement provided to Next Reality.
The remote AR support field is crowded, though. ScopeAR recently updated their platform with ARKit support. RE'FLEKT just added live video to their enterprise solution. Vuforia even has their own ARKit app that does the same for everyone below the enterprise level, with support for more devices on the way.
Now, other companies looking to compete in this arena can just plug in Twilio's services. Streem, makers of an eponymous ARKit app for home services, is one such company.
"With first-class support and collaboration from the very beginning, Twilio is instrumental to Streem's success, and they will play a large role as we continue to quickly scale and change the future of communication within augmented reality environments," said Sean Adkinson, CTO and co-founder of Streem.
Based on the WebRTC standard for real-time communications, the Programmable Video platform gives developers the power to put real-time communications in any web or mobile app, with SDKs for iOS, Android, and JavaScript available. The new multi-user AR features, though, are achieved through a pair of APIs and cloud infrastructure.
The DataTrack API shares metadata for AR objects, audio, video, and more between users over the same communications channel. The Media Sync API, which is not yet available, synchronizes the AR metadata with video and audio, so the experience is accurate on at all points. Media servers give developers client-side control over audio and video layout and spatialization to minimize latency.
Sample multi-user AR applications and more information for developers, including a sign-up form for the Media Sync API, are available through Twilio's website.
Just updated your iPhone? You'll find new emoji, enhanced security, podcast transcripts, Apple Cash virtual numbers, and other useful features. There are even new additions hidden within Safari. Find out what's new and changed on your iPhone with the iOS 17.4 update.
Be the First to Comment
Share Your Thoughts