Posted on

AR, VR, and a Model for 3D in HTML

Tucked down somewhere in the Safari Technology Preview 161 release notes is a seemingly innocous line about support for a new HTML element and attribute:

Added support for <model src> and honor <source type> attributes (257518@main)

Anytime I see mention of some element I don’t recognize, my mind goes straight to Huh! New to me, but probably old news for everyone else. It’s poor posture, I know, as it could just as easily be:

  • Hmm, looks like some propriatary experiment.
  • Wow, a truly new thing!

Truth is, it’s sorta all three.

It’s an evolving concept

As in, the first somewhat official-sounding thing I found on <model> wasn’t in the W3C spec but in WebKit’s repo for explainers. All that’s in the README is a giant note from 2021 that “The <model> element has moved to the Immersive Web CG.” I was about to hop over but my eye caught the HistoryAndEvolution.md file which has a nice rundown of early context on the <model> concept:

The <model> element was born out of a desire to take the next step and improve the experience of Safari’s integration with iOS’s AR Quick Look feature.

I had to look at Apple’s splash page for AR Quick Look. You know the new feature that some stores have where you can transpose a 3D rendering of a product in your own home using your phone camera? That’s the sort of stuff we’re talking about, and Apple links up a nice case study from the Metropolitan Museum of Art.

As I understand it from this limited context:

  • Drop a <model> element in the document.
  • Add an external source file, e.g. <model src="assets/example.usdz">.

The original proposal is from the Immersive Web Committee Group

That’s the team looking make Virtual Reality (VR) and Augmented Reality (AR) part of the web. Apple linked up their repo, so I made the jump and went straight to the explainer. This isn’t the spec or anything, but the original proposal. A much better definition of the element!

HTML allows the display of many media types through elements such as <img>, <picture>, or <video>, but it does not provide a declarative manner to directly display 3D content. Embedding 3D content within a page is comparatively cumbersome and relies on scripting the <canvas> element. We believe it is time to put 3D models on equal footing with other, already supported, media types.

[…]

The HTML <model> element aims to allow a website to embed interactive 3D models as conveniently as any other visual media. Models are expected to be created by 3D authoring tools or generated dynamically, but served as a standalone resource by the server.

The basic example pulls this together. It really does feel like the <video> or <picture> elements:

<model style="width: 400px; height: 300px">
  <source src="assets/example.usdz" type="model/vnd.usdz+zip">
  <source src="assets/example.glb" type="model/gltf-binary">
</model>

.usdz? .glb? Not the type of files that typically cross my desk. Guess I’ll need to brush up on those and any other file types that <model> might support. Again, all of this is merely the original proposal.

The draft proposal isn’t stubbed out quite yet

But it does provide a nice outline of where things could possibly go:

  • Adding a model to a document
  • Enabling interactivity
  • Supporting multiple formats
  • Providing fallback content
  • Making it accessible

There’s a lot to figure out. Most of what’s there are documented issues that need addressing. It does, however, shed more light on <model> like proposed attributes that make it feel even more like <video> such as autoplay, controls, loop, muted, poster, etc.

It goes back even further

The very earliest mention of 3D modeling I found was Keith Clark’s 2018 post in which he prototypes a custom element called <x-model>. He describes it as “a placeholder that provides access to the DOM and CSSOM” where the loading and rendering is done in three.js.

Keith’s idea is followed by the <model-viewer> component Joe Medley shared in 2020 (and a subsequent update to it). There’s even a homepage for it and it’s fun to drag Neil Armstrong around in space.

It’s possibly just an experiment?

I mean, the draft spec hasn’t been fleshed out. Apple seems willing to play ball thanks to the Safari TP 161 announcement. That makes total sense given how bullish Apple is on AR as a whole. (Apple Glasses, anyone?)

Google seems to have its foot in the door, albeit on the Web Components side of things. It’s easy to see how there may be a conflict of interest between what Apple and Google want from AR on the web.

These are all just my notes from trying to grok everything. There’s gotta be a lot more nuance to it than what little I know about it so far. I’m sure someone smarter can tie neater bow around <model> in the comments. 😉

And while we’re talking Safari Technology Preview, 162 just released the other day and it enables CSS nesting and the CSS relative color syntax.

AR, VR, and a Model for 3D in HTML originally published on CSS-Tricks, which is part of the DigitalOcean family. You should get the newsletter.

Posted on

Collective #740

Building conscious design systems

Amy Hupe’s article based on her recent talk on how we can focus less on artefacts in design systems, and work consciously to build inclusive design systems that put people at their heart.

Read it

unjs

A great collection of useful JavaScript projects from the Nuxt team.

Check it out

Curations

A site that gives you interesting resources to learn and grow as a developer, designer or entrepreneur.

Check it out

Life After Tweets

Whether Twitter bites the big one or you decide you’ve simply had enough, Live After Tweets can help you stay in touch with the community you loved.

Check it out

Lumi

Lumi is an nano framework to convert your Python functions into a REST API without any extra headache.

Check it out

Futicons

Futicons are Figma designer icons for Virtual Reality, Augmented Reality, Interplanetary Travel, Artificial Intelligence, Crypto, Blockchain, Cyborg, Future Transportation and Sustainability.

Check it out

Wasp

With Wasp you can rapidly develop full-stack web apps in React and Node.js.

Check it out

Resource.fyi

Handpicked free tools, resources and products curated for developers, designers, marketers and more.

Check it out

Posted on

The Role of AR/VR in Transforming the Manufacturing Industry Digitally

With the emergence of the fourth industrial revolution, the concept of Industry 4.0 has been introduced. It is actually the digital transformation of the manufacturing industries by the use of advanced technology like robotics, the Internet of Things, cybersecurity, artificial intelligence, augmented reality (AR), and virtual reality (VR).  It is also called smart factories.

Out of these advanced techs, AR and VR are forming a bridge between the physical and imaginary world of manufacturing by forming a cyber-physical system. Augmented reality and virtual reality have revolutionized the manufacturing businesses from designing the product to its delivery in the real world. According to IDC, after 2021, worldwide spending on virtual and augmented reality is expected to double each year. 

Posted on

Virtual Reality and Augmented Reality in the Browser

Introduction

This article about Progressive web apps is about implementing Virtual Reality (VR) and Augmented Reality (AR) in your web app. For this, we use A-Frame and the WebXR Web API.

VR moves users to another world and lets them experience it as if they were themselves in this virtual world. AR, on the other hand, offers the possibility to blend the real and virtual worlds into one another. Both VR and AR offer unique opportunities for behavioral change, knowledge transfer, training, and research. For example, think of VR live streaming, social VR, and art. All this is now also possible in the browser!

Posted on

AR: Shaping the Next Decade of Fintech

Introduction

Augmented reality is already becoming more commonplace by the day. Via smartphones, AR is already widely accessible throughout the world of gaming, entertainment, and even finance-based consumer applications. In terms of business, cloud access means that AR users can operate seemingly free from hardware while applications enable businesses to link the data-driven power of the computer with human judgment and expertise. 

It’s this collaboration between rich data and human intervention that makes AR an ideal asset in the world of fintech. The sector is growing while augmented reality represents the next logical step in making finance run on brand new tech. 

Posted on

Augmented Reality Demo With Apache Kafka and Machine Learning

Augmented Reality (AR) and Virtual Reality (VR) get traction across industries far beyond gaming. Retail, manufacturing, transportation, healthcare, and other verticals leverage it more and more. This blog post explores a retail demo that integrates a cutting-edge augmented reality mobile shopping experience with the backend systems via the event streaming platform Apache Kafka.

Augmented Reality (AR) and Virtual Reality (VR)

Augmented reality (AR) is an interactive experience of a real-world environment where the objects in the real world are enhanced by computer-generated perceptual information. AR is a system that fulfills three basic features: a combination of real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects.

Posted on

How We Trained a Neural Network to Generate Shadows in a Photo: Part 2

In this series, Artem Nazarenko, Computer Vision Engineer at Everypixel shows you how you can implement the architecture of a neural network. In the first part, we were talking about the working principles of GAN and methods of collecting datasets for training. This part is about preparing for GAN training.

Loss Functions and Metrics

Attention. At this point, we deviate from the reference article. We take the loss function to solve the segmentation problem. Generation of attention maps (masks) can be considered as a classic image segmentation problem. We take Dice Loss as the loss function. It is well resilient to unbalanced data.

Posted on

Intelligent Vision: AR Contact Lenses Coming Soon?

Given the number of variables and lack of empirical data, there’s simply no way to measure the number of people who need to wear glasses or contact lenses in the world. For instance, some people only need visionary aid to read, some to drive, some experience short or long-sightedness and others may simply turn to glasses as a fashion accessory. Of those who use contact lenses as well as eyeglasses, over 62% wear contact lenses more often. 

Convenience, comfort, and flexibility of contact lenses are perhaps the most attributable factors to this high statistic. The small plastic lenses are incredibly portable, invisible, and negate the need to wear a whole frame on a user’s face. Contact lenses sit seamlessly on the curvature of the eye, giving the user a far wider field of view than glasses lenses - which often create blind spots and restrict vision. Contact lenses are typically less affected by adverse weather conditions such as fog and rain and are more suitable to wear when taking part in physical activity and competitive sports. 

Posted on

Top 5 Virtual and Augmented Reality Trends to Watch Out for in 2019

Virtual reality and augmented reality, along with AI and automation, are transforming the lives of people both at home and in the workplace.

VR, which is all about offering humans a completely computerized, digital world, and AR technology, which is about building computer graphics that can be viewed in a normal environment, have now been used and adapted at the industry level as well as for entertainment purposes. For example, in an online business, such as one selling furniture, the AR application helps shoppers visualize how a particular piece of furniture will look in their living room without having to actually buy it or go into a store.

Posted on

Web AR Allows Accessing Web-Based AR Through a Web Browser

You can easily access Web-based Augmented Reality through Web browsers without any need for apps through Web AR. It is a modern advance technology in Augmented Reality.

A step-by-step guide explaining how to create an augmented reality (AR) application within 10 minutes using the PlugXR platform. PlugXR’s cloud-based platform is equipped with essential features to create seamless Augmented Reality Apps & Experiences and publish it in WebAR or a white-labeled app on Android & iOS or in the PlugXR app with absolutely no coding or dependency.

Posted on

Chrome 81 Expands Support for Augmented Reality and NFC

The latest version of Google's Chrome browser packs in a bevy of new features for developers and users alike. The two big additions are support for augmented reality and hit-testing thanks to new APIs, as well as web NFC for mobile. As always, there are lots of smaller changes to the browser. Let's dig in and go over each. 

Posted on

Building an AR Game With Hololens, Part 2: Models and Assets

Enjoy this second installment on building an Hololens AR app. We focus on models and assets.

In this chapter, we will go through all the elements that you need to have in place to start building your game. I will use terminology that you should be familiar with, as well as software and tools that you will need to make the game. So we will be covering a lot of elements here. Once your environment is set up and you have a basic familiarity with Hololens/Unity development, we can really dig in.

You may also like: Building an AR Game With Hololens, Part 1: Introduction

Unlike many other series, I want to provide you with what you need so that you can jump into development. I will not be spending any amount of time on things like how to use the device or basic functionality. For that, I would encourage any reader to visit the Mixed Reality Acadamy on the Microsoft site. It is an excellent source of information that is designed to kick start your understanding and development.

Posted on

Meatspace Augmented Reality: From Chester to Nagoya

Check out the newest AR use cases disguised as history lessons!

I love Chester — it's a walled Roman fort town in the North West of England, with heaps of history. On a walk through the Roman Gardens, we came across a real-world Augmented Reality experience that described the breach of the walls in the time of the English Civil War.

Image of Roman Gardens and Civil War Breach AR experience

I thought it was rather neat: With the associated information on the board, it gave me a sense of what was happening in that period. The experience got me thinking about Augmented Reality on modern smartphones and headsets (I've been lucky enough to play with a Magic Leap) — the use cases we will demo with these devices are just not worth the cost of development right now.

Posted on

Splunk Connected Experiences: The Power of Splunk Wherever You Are

With an always-on life—and one where your systems are also ‘always on’—you need to be able to act and even interact with your data to run your business day-to-day. We’ve already heard and continue to hear from customers the need to follow or stay plugged into their KPIs when they’re away from their desks. In response, Splunk’s engineering teams are doubling down on building new capabilities in mobile and augmented reality (AR) to bring your data to where you are—in the palm of your hands (that’s where you want it, right?). Through Splunk Connected Experiences (whose capabilities include the Splunk Cloud Gateway, Splunk Mobile, Splunk TV, and Splunk AR), we're making these game-changing advances available to apply to your most pressing business challenges.

Within the manufacturing industry, the AR revolution has already begun its early stages. There are low-hanging fruit opportunities to use AR to impact productivity gains on the factory floor, assist with visibility into the real-time health of machine assets, and coordination of field logistics. And as we continue to hear the refrain of the challenges associated with collecting and merging data from different systems, that’s where Splunk’s original secret sauce comes in to make all these new capabilities a seamless addition to your analysis.

Posted on

How Android App Development Capabilities With AR Boost Your Business in 2019 and Beyond

2025 is the year where AR is estimated to be crucial in the business world. In this era of modern technology’s development, businesses must focus on implementing AR technology itself, as it is expected to expand up to $150 billion by 2020, according to Digi-Capital fundamental.

Augmented reality is the next big technology that delivers 3D high definition audio and video experience to its users. This technology uses markers and sensors to overlay digital environment in the real world that enable users with more intensely and vividly experiences.

Posted on

Google Announces Beta Testing of Augmented Reality Walking Directions in Maps App

As anyone who has used Google Maps’ walking navigation feature in a major city can attest, the app doesn’t always get it right. Relying on GPS technology to guide users from one precise location to another can be a rather tenuous endeavor in an urban environment, where visibility from the ground can be low and GPS signals are often interrupted by tall structures.

“We’re experimenting with a way to solve this problem using a technique we call global localization, which combines Visual Positioning Service (VPS), Street View, and machine learning to more accurately identify position and orientation. Using the smartphone camera as a sensor, this technology enables a more powerful and intuitive way to help people quickly determine which way to go,” the company said Monday on the Google AI Blog.