Physical and digital design come together to create a multidimensional augmented reality.
Architecture profoundly influences our lives, even when we are unaware of it. Every aspect of a building is designed to convey a message, promote a point of view, or even reinforce a brand. Augmented reality (AR) is one of many emerging technologies that designers can leverage to communicate a building’s message or intent. While the built environment has a significant impact on our physical experiences, it is the least fluid of our cultural and brand experiences. The life cycles of a building, its interior, and its brand must work in unison to provide a seamless experience. While the most fluid elements of a brand should be separate from architecture’s fixed elements, adding a digital layer can create much-needed flexibility. AR has the potential to connect people, revitalize existing spaces, and create dynamic experiences. As end users become increasingly tech-savvy, including AR in the design toolkit is more critical than ever.
The Rise of AR Technology
Although many think of AR as futuristic, it has actually been around for decades — since the 1960s. As it has evolved, so too have our interactions with it. We see it everywhere, from social media filters and games like Pokémon Go to commercial applications like the IKEA Place app, which allows consumers to virtually place items in their homes to try them out. The pandemic accelerated our use of AR technology as people sought new modes of socially distanced entertainment and communication, adding an extra layer of interactivity and information to our homes. With technology titans like Google, Apple, Nvidia, Microsoft, and Meta competing to provide the next notable virtual experience, software and hardware development have become more user-friendly and accessible for creators from diverse technology and design disciplines.
Kate O’Neill, founder and CEO of KO Insights, explores how data and emerging technologies are shaping the future of human experience, and advocates with “strategic optimism” for humanity’s future in an increasingly tech-driven and rapidly changing world. She believes that to understand the potential of AR, one must first understand the fundamentals of how it functions. O’Neill notes that low-tech AR has been around for a long time, but it hasn’t necessarily been defined and perceived as such. She uses the South Rim Grand Canyon scope as an example of “old school” AR, citing the movement of the scope from one groove to another which allows viewers to see specific locations within the canyons. The scope design adds a secondary layer of information that creates a multidimensional experience.
AR is most successful when it conveys a message — a notion that is already familiar within the field of architecture — when it acts as just one more element that can be leveraged to reinforce a project’s design intent. O’Neill explains that when we understand the meaning behind a design, we can use AR as a tool to reinforce the concept with an added multisensory, spatial layer. However, AR’s real potential in the built environment is still largely unknown. There seems to be an analogous relationship between the beginnings of the internet in the 1980s and AR today. As before, the arrival of new technology creates excitement, and with time and continued exploration, more opportunities for its application become apparent. O’Neill predicts that different kinds of experience creators will continue evolving the way AR is used in the built environment and will give us new ways to incorporate this technology into our physical spaces.
What we understand as AR today is really just the tip of the iceberg. Architects and designers must be intentional with AR to create meaningful experiences that feel rich and purposeful and that offer opportunities for people to connect with one another. Similarly, the belief that technology cannot create community is one that O’Neill does not share. “The idea of people being isolated by technology often ignores the fact that users are very connected to people on the other end of the technology,” says O’Neill. “It’s not a proximal space, yet they are very connected.”
AR in the Built Environment
As with any technology, AR can be a shiny new tool that some may be eager to adopt but that fails to add real value. Many applications fail to accomplish a simple goal while adding complexity that, in the end, actually deters users. AR travel apps, like Visit Houston AR and the Travel Texas web app (texastourism.arweb.app), provide on-site information on destinations through a smart phone. While the intent is to enhance the travel experience by offering the audience access to real-time information, one could argue that Google Maps currently provides faster and superior information for this purpose. Hardware technology for practical AR experiences is still in its infancy, and early adopters often take a considerable risk and endure criticism from naysayers during its deployment. As wearable tech develops, people will likely become more open to adopting AR applications.
The Bullock Texas State History Museum in Austin provides another example of how AR is being applied in a practical way, with digital content overlayed onto physical exhibition space. One exhibit tells the story of La Belle, a 17th-century French ship wrecked off the coast of Matagorda Bay. The museum’s director, Margaret Koch, shares that it all started in 2011 when the Texas A&M marine archaeology lab began conservation work on the ship’s timbers. “For our museum, we start with the interpretative message — what do we want visitors to be able to gain from the experience — and whether technology is the right medium to deliver that,” explains Koch. “It’s never about using the latest technology.”
The initial curatorial plan was to enclose the 300-year-old ship’s remains and reconstruct the two-thirds that were missing. However, because the vessel measured about 54 feet in length, this approach wasn’t viable dues to space limitations. Instead, Koch explains, AR was used to digitally reconstruct the missing portion so that museum visitors could see the ship’s original form without requiring the construction of additional physical space. In 2018, the museum installed two AR experiences displaying archaeological records and simulating sailing conditions on the open sea. The ship remains are in a three-story space surrounded by walkways with several AR stations for interaction. At ground level, a pivoting glass kiosk allows users to align the image on the screen over the ship’s remains to see how it looked in the past. It also allows users to simulate ship activities, such as raising the sails, firing cannons, and exploring below deck. The second AR experience, located throughout the surrounding walkways, takes the form of telescopes. The telescopes superimpose the image of the complete ship in its original condition, allowing museum guests to experience the vessel’s large sails from the third floor.
AR experiences can also be immersive. Joshuah Jest is an integrated media designer and video artist who also holds a degree in architecture from Rice University. He has been examining ways to integrate digital and interactive content into the constructed world by utilizing projection mapping and motion detection to create an incredibly rich immersive experience. Specifically designed to fit the structure, Jest’s content creates experiences not previously possible. Through his spatial installations, he has focused on creating simulated ‘campfire’ environments where people share an experience collectively instead of deploying the experience through the traditional handheld device, which can isolate users. The ideal experience combines the physical and digital worlds, creating a sense of connection and entertainment.
AR offers the flexibility to curate space to appeal to the ever-changing needs of clientele or to tailor a public-facing image with unique, artistic, and timely messages. The challenge with such applications, however, is in providing ever-changing content and in designing a building that supports this. Jest aims to streamline and automate the system to make it accessible to other artists. When asked about his early introduction to the augmented world, he said that early information was limited in the United States. (He put together the curriculum on AR for his master’s program at MIT and traveled to Germany to identify AR developers and other resources.) Jest prefers using AR with digital content that is integrated into the built environment because it puts the technology load onto the venue. From an architectural standpoint, it is inclusive to anyone who walks into the space. Says Jest, “As an artist, I’m a big proponent of building really cool digital campfires for us to gather around in groups and witness together, to judge together.” The practice of augmenting physical reality with digital content could accelerate if architects considered their building and space design as a framework for projection from the outset. What is critical, however, is that the space be successful as a stand-alone physical space, not just as space for projection mapping.
AR in the Architecture Profession
Fifteen years ago, the terms “AR” and “reality headset” were exclusive to the domain of computer science and engineering — and many were not sure if these technologies would ever be accessible to the masses. But with the launch of the smartphone, many developers saw the potential of these devices for widespread deployment of AR. While other industries are embracing these new technologies and changing users’ everyday experiences, the architecture profession has largely taken a back seat. Successfully incorporating AR into architecture must begin with knowledge sharing, and integrating this design tool into the curricula of architecture schools more broadly is a critical step toward accomplishing this.
Dr. Tony Liao, founding director of the CougAR LAB at the University of Houston, has increasingly seen more interest in this technology from architecture students even though it is not a degree requirement. The lab studies the social and psychological effects of AR, ways that it might be applied to various industries, and how it changes people’s perception of buildings in the physical environment. Dr. Liao notes that the idea for CougAR Lab was not only to teach students to create augmented environments, but also to study this emerging phenomenon. Their research team is looking beyond AR’s entertainment value to study how humans connect, interact, and experience emotions through this technology.
The modern era of AR has illuminated more pathways for implementation. Whether through a handheld device, portable projection, or a wearable headset, these avenues provide distinct experiences, each with unique spatial requirements: For instance, an entire room of projectors may be used in a learning environment to create different scenarios for first responder training. Enterprise meetings or collaborative sessions may employ a tabletop projection. In an edutainment environment like a museum, a combination of applications enables people to simultaneously interact with the same artifact or model.
According to Liao, one of the first promises of AR was that it allowed users to change what they saw and how the world looked, which was a radical shift in our understanding of the world. To fully grasp its complexity as well as its potential, it is important to recognize that AR is not a single technology but rather a convergence of existing technologies that merges the real and virtual worlds using 3D, interactive, and real-time capabilities. The list of AR applications is vast, with experiences ranging from just a simple phone filter overlay to a complicated system that uses visual simultaneous localization and mapping (SLAM) integrated with geolocational and sub-meter accuracy. With the latter, the result is an augmented reality that moves interactively with the physical world in real time.
But the barriers to adoption are real. Architects and designers accustomed to software such as Revit, Adobe Creative Suite, and other 3D modeling programs may face a learning curve as they explore these new tools. The prices of required equipment must come down and application reliability must rise before AR can be successfully incorporated into daily design workflows. As these technologies become more accessible, designers can begin to experiment, and coding may no longer be the primary requirement for visualizing ideas.
Even so, visualization in architecture has evolved rapidly, and whereas elevation drawings sufficed 20 years ago, now many clients want to experience the space through animations or immersive experiences via headsets — the 2D experience is no longer enough. “The future challenge for design work will not be so much about the software, but about finding the best modality to design in,” says Liao. “Do I go to my computer and do it, or do I put on my headset? Developers are working hard to make the technology more natural and intuitive.”
AR is already being applied to many building typologies, including education and healthcare facilities, museums, and even public spaces. Digital displays are providing museums and other exhibition spaces with greater ease and flexibility for updating content than is afforded by a traditional building or physical displays. The use of AR in public spaces is providing a greater sense of discovery to users. For example, Artscape, a street art festival in Sweden, uses buildings as the canvas for artists to display their art, the buildings’ exterior walls coming to life through visitors’ phones. Artists have also used AR as a means of passive protest, as in Seattle, where images of racial injustice overlayed on Kane Hall at the University of Washington could be seen through a smart phone. Retail spaces also can create dynamic displays, as with one seen at New York City’s PacSun store featuring the experience of a wave crashing, among others. In 2021, the city of Amsterdam even went so far as to create AR fireworks experiences once real fireworks were banned.
As AR and other digital technologies become more accessible and their application within architecture better defined, we designers must evolve our own thinking about how we can use them to communicate our messages in stronger, bolder, and smarter ways. Design innovation has the potential to transform lives through intelligent, engaging human experiences that connect people. Architects can help lead the charge in determining how AR technologies impact our built environment and our lives — but are we ready? We will soon find out.
Adry Suryadi is a lead graphic designer at Kirksey Architecture. Michelle Giuseppetti-Old, AIA, is an architect at Kirksey Architecture.