Augmented reality


Augmented reality, also known as mixed reality, is a form of 3D human–computer interaction that overlays real-time 3D-rendered computer graphics into the real world through a display, such as a handheld device or head-mounted display. This experience is seamlessly interwoven with the physical world such that it is perceived as an immersive aspect of the real environment. In this way, augmented reality alters one's ongoing perception of a real-world environment, compared to virtual reality, which aims to completely replace the user's real-world environment with a simulated one. Augmented reality is typically visual, but can span multiple sensory modalities, including auditory, haptic, and somatosensory.
The earliest functional AR systems that provided immersive mixed reality experiences for users were invented in the early 1990s, starting with the Virtual Fixtures system developed at the U.S. Air Force's Armstrong Laboratory in 1992. Commercial augmented reality experiences were first introduced in entertainment and gaming businesses. Subsequently, augmented reality applications have spanned industries such as education, communications, medicine, and entertainment.
Augmented reality frameworks include ARKit and ARCore. Commercial augmented reality headsets include the Magic Leap 1 and HoloLens. A number of companies have promoted the concept of smartglasses that have augmented reality capability.
Augmented reality refers to experiences that are artificial and that add to the already existing reality. In AR, information about the environment and its objects can be overlaid on the real world. This information can be virtual or real, e.g. seeing other real sensed or measured information such as electromagnetic radio waves overlaid in exact alignment with where they actually are in space. Augmented reality also has a lot of potential in the gathering and sharing of tacit knowledge.
Augmented reality can be defined as a system that incorporates three basic features: a combination of real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects. The overlaid sensory information can be constructive, or destructive.

Hardware and displays

AR visuals appear on handheld devices or head-mounted displays. Systems pair a display with sensors to register virtual content to the environment; research also explores near-eye optics, projection-based AR, and experimental concepts such as contact-lens or retinal-scanned displays.

Head-mounted displays

AR HMDs place virtual imagery in the user's view using optical see-through or video passthrough and track head motion for stable registration.

Handheld

Phone and tablet AR uses the rear camera plus on-device SLAM/VIO for tracking.

Projection mapping

Projectors overlay graphics onto real objects/environments without head-worn displays.

AR glasses

Glasses-style near-eye displays aim for lighter, hands-free AR; approaches vary in optics, tracking, and power.

3D tracking

AR systems estimate device pose and scene geometry so virtual graphics stay aligned with the real world. Common approaches include visual–inertial odometry and SLAM for markerless tracking, and fiducial markers when known patterns are available; image registration and depth cues maintain realism.

Software and standards

AR runtimes provide sensing, tracking, and rendering pipelines; mobile platforms expose SDKs with camera access and spatial tracking. Interchange/geospatial formats such as ARML standardize anchors and content.

Interaction and input

Input commonly combines head/gaze with touch, controllers, voice, or hand tracking; audio and haptics can reduce visual load. Human-factors studies report performance benefits but also workload and safety trade-offs depending on task and context.

Design considerations

Key usability factors include stable registration, legible contrast under varied lighting, and low motion-to-photon latency. Visual design often uses depth cues to support spatial judgment; safety-critical uses emphasize glanceable prompts and minimal interaction.

Comparison with mixed reality/virtual reality

Augmented reality is largely synonymous with mixed reality. There is also overlap in terminology with extended reality and computer-mediated reality. However, In the 2020s, the differences between AR and MR began to be emphasized.
In augmented reality, users are not only able to view digital content within their real environment but can also interact with it as if it were a tangible part of the physical world. This is made possible through devices such as Meta Quest 3S and Apple Vision Pro, which utilize multiple cameras and sensors to enable real-time interaction between virtual and physical elements. Mixed reality that incorporates haptics has sometimes been referred to as visuo-haptic mixed reality.
In virtual reality, the users' perception is completely computer-generated, whereas with augmented reality, it is partially generated and partially from the real world. For example, in architecture, VR can be used to create a walk-through simulation of the inside of a new building; and AR can be used to show a building's structures and systems super-imposed on a real-life view. Another example is through the use of utility applications. Some AR applications, such as Augment, enable users to apply digital objects into real environments, allowing businesses to use augmented reality devices as a way to preview their products in the real world. Similarly, it can also be used to demo what products may look like in an environment for customers, as demonstrated by companies such as Mountain Equipment Co-op or Lowe's who use augmented reality to allow customers to preview what their products might look like at home.
Augmented reality differs from virtual reality in the sense that in AR, the surrounding environment is real and AR is just adding virtual objects to the real environment. On the other hand, in VR, the surrounding environment is completely virtual and computer generated. A demonstration of how AR layers objects onto the real world can be seen with augmented reality games. WallaMe is an augmented reality game application that allows users to hide messages in real environments, utilizing geolocation technology in order to enable users to hide messages wherever they may wish in the world.
The use of the terms "mixed reality" and "interreality" is clearly defined in the context of physics and may be slightly different in other fields, however, it is generally seen as, "bridging the physical and virtual world".
Recent improvements in AR and VR headsets have made the display quality, field of view, and motion tracking more accurate, which makes augmented experiences more immersive. Improvements in sensor calibration, lightweight optics, and wireless connectivity have also made it easier for users to move around and be comfortable.

History

Precursors to augmented reality

  • 1901: Author L. Frank Baum, in his science-fiction novel The Master Key, first mentions the idea of an electronic display/spectacles that overlays data onto real life. It is named a 'character marker'.
  • Heads-up displays, a precursor technology to augmented reality, were first developed for pilots in the 1950s, projecting simple flight data into their line of sight, thereby enabling them to keep their "heads up" and not look down at the instruments. It is a transparent display.

    Earliest developments

  • 1968: Ivan Sutherland creates the first optical-see through head-mounted display that has graphics rendered by a computer.
  • 1975: Myron Krueger creates Videoplace to allow users to interact with virtual objects.
  • 1980: The research by Gavan Lintern of the University of Illinois is the first published work to show the value of a heads up display for teaching real-world flight skills.
  • 1980: Steve Mann creates the first wearable computer, a computer vision system with text and graphical overlays on a photographically mediated scene.
  • 1986: Within IBM, Ron Feigenblatt describes the most widely experienced form of AR today, use of a small, "smart" flat panel display positioned and oriented by hand.
  • 1987: Douglas George and Robert Morris create a working prototype of an astronomical telescope-based "heads-up display" system which superimposed in the telescope eyepiece, over the actual sky images, multi-intensity star, and celestial body images, and other relevant information.
  • 1990: The term augmented reality is attributed to Thomas P. Caudell, a former Boeing researcher.
  • 1992: Louis Rosenberg developed one of the first functioning AR systems, called Virtual Fixtures, at the United States Air Force Research Laboratory—Armstrong, that demonstrated benefit to human perception.
  • 1992: Steven Feiner, Blair MacIntyre and Doree Seligmann present an early paper on an AR system prototype, KARMA, at the Graphics Interface conference.
  • 1993: Mike Abernathy, et al., report the first use of augmented reality in identifying space debris using Rockwell WorldView by overlaying satellite geographic trajectories on live telescope video.
  • 1993: A widely cited version of the paper above is published in Communications of the ACM – Special issue on computer augmented environments, edited by Pierre Wellner, Wendy Mackay, and Rich Gold.
  • 1995 - Augmented reality was described as a key technology in the reality-virtuality continuum.
  • 1995: S. Ravela et al. at University of Massachusetts introduce a vision-based system using monocular cameras to track objects across views for augmented reality.
  • 2004: An outdoor helmet-mounted AR system was demonstrated by Trimble Navigation and the Human Interface Technology Laboratory.

    Smartphone AR and modern headsets

  • 2009: ARToolkit was ported to Adobe Flash by Saqoosha, bringing augmented reality to the web browser.
  • 2015: Microsoft announced the HoloLens augmented reality headset, which uses various sensors and a processing unit to display virtual imagery over the real world.
  • 2016: Niantic released Pokémon Go for iOS and Android in July 2016. The game quickly became one of the most popular smartphone applications and in turn spikes the popularity of augmented reality games.
  • 2018: Magic Leap launched the Magic Leap One augmented reality headset. Leap Motion announced the Project North Star augmented reality headset, and later released it under an open source license.
  • 2019: Microsoft announced HoloLens 2 with significant improvements in terms of field of view and ergonomics.
  • 2022: Magic Leap launched the Magic Leap 2 headset.
  • 2023: Meta Quest 3, a mixed reality VR headset was developed by Reality Labs, a division of Meta Platforms. In the same year, Apple Vision Pro was released.
  • 2024: Meta Platforms revealed the Orion AR glasses prototype.