This page concerns the Vuforia Engine API version 9.8 and earlier. It has been deprecated and will no longer be actively updated.
This article presents the concept of continued experiences for paused AR applications built with Vuforia Engine 9.0 and later. Continued AR experiences are applications which can resume the AR experience after pause or interruption. You are now able to provide a convincing and coherent story to your application where the AR content is sustained from where the users left off.
Continued AR experiences work on all fusion providers and device platforms by maintaining anchors and Vuforia targets between pauses and application interrupts. It is required to enable the Device Tracker if you wish to have a continued AR experience. Additionally, familiarize yourself with Vuforia’s Device Tracking API Results as these states are essential for developing and specifying continued AR sessions.
Requirements for experiences continuity to function optimally
- Stable tracking – feature rich area
- Prior mapping of the surroundings
- Stable lighting settings
Definition of levels of interruptions of an AR experience
Limited Tracking – While it is not a temporal interruption of the application itself, tracking loss can be abrupt and easily break the immersion of an AR experience. It is one of the trickiest elements of working with AR; an application with poor tracking stability will not be adopted by the users. To solve this, Vuforia offers Extended Tracking via Device Tracking at runtime, which allows you to track a target even when the detection is limited, or the target is no longer in the camera frame.
Application interrupt – Pausing, push notifications or briefly changing applications are all examples of an application interrupt. The disruption from other applications would often require users to restart the AR session and repeat the progress that was made. To maintain the AR session, Vuforia records the position of targets, anchors and virtual content in the environment which will be ready once the users resume the AR session. For the continuity support to work optimally, it is recommended that users resume the AR session in roughly the same position that it was paused.
Application Relaunch – There will be cases where the application is shut down and resumed at a later point. Such continuation of a closed AR session and longer-term persistence is currently not supported by Vuforia.
Here’s how to create continuity in your app after resume
As mentioned above, it is the management of states that allows you to create continued AR experiences after application resume and getting familiar with the statuses is highly recommended. In addition, it also depends on the platform you are using. Below is a description of platform specific approaches for realizing the continued behavior:
Important: AR experience continuity will not work if you reset the Device Tracker or deinitialize Vuforia when the app is paused. In this case you will lose all the information about previously tracked targets. The correct approach to get continued behavior is to stop the Trackers on pause, and then start the Trackers on resume. For an overview of the lifecycle of a Vuforia app, please refer to lifecycle index.
ARCore, VISLAM and HoloLens
The behaviors are alike on these platforms. Upon recovering an AR session, the DeviceTracker will report as TRACKED-NORMAL. It is possible to start Extended Tracking of new targets before recovering the targets that was tracked before the interrupt. And, the new targets will not be affected when restoring tracking of the preceding targets.
ARKit
Upon resuming an interrupted AR session, the DeviceTracker will report a LIMITED-INITIALIZING status for 1-2 frames before it switches to LIMITED-RELOCALIZING status info. There will be no reports of extended tracked targets or anchors during this time. To return to a TRACKED-NORMAL status, developers should, therefore, inform users to return to the position where targets were previously tracked and the DeviceTracker will relocate itself.
NOTE: ARKit is not able to start extended tracking of new targets until it has recovered from the LIMITED-RELOCALIZING status and is tracking normally. For this reason, you may wish to provide users with a way to reset the device tracker manually, or to do so automatically yourselves if this status continues for some time. Consult the example for handling tracking status or this article about the Vuforia sample app set up.
NOTE: Resetting the device tracker will lose all the previously detected targets, and they will have to be redetected again. However, the reset will enable the system to start extended tracking of new targets. This applies only when you are using ARKit and Vision-only SLAM.
Vision-only SLAM
For devices that track only on visual cues will return a NO_POSE-RELOCALIZING status as the Vuforia Engine attempts on restoring tracking of targets between interrupts. To restore tracking to TRACKED-NORMAL users should return to a position where tracking worked with previously identified targets.
NOTE: The vision-only SLAM system will automatically reset itself if it detects a new target while it is in the NO_POSE-RELOCALIZING mode.
Re-establish Tracking
It should now be evident that, depending on the platform, certain statuses can be used to inform users how to restore tracking. It is best practice to indicate what users should do when they resume the AR session from a pause or interruption. These indicators can be messages or an on-screen overlay asking users
- to return to a position from the previous session, or
- to map the area by moving the device left-right, or front-back
until relocalization is successful.
You may provide a message to indicate a time interval until which relocalization can be attempted, and if that time interval is exceeded, the session will be restarted. Another option is to simply offer a reset button to the user to restart the experience after a time-period upon resume, if relocalization did not succeed.
Disabling the Continuation Behavior
It is possible to return to the application behavior prior SDK 9.0 by resetting the DeviceTracker whenever the application is resumed. This will break the continuity whenever the app is paused or interrupted. Similarly, all mapping stored by the DeviceTracker will be lost if you deinitialize Vuforia during a pause.
To learn more about continuity using ARKit, please visit the below external documentation on the subject:
https://developer.apple.com/documentation/arkit/managing_session_lifecycle_and_tracking_quality