How to Use Cloud Recognition

This article will introduce you to the steps for creating a basic Vuforia Engine Cloud Recognition app in Unity and the native SDKs for Android, iOS, and UWP. Use Cloud Recognition in your application when you plan to track many targets or if your targets are frequently updated.

Sections

To learn more about the benefits, licensing, and use cases of Cloud Recognition, please refer to the Cloud Recognition article.

Accessing Sample Code

The sample that present the Cloud Recognition feature is available only for Unity at Vuforia Engine Developer Portal Samples. The Cloud Recognition for native SDKs only provide examples of the basic handling. The Cloud Recogniton Books sample for Unity, shows how to use the CloudRecoEventHandler.cs script that is attached to a CloudRecognition prefab. The OnNewSearchResult method shows how to get a targetSearchResult object, from which you can retrieve the metadata.

Implementing in Unity

The following steps explain how to create a cloud recognition app. The app reacts to a cloud target detection event, retrieves the metadata associated with the target, and displays the metadata text.

Prerequisites 

Instructions

  1. Create a new project in Unity.
  2. Create a license key for your project.
  3. Add the license key to your Vuforia Engine app.
  4. In the Hierarchy window, delete the Main Camera object.
  5. In the GameObject menu, select and add Vuforia Engine -> AR Camera. 
  6. In the GameObject menu, select Vuforia Engine -> Cloud Image -> Cloud Recognition. A new Cloud Recognition object is created.
  7. In the Inspector window of your Cloud Recognition GameObject, enter your cloud database client keys in the Access Key and Secret Key fields. For information about creating a cloud database, refer to the Cloud Databases article. 
  8. Create a C# script named SimpleCloudRecoEventHandler.cs
  9. Attach the SimpleCloudRecoEventHandler script to the Cloud Recognition GameObject.
  10. Implement the SimpleCloudRecoEventHandler script by referencing the following code samples:
  1. Implement the RegisterEventHandlers interface and register it with the CloudRecoBehaviour:
using UnityEngine;
using Vuforia;

public class SimpleCloudRecoEventHandler : MonoBehaviour
{
    private CloudRecoBehaviour mCloudRecoBehaviour;
    private bool mIsScanning = false;
    private string mTargetMetadata = "";
    
    // Register cloud reco callbacks
    void Awake()
    {
        mCloudRecoBehaviour = GetComponent<CloudRecoBehaviour>();
        mCloudRecoBehaviour.RegisterOnInitializedEventHandler(OnInitialized);
        mCloudRecoBehaviour.RegisterOnInitErrorEventHandler(OnInitError);
        mCloudRecoBehaviour.RegisterOnUpdateErrorEventHandler(OnUpdateError);
        mCloudRecoBehaviour.RegisterOnStateChangedEventHandler(OnStateChanged);
        mCloudRecoBehaviour.RegisterOnNewSearchResultEventHandler(OnNewSearchResult);
    }
    //Unregister cloud reco callbacks when the handler is destroyed
    void OnDestroy()
    {
        mCloudRecoBehaviour.UnregisterOnInitializedEventHandler(OnInitialized);
        mCloudRecoBehaviour.UnregisterOnInitErrorEventHandler(OnInitError);
        mCloudRecoBehaviour.UnregisterOnUpdateErrorEventHandler(OnUpdateError);
        mCloudRecoBehaviour.UnregisterOnStateChangedEventHandler(OnStateChanged);
        mCloudRecoBehaviour.UnregisterOnNewSearchResultEventHandler(OnNewSearchResult);
    }
}
  1. Implement the OnInitialized(), OnInitError() and OnUpdateError() methods for the interface:
  public void OnInitialized(TargetFinder targetFinder) {
      Debug.Log ("Cloud Reco initialized");
  }
  public void OnInitError(TargetFinder.InitState initError) {
      Debug.Log ("Cloud Reco init error " + initError.ToString());
  }
  public void OnUpdateError(TargetFinder.UpdateState updateError) {
      Debug.Log ("Cloud Reco update error " + updateError.ToString());
  }
  1. Implement the OnStateChanged() method of the interface to determine whether Vuforia Engine is scanning the cloud:
  public void OnStateChanged(bool scanning) {
  mIsScanning = scanning;
      if (scanning)
      {
      // clear all known trackables
      var tracker = TrackerManager.Instance.GetTracker<ObjectTracker>();
      tracker.GetTargetFinder<ImageTargetFinder>().ClearTrackables(false);
      }
  }
  1. Implement the OnNewSearchResult() method:
  // Here we handle a cloud target recognition event
  public void OnNewSearchResult(TargetFinder.TargetSearchResult targetSearchResult) {
  TargetFinder.CloudRecoSearchResult cloudRecoSearchResult = 
      (TargetFinder.CloudRecoSearchResult)targetSearchResult;
  // do something with the target metadata
  mTargetMetadata = cloudRecoSearchResult.MetaData;
  // stop the target finder (i.e. stop scanning the cloud)
  mCloudRecoBehaviour.CloudRecoEnabled = false;
 }
  1. Implement the OnGUI() method to display the current scanning state and the metadata of the last cloud target detected:
  void OnGUI() {
      // Display current 'scanning' status
      GUI.Box (new Rect(100,100,200,50), mIsScanning ? "Scanning" : "Not scanning");
      // Display metadata of latest detected cloud-target
      GUI.Box (new Rect(100,200,200,50), "Metadata: " + mTargetMetadata);
      // If not scanning, show button
      // so that user can restart cloud scanning
      if (!mIsScanning) {
          if (GUI.Button(new Rect(100,300,200,50), "Restart Scanning")) {
          // Restart TargetFinder
          mCloudRecoBehaviour.CloudRecoEnabled = true;
          }
      }
  }
  1. Save the current scene.
  2. Build and run the project.

Showing a 3D augmentation object

Perform the following steps to show a 3D augmentation object on top of your cloud target upon detection:

  1. In the GameObject menu, select Vuforia Engine > Cloud Image > Cloud Image Target to add a new Image Target object. 
  2. Right-click ImageTarget and select 3D Object > Cube to create a simple cube object. 
  3. Center and scale the cube object appropriately so that it fits nicely on top of the Image Target plane in the scene.
  4. Add directional lighting to the scene so that any augmentation appears shaded.
  5. Modify the SimpleCloudRecoEventHandler script by adding the following code to the OnNewSearchResult() method. This code programmatically instantiates an ImageTarget that corresponds to the one detected by the Cloud Recognition engine:
// Build augmentation based on target 
        if (ImageTargetTemplate) { 
            // enable the new result with the same ImageTargetBehaviour: 
            ObjectTracker tracker = TrackerManager.Instance.GetTracker<ObjectTracker>(); 
            tracker.GetTargetFinder<ImageTargetFinder>().EnableTracking(targetSearchResult, ImageTargetTemplate.gameObject); 
        }
  1. Add a public member variable named ImageTargetTemplate to the SimpleCloudRecoEventHandler script:
public ImageTargetBehaviour ImageTargetTemplate;
  1. Select the CloudRecognition object in the scene.
    The Inspector should display the Image Target Template field in the Simple Cloud Handler component.
  2. In the Hierarchy window, drag ImageTarget onto the Image Target Template in the Inspector window.
  3. Save the current scene.
  4. Build and run your project.

Implementing for native SDKs

The Cloud Recognition APIs is accessible from the ObjectTracker class as a service by calling getTargetFinder(). The Cloud Recognition Service is controlled with the TargetFinder class and detected recognition events are managed with the CloudRecoSearchResult class.
Differently from other tracking API in the SDK, where you get continuous tracking updates via the State, Cloud Recognition does not necessarily get continuous information; you need to explicitly make an update. This is done by calling updateQueryResults() on the TargetFinder.

The code examples are exposed in C++.

Initializing the system

To use Cloud Recognition you first need to initialize the system and set the credentials for your image database. The following code initializes and de-initializes Cloud Reco:

  Vuforia::TargetFinder* finder;
  
  const char* const kAccessKey = "Insert access key here";
  const char* const kSecretKey = "Insert secret key here";
  
  onApplicationInit()
  {
   Vuforia::TargetFinder* targetFinder = objectTracker->getTargetFinder   (Vuforia::ObjectTracker::TargetFinderType::CLOUD_RECO);
      assert(targetFinder != NULL);
  
      // Initialize Cloud Reco with project credentials:
      if (targetFinder->startInit(kAccessKey, kSecretKey))
      {
          targetFinder->waitUntilInitFinished();
      }
  }
  
  onApplicationDeinit()
  {
      // Deinitialize Visual Search
      targetFinder->deinit());
  }

Starting and Stopping a Cloud Recognition Search

The following sample code shows what a simple application needs to do to start Cloud Recognition when the application is started (or resumed) and to stop it when the application is paused:

  Vuforia::TargetFinder* finder;
  
  onApplicationResume()
  {
      // Start cloud based recognition
      finder->startRecognition();
  }
  
  onApplicationPause()
  {
      // Stop cloud based recognition
      finder->stop();
  }

Handling Search Results

During the execution, the developer must regularly check for the availability of new Cloud Recognition search results and, if desired, enable the target(s) for tracking.

This can be done in the Vuforia_onUpdate() method of the UpdateCallback class.

Vuforia::TargetFinder* finder;

void MyVuforia_OnUpdate()
{
    // Check if there are new results available:

    TargetFinderQueryResult queryResults = finder->updateQueryResults();
    if (queryResults.status == Vuforia::TargetFinder::UPDATE_RESULTS_AVAILABLE)
    {
        // Iterate through the new results:
        for (const Vuforia::TargetSearchResult* result: queryResults.results) {
        {
         const Vuforia::CloudRecoSearchResult* cloudRecoResult = static_cast<const    Vuforia::CloudRecoSearchResult*>(result);
            // Get the target metadata
            const char* metadata = cloudRecoResult->getMetaData();

            // Check if this target is suitable for tracking:
            if (cloudRecoResult->getTrackingRating() > 0)
            {
                // Enable this target for tracking:
                finder->enableTracking(cloudRecoResult);
            }
        }
    }
}

For more information on counting and handling the Cloud Recognition search results, please see Counting Cloud Recognition Events. Here, you will also find the section for Cloud Recognition Filter Mode

Learn More

How to Perform an Image Recognition Query

How to Use Rendering Primitives

Counting Cloud Recognition Events

Can this page be better?
Share your feedback on our community forum