Creating a Custom Driver

This page concerns the Vuforia Engine API version 9.8 and earlier. It has been deprecated and will no longer be actively updated.

Developers of external systems looking to take advantage of Vuforia Engine can write their own driver that provides data from external systems for Vuforia Engine to work with..

A driver is a native C/C++ dynamic library (.so or .dll file) that implements specific calls. During runtime, Vuforia Engine loads the library and uses it as a data source (e.g., device poses and image frames).

To get started, download the Native SDK package appropriate for your platform from the Developer Portal. We are mainly interested in the Driver.h file. This file defines several functions and classes which must be extended by the custom driver. For more information, refer to the comments in Driver.h file. The following diagram illustrates the high-level integration of custom drivers. 

Supported Camera Frame Formats

Vuforia Engine requires data input from the Vuforia Driver in the following data formats. 

• Frames from the camera must be in formats YUYV, NV12, or NV21. 
• The 6DOF position of the camera for every frame that was captured and transmitted. 

By fulfilling these, the driver you create can communicate with Vuforia Engine and the engine can interpret the data to accurately render augmentations in relation to targets and device movements. For more information on the Vuforia Driver integration, see our guide on Vuforia Support for Custom Devices using Vuforia Driver

Functions

• vuforiaDriver_getAPIVersion() – Returns a number containing the Driver Framework version that the library has been built against and conforms to. This constant is defined in Driver.h as VUFORIA_DRIVER_API_VERSION
• vuforiaDriver_getLibraryVersion() – Returns a string containing the version describing the library version. The driver developer selects a version value that is non-empty and generally under 50 characters long (e.g., “AndroidUVC-1.0.0”)
• vuforiaDriver_init() – Constructs and returns an instance of the VuforiaDriver object which defines the custom driver. The memory and the lifetime of this object is owned  by the library. Object is expected to remain valid until vuforiaDriver_deinit() is called.
• vuforiaDriver_deinit() – Destructs the specified instance of the VuforiaDriver object.

VuforiaDriver Class

• createExternalCamera() – Constructs a new ExternalCamera object. The memory and lifetime of this object is owned by the library. Object should remain valid until destroyExternalCamera() is called.
• destroyExternalCamera() – Destructs the specified instance of the ExternalCamera object
• createExternalPositionalDeviceTracker() - Constructs a new ExternalPositionalDeviceTracker object. The memory and lifetime of this object is owned by the library. Object should remain valid until destroyExternalPositionDeviceTracker is called.
• destroyExternalPositionalDeviceTracker() - Destructs the specified instance of the ExternalPositonalDeviceTracker object.
• getCapabilities() - Queries the driver for supported capability. The driver can support camera frames with corresponding device poses or camera frames only. There are two different capabilities supported by Vuforia Driver. Use Vuforia::Driver::Capability::CAMERA_IMAGE for camera only sequence. For camera and pose sequence, use (Vuforia::Driver::Capability::CAMERA_IMAGE | Vuforia::Driver::Capability::CAMERA_POSE)

ExternalCamera Class

• open() – Allows the Driver to allocate any resources needed. After calling open() the supported camera modes must be available.
• getNumSupportedCameraModes() – Returns the number of supported camera modes.
• getSupportedCameraMode(index) – Returns a specific supported camera mode by index.
• start(cameraMode, CameraCallback) – Starts providing camera frames to the specified callback class for the specified cameraMode.
• stop() – Stops producing new camera frames to the callback.
• close() – Closes the camera and release any resources obtained from open()

The ExternalCamera class also provides access to the following information through various functions (e.g., gets/sets). For more information, refer to the comments in Driver.h:

• ExposureMode
• ExposureValue
• FocusMode
• FocusValue

CameraCallback Class

The driver provides camera frames to Vuforia Engine through a callback class. The driver pushes CameraFrame structure to Vuforia Engine through the CameraCallback::onNewCameraFrame function. Each CameraFrame structure points to a buffer containing the actual frame data. Frame data also includes an accurate timestamp, exposure time and frame width/height. Along with the frame data, camera details must also be provided through the intrinsics parameter of type CameraIntrinsics. For more details on the CameraIntrinsics structure, refer to the External Camera Calibration article. The information provided by the CameraFrame structure must be accurate to achieve optimal performance.

ExternalPositionalDeviceTracker Class

• open() – Allows the Driver to allocate any needed resources. After calling open() the supported positional device tracker is available
• start(PoseCallback) – Start providing pose data to the specified callback class
• stop() – Stop producing new pose data to the callback
• close() – Close the positional device tracker and release any resources obtained from open()

PoseCallback Class

The driver provides pose data to Vuforia Engine through a callback class. The driver pushes Pose structure to Vuforia Engine through the PoseCallback::onNewPose function. Each Pose structure  includes pose data (e.g., an accurate timestamp and translation (vector) along x, y and z axes followed by a 3*3 rotation matrix for the pose). For a pose and camera frame to be associated together, the timestamp in Pose  and CameraFrame structures should match exactly. Driver implementation should first feed pose and then the matching camera frame. Pose is optional and in such scenarios, driver implementation can just feed the camera frames.

Driver Lifetime

Vuforia Engine manages the lifetime of the driver by making function calls in the following order:

1. vuforiaDriver_init() – Creates a new instance of the VuforiaDriver class
2. VuforiaDriver::getCapabilities() – Queries driver capability.
3. VuforiaDriver::createExternalPositionalDeviceTracker() – Called if driver capability includes Vuforia::Driver::Capability::CAMERA_POSE to create a new instance of the ExternalPositionalDeviceTracker class.
4. ExternalPositionalDeviceTracker::open() – Allocates any resources required by the external system. This call is made only if capability includes Vuforia::Driver::Capability::CAMERA_POSE
5. VuforiaDriver::createExternalCamera() – Called if driver capability includes Vuforia::Driver::Capability::CAMERA_IMAGE to create a new instance of the ExternalCamera class.
6. ExternalCamera::open() – Allocates any resources required by the external system. This call is made only if capability includes Vuforia::Driver::Capability::CAMERA_IMAGE
7. ExternalPositionalDeviceTracker::start() – Signals the external system to start providing Pose structures through the PoseCallback
8. ExternalCamera::start() – Signals the external system to start providing CameraFrame structures through the CameraCallback
9. Vuforia Engine now consumes Pose and  CameraFrame structures provided through the PoseCallback  and CameraCallback, respectively.
10. ExternalPositionalDeviceTracker::stop() – Signals the external system to stop providing pose
11. ExternalPositionalDeviceTracker:close() – Deallocates any resources that the external system was using
12. ExternalCamera::stop() – Signals the external system to stop providing camera frames
13. ExternalCamera:close() – Deallocates any resources that the external system was using
14. VuforiaDriver:: destroyExternalPositionalDeviceTracker() – Destructs the ExternalPositionalDeviceTracker instance
15. VuforiaDriver:: destroyExternalCamera() – Destructs the ExternalCamera instance
16. vuforiaDriver_deinit() – Destructs the VuforiaDriver instance

Using a Driver in Vuforia Engine

After the driver is compiled, it may be shared with other developers who want to use the external system. To use a driver within Vuforia Engine, the application developer must make the following call:

The setDriverLibrary();call must be made before calling Vuforia::init();

To disable an active driver and return to using default Vuforia Engine data capture mechanisms, pass a zero-length string to setDriverLibrary():

Learn More

Vuforia Engine Driver Framework

External Camera

External Positional Device Tracker

Format of Camera and Pose XML Sequence File

Building and Using the File Driver Sample

Building and Using the UVC Driver Sample

Vuforia Support for Custom Devices using Vuforia Driver

External Camera Calibration