Building and Using the File Driver Sample

The File Driver Sample demonstrates how to implement a driver which uses the External Camera and the External Positional Device Tracker APIs to feed Vuforia Engine camera frame and device pose, respectively.

The File Driver sample can be downloaded from the Developer Portal. From device storage, the sample loads a sequence of pre-recorded camera frames and device poses. The pose, followed by the corresponding frame data, are then fed into Vuforia Engine. The package includes two sets of frame and pose sequences: Stones Image Target and Astronaut Image Target. Native developers should use the Stones Image Target while Unity developers should use the Astronaut Image Target sequence.

The FileDriver contains a sample sequence (XML) file which contains camera frame and pose information. For more information, refer to the Format of Camera and Pose XML Sequence File article.

In addition to the included sample sequences, Vuforia Engine also allows to record sessions from any app using the Session Recorder API.

Building the File Driver

Prerequisites

Supported image file formats for the File Driver

.xpgm, .png, and .jpg are each supported on all platforms. Other formats are determined by the individual platform image loader.

Instructions

  1. Extract the File Driver package and the Vuforia SDK package.
  2. Run the appropriate build command:
    Note: The -vh  or -vf  command line options may be omitted if the driver is put in the SDK's samples directory.

Android:

python build.py android -vh [path to Vuforia SDK]/build/include

UWP:

python build.py uwp -vh [path to Vuforia SDK]/build/include

iOS:

Correct the file line of the ios.toolchain.cmake file, located in the cmake directory to point to the correct Xcode location. Alternatively, remove this block completely so cmake will find it for you.

set(CONST_DIR_XCODE_DEVELOPER/Applications/Xcode_93.app/Contents/Developer)

Then run the following command (the iOS option is -vf and the value is the folder containing Vuforia.framework):

python build.py ios -vf [path to Vuforia SDK]/build
  1. Output files should be placed in the build/bin directory by default: libFileDriver.so for Android, FileDriver.dll for Windows, and FileDriver.framework

Using the File Driver

Android

  1. Add libFileDriver.so for each needed architecture from build/bin to your Android app project by modifying either Android.mk or CMakeLists.txt or build.gradle. Select the option that best fits your project:
    • Android.mk:

      Add libFileDriver prebuilt definition to your Android.mk file:

      include $(CLEAR_VARS)
LOCAL_MODULE := libFileDriver-prebuilt
LOCAL_SRC_FILES = [path-in-your-filesystem]/FileDriver/build/bin/Android/Release/$(TARGET_ARCH_ABI)/libFileDriver.so   
include $(PREBUILT_SHARED_LIBRARY)

      When defining your local module in the same Android.mk add libFileDriver-prebuilt as a dependency to your LOCAL_SHARED_LIBRARIES:

      LOCAL_SHARED_LIBRARIES := libFileDriver-prebuilt
    • CMakeLists.txt:

      Add libFileDriver prebuilt definition to your CMakeLists.txt file:

      add_library(VUFORIA_FILEDRIVER_LIBRARY SHARED IMPORTED)
set_property(TARGET VUFORIA_FILEDRIVER_LIBRARY PROPERTY IMPORTED_LOCATION
[path-in-your-filesystem]/FileDriver/build/bin/Android/Release/${ANDROID_ABI}/libFileDriver.so)

       

      When linking libraries in CMake, link libFileDriver library in the same CMakeLists.txt: 
      target_link_libraries(
VuforiaSample
${ANDROID_LIBRARY}
${LOG_LIBRARY}
${GLES3_LIBRARY}
VUFORIA_LIBRARY
VUFORIA_FILEDRIVER_LIBRARY
)
    • Gradle: 

      If using either Android.mk or CMakeLists.txt you will also need to update your build.gradle to include the library in the APK. Use the following code in your app/build.gradle:

      android {
    sourceSets.main {
        jniLibs.srcDirs += '[path-in-your-filesystem]/FileDriver/build/bin/Android/Release/'
    }
}

       

      NOTE: If you are using the Android Plugin version 4.x and a CMakeLists.txt then the above change to build.gradle is not required.
  2. Add FileDriver.jar from build/bin to your Android-app project. Use the following code in your app/build.gradle: 
    dependencies {
    implementation files("[path-in-your-filesystem]/FileDriver/build/bin/Android/Release/FileDriver.jar")
}

     

  3. Add the sample sequence from the data directory to your Android-app project. Use the following code in your app/build.gradle: 
    android {
    sourceSets.main {
        assets.srcDirs += '[path-in-your-filesystem]/FileDriver/data/stones'
    }
}

     

  4.  Before calling Vuforia::init(), add the following call to your source code: C++ 
    Vuforia::setDriverLibrary("libFileDriver.so", nullptr);

     

UWP

  1. Add FileDriver.dll from build/bin/uwp into your Visual Studio UWP app project.
    1. Import the FileDriver.dll to the root of your project. Remember to use a .dll that matches your architecture (x86/x64) and build type (Debug/Release) configurations.
    2. Click FileDriver.dll from the project file list and set property Content to True.
  2. Add a sample sequence from the data directory into your Visual Studio UWP app project. For example, to use the “stones” sequence:
    1. Import FileDriver/data/stones/FileDriverSequence.xml into your project.
    2. Copy ‘Camera' folder containing images from Driver sequence into the root of the VuforiaSamples project.
      • NOTE: the root is the Vuforia Samples folder where the VuforiaSamples.vcxproj is present.
    3. Right click on the project ‘VuforiaSamples’ and click Add->Existing item.
    4. Select all the images from the Camera folder that was copied into the root of the VuforiaSamples project. Visual Studio will proceed to add all the selected images.
    5. Select all images in the Solution Explorer and right click and select Properties.
    6. Mark Content as ‘Yes’, save the project and proceed to the App package creation process.
    7. Select all files and set property Content to True.
  3. Add the following call before Vuforia::init():
    Vuforia::setDriverLibrary("FileDriver.dll", nullptr);

iOS

  1. In your Xcode project, open the build settings screen and select the General tab. 
  2. In the Embedded Binaries section, click the + icon to add a new binary. 
  3. From the file selection menu, locate the iOS File Driver Framework in bin/Release-iphoneos/FileDriver.framework
  1. In the Linked Frameworks and Libraries section, remove the FileDriver entry. The entry should not be listed in this section and the application should not try to link against it.
  1. Drag the FileDriverSequence.xml file and the camera image files into the left hand project pane. Package the image files into a Camera folder in the app bundle with the folder structure as seen below.
  1. Add the following call before Vuforia::init():
    Vuforia::setDriverLibrary("FileDriver.framework”, nullptr);

Changing the Sequence

To load a specific sequence of pose and images, provide a new path using the FileDriverUserData struct and the second parameter of the setDriverLibrary call. Refer to the example below:

#include <filedriveruserdata.h>
static FileDriverUserData fdUserData;
fdUserData.sequenceDirectoryAbsolutePath = "/sdcard/sequence/";
bool ret = Vuforia::setDriverLibrary(VUFORIA_DRIVER_FILEDRIVER_LIB, &fdUserData);
</filedriveruserdata.h>

Custom Sequences

Developers looking to create their own sequence, should consider the following:

  • A top level XML file groups a device pose and a matching camera frame under one <Element> tag. This XML file also contains global attributes, such as total number of camera frames, width,  and height, which apply to the entire sequence. For more information, refer to the Format of Camera and Pose XML Sequence File article.
  • The width and height of each frame should remain consistent and match the intrinsic information provided with each frame.
  • The framerate at which frames are provided to Vuforia Engine does not necessarily need to match the rate at which they were captured.
  • The samples currently use PNG format, but other image formats may be supported depending on the platform. Ensure that the files can be read at the rate which matches the desired framerate. For instance, if the desired framerate is 30 FPS, then the device needs to be able to read 30 files per second.
  • Avoid compression artifacts in the files.
  • Developers need to create/name their file sequences to easily process the sequence in the appropriate order. The sample uses 'filename#####.png' notation, where ##### represents the frame index, padded to 5-digits.

Unity

  1. Copy the compiled binaries from build/bin to:
    • Android: Assets/Plugins/Android/
    • UWP: Assets/Plugins/WindowsStoreApps/[x64, x86]
    • iOS: Assets/Plugins/iOS/

Note: Ensure the correct plugin platform options are set for the imported libraries. The selection options should match the intended platform (see below).

  1. Add the sequence to your Unity project. Copy the Camera folder and FileDriverSequence.xml file into the Assets/StreamingAssets folder. This ensures it's included in the application build:

  1. In the VuforiaConfiguration window, select Delayed initialization. This sets the driver library name before Vuforia Engine is initialized in order to use the file driver from Unity.

The driver library can then be set from any app level script, e.g. in an Awake() method. After that, Vuforia Engine can be initialized:

  VuforiaUnity.SetDriverLibrary("libFileDriver.so"); // Android
  VuforiaRuntime.Instance.InitVuforia();

On Android, the image sequence should play in a loop instead of the device camera being used. On UWP, the image files stored in Assets/StreamingAssets end up in a different application build folder than the file driver library expects. As a result, the path needs to be set via the FileDriverUserData struct:

struct FileDriverUserData
  {
      public string sequenceDirectoryAbsolutePath;
  };
  
  private static string filePath;
  private static FileDriverUserData userData;
  private static IntPtr userDataPtr = IntPtr.Zero;
  
   void Awake()
  {
    filePath = Application.streamingAssetsPath.Replace("/", "\\"); // get the absolute   path to the StreamingAssets folder
      
      userData = new FileDriverUserData()
      {
          sequenceDirectoryAbsolutePath = filePath
      };
      userDataPtr = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(FileDriverUserData)));
      Marshal.StructureToPtr(userData, userDataPtr, false);
      
      VuforiaUnity.SetDriverLibrary("FileDriver.dll", userDataPtr); // UWP
      VuforiaRuntime.Instance.InitVuforia();
  }

On iOS, the required changes are similar to the other platforms. However, you need to use FileDriver.framework and changing the absolute path is not required. The Unity-iPhone.xcodeproj should be modified to include the FileDriver.framework as an embedded framework. Remember to remove it from the list of linked libraries (refer to the step relating to linked libraries in the Using the File Driver > iOs section). The following is the full code:

struct FileDriverUserData
    {
        public string sequenceDirectoryAbsolutePath;
    }

    private static string filePath;
    private static FileDriverUserData userData;
    private static IntPtr userDataPtr = IntPtr.Zero;
    private bool okSetFileDriver = false;

    void Awake()
    {
     
        filePath = Application.streamingAssetsPath; 
   userData = new FileDriverUserData()
        {
            sequenceDirectoryAbsolutePath = filePath
        };

        userDataPtr = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(FileDriverUserData)));
        Marshal.StructureToPtr(userData, userDataPtr, false);

        okSetFileDriver = VuforiaUnity.SetDriverLibrary("FileDriver.framework", userDataPtr);
        if (!okSetFileDriver)
        {
            Debug.Log("Failed to set file driver");
        }
        VuforiaRuntime.Instance.InitVuforia();
    }

 

Learn More

Can this page be better?
Share your feedback on our community forum