Building and Using the UVC Camera Driver Sample

This page concerns the Vuforia Engine API version 9.8 and earlier. It has been deprecated and will no longer be actively updated.

The UVC (USB Video Class) Camera Driver Sample demonstrates how to access a USB camera from within Android.

Overview

The UVC Camera Driver Sample is an implementation of an External Camera using the Vuforia Engine Driver Framework and has been designed to work on both handheld devices and Qualcomm’s Dragonboard 410C . You can download the UVC Camera Driver Sample from the Vuforia Developer Portal.

The sample plugin uses an open source UVCCamera library to access camera frame data from USB-cameras. This library uses modified versions of libusb and libuvc to handle the USB-cameras. Unmodified versions of libuvc and libub do not work directly on Android because the USB-device handles can't be obtained in the same way as on a standard desktop Linux-system. So Android's USBManager is needed to obtain a device handle and then inject it into libusb through libuvc. The UVCCamera library that this sample uses have implemented the device handle injection.

The current supported cameras profiles are:

Logitech HD Webcam c310 / c910
Microsoft LifeCam Studio_

The UVC Camera Driver uses the Java-based Android USBManager API to access camera data. As the Driver needs to be implemented in C++, a pointer to the current JVM-instance needs to be injected through the Vuforia Engine stack into the Driver using the userData pointer passed in with the Vuforia::setExternalProviderLibrary() call. Then inside the Driver a JNI (Java Native Interface) is used to call the USBManager using the injected JVM-pointer. The Android USBManager is used to obtain a device handle for the attached webcam, which is then passed to libuvc and libusb.

Minimum Requirements

The UVC Camera Driver requires phones, tablets or the Dragonboard running Android 6 or later.

Building the UVC Camera Driver

Prerequisites

Operating System: Refer to Vuforia Engine Supported Versions for supported systems
Python 2 or later
Bash (required on Windows https://git-scm.com/downloads)
CMake 3.6 or greater
Ninja build system (https://ninja-build.org)
Android NDK 13b
git
Android SDK with API level 22 support
libuvc, libusb, libjpg-turbo

Build the Dependencies

Download Vuforia Engine for Android from the Vuforia Engine Developer Portal
Download the UVCDriver Sample from the Vuforia Engine Developer Portal
Extract the Vuforia Engine for Android package
Extract the UVC Driver Sample package to the VuforiaSDK/Samples directory. The UVCDriver directory should be located in VuforiaSDK/samples/UVCDriver/
Clone the https://github.com/ptc-shunt/UVCCamera repository to VuforiaSDK/Samples/UVCDriver directory. The UVCCamera directory should be located in VuforiaSDK/samples/UVCDriver/UVCCamera
Open terminal and change directory to the jni-build directory: UVCCamera/libuvccamera/src/main/jni
Run ndk-build from NDK 13b:
[path-to-ndk-on-your-machine]/ndk-build (ndk-build.cmd on Windows)

Compile UVC Camera Driver

These instructions assume that the “Compile Dependencies” section above was followed.

Go to the VuforiaSDK/samples/UVCDriver directory
Run python build.py (For more build options, use python build.py --help)
This should compile the plugin and copy all the dependencies into UVCDriver/build/bin.

NOTE: It also creates an intermediate directory to ../../build_android.

Using UVC Camera Driver

Add libUVCDriver.so, libuvc, libusb and libjpeg-turbo from build/bin into your Android-app project. The libraries can be added to an app depending on your build system: Using Android.mk or CMakeLists.txt or Gradle. Please note that you need to substitute '[path-in-your-filesystem]' with the correct path.

Using `Android.mk`

Add library definitions to your Android.mk:

include $(CLEAR_VARS)
LOCAL_MODULE := libUVCDriver-prebuilt
LOCAL_SRC_FILES = [path-in-your-filesystem]/UVCDriver/build/bin/Android/$(TARGET_ARCH_ABI)/libUVCDriver.so
include $(PREBUILT_SHARED_LIBRARY)

include $(CLEAR_VARS)
LOCAL_MODULE := libuvc-prebuilt
LOCAL_SRC_FILES = [path-in-your-filesystem]/UVCDriver/build/bin/Android/$(TARGET_ARCH_ABI)/libuvc.so
include $(PREBUILT_SHARED_LIBRARY)

include $(CLEAR_VARS)
LOCAL_MODULE := libusb-prebuilt
LOCAL_SRC_FILES = [path-in-your-filesystem]/UVCDriver/build/bin/Android/$(TARGET_ARCH_ABI)/libusb100.so
include $(PREBUILT_SHARED_LIBRARY)

include $(CLEAR_VARS)
LOCAL_MODULE := libjpeg-turbo-prebuilt
LOCAL_SRC_FILES = [path-in-your-filesystem]/UVCDriver/build/bin/Android/$(TARGET_ARCH_ABI)/libjpeg-turbo1500.so
include $(PREBUILT_SHARED_LIBRARY)

When defining your local module in the same Android.mk add prebuilt libraries as dependencies to your LOCAL_SHARED_LIBRARIES:

LOCAL_SHARED_LIBRARIES := Vuforia-prebuilt libUVCDriver-prebuilt libuvc-prebuilt libusb-prebuilt libjpeg-turbo-prebuilt

`CMakeLists.txt:`

Add prebuilt library to your CMakeLists.txt file:

add_library(UVC_DRIVER_LIBRARY SHARED IMPORTED)
set_property(TARGET UVC_DRIVER_LIBRARY PROPERTY IMPORTED_LOCATION
[path-in-your-filesystem]/UVCDriver/build/bin/Android/${ANDROID_ABI}/libUVCDriver.so)
add_library(UVC_LIBRARY SHARED IMPORTED)
set_property(TARGET UVC_LIBRARY PROPERTY IMPORTED_LOCATION
[path-in-your-filesystem]/UVCDriver/build/bin/Android/${ANDROID_ABI}/libuvc.so)
add_library(USB_LIBRARY SHARED IMPORTED)
set_property(TARGET USB_LIBRARY PROPERTY IMPORTED_LOCATION
[path-in-your-filesystem]/UVCDriver/build/bin/Android/${ANDROID_ABI}/libusb100.so)
add_library(JPEG_TURBO_LIBRARY SHARED IMPORTED)
set_property(TARGET JPEG_TURBO_LIBRARY PROPERTY IMPORTED_LOCATION
[path-in-your-filesystem]/UVCDriver/build/bin/Android/${ANDROID_ABI}/libjpeg-turbo1500.so)

When linking libraries in CMake, link libraries in the same CMakeLists.txt:

target_link_libraries(
VuforiaSample
${ANDROID_LIBRARY}
${LOG_LIBRARY}
${GLES3_LIBRARY}
VUFORIA_LIBRARY
UVC_DRIVER_LIBRARY
UVC_LIBRARY
USB_LIBRARY
JPEG_TURBO_LIBRARY
)

`Using Gradle`

This can be done in your app/build.gradle with the following:

android {
    sourceSets.main {
        jniLibs.srcDirs += '[path-in-your-filesystem]/UVCDriver/build/bin/Android/'
    }
}

Add UVCDriver.jar from build/bin into your Android-app project.
This can be done in your app/build.gradle with the following:
```
dependencies {
    implementation files("[path-in-your-filesystem]/UVCDriver/build/bin/Android/UVCDriver.jar")
}
```
Add following call to your source code before calling Vuforia::init();
```
Vuforia::setDriverLibrary("libUVCDriver.so", nullptr);
```

It is required to call requestUSBPermission to start the external USB camera.
1. For Kotlin, after the driver is loaded, this code must be added to the applications Activity to request permission to access the camera.
```
import com.vuforia.samples.uvcDriver.USBController;
// .. code omitted
USBController.requestUSBPermission(this@YourAppActivity)
```

In Unity, after the driver is loaded, this code must be added to the application to request permission to access the camera.

#if UNITY_ANDROID
        bool driverLibrarySet = false;
        driverLibrarySet = VuforiaUnity.SetDriverLibrary("libUVCDriver.so");

        if (driverLibrarySet)
        {
            // Load your applications scene here 
            // InitAndLoadScene(VUFORIA_DRIVER_CAMERA_SCENE_INDEX);

            // The application needs to ask for USB permissions to run the USB camera
            // this is done after the driver is loaded. We call a method in the UVC driver
            // Java code to request permissions, passing in the Unity app's activity.
            AndroidJavaClass unityJC = new AndroidJavaClass("com.unity3d.player.UnityPlayer");
            AndroidJavaObject unityActivity = unityJC.GetStatic<AndroidJavaObject>("currentActivity");

            AndroidJavaClass usbControllerJC = new AndroidJavaClass("com.vuforia.samples.uvcDriver.USBController");
            usbControllerJC.CallStatic("requestUSBPermission", unityActivity);
        }
        else
        {
            Debug.Log("Failed to initialize the UVC driver - defaulting to the standard scene");

            // Fall back to the in-built camera
        }
#endif

Creating a New Camera Profile

Calibration Information

To add additional camera profiles to the UVC Camera Driver, please follow the steps outlined in External Camera Calibration Instructions. Once the calibration data has been produced, create a new CameraDevice-node in the ExternalCameraCalibration.xml file that is in the UVCDriver/src/main/assets directory. A sample node is shown below.

<cameradevice>
    <calibration
      size="640 480"
      principal_point="315.558 227.668"
      focal_length="898.57 813.503"
      distortion_coefficients="-0.043780 0.226497 0.004928 0.000606 0 0 0 0">
    </calibration>
</cameradevice>

Device Information

In addition to the calibration data, the following camera information must also be provided:

VendoriID, integer in hexadecimal format
ProductID, integer in hexadecimal format

Both of these values can be obtained from a desktop operating system such as Windows or Mac.

Windows

Go to “Control panel” -> “Device manager”
Find your webcam. It will be under “Cameras” or “Imaging Devices”
Right click on your webcam and select “Properties”
Go to the “Details”-page and select “Hardware Ids” from the dropdown menu

Mac:

Click the “Apple” Icon on top-left corner of the screen.
Select “About this Mac” -> “System Report”
From the Hardware-list select the Camera and find the correct camera.

Once the device information is obtained, they must be added to the XML file in hexadecimal format. If the values are not already in hexadecimal, they must be converted before updating the XML file. The hexadecimal values should be prefixed by “0x”. The values should be added as attributes of the CameraDevice node. An example of the ExternalCameraCalibration.xml file is shown below (there can be multiple CameraDevice-nodes):

<CalibrationRoot>
  <CameraDevice VID="0x046D" PID="0x081B">
    <Calibration
      size="640 480"
      principal_point="315.558 227.668"
      focal_length="898.57 813.503"
      distortion_coefficients="-0.043780 0.226497 0.004928 0.000606 0 0 0 0"
    />
  </CameraDevice>
</CalibrationRoot>