Understanding the C++ Sample Application

This article discusses the C++ sample application included with the Leap SDK. After reading this article, you should be ready to access Leap hand tracking data from your own C++ applications.

Topics:

In the Leap SDK folder, you can find the following files used for this article:

Windows:

Mac:

Also note that you can find the Leap API reference documentation at LeapSDK/docs/API_Reference/annotated.html.

Overview

In a nutshell, the Leap motion tracking device detects and tracks hands and fingers placed within its field of view. The Leap captures this data one frame at a time. Your applications can use the Leap API to access this data.

The sample application demonstrates how to use the Leap API to listen for frame events dispatched by the Leap and how to access the hand and finger data in each frame. The application is a small command-line program that prints information about detected hands and fingers to standard output. The application is contained in a single file, Sample.cpp.

The sample application uses most of the key classes in the Leap API, including:

For more detailed information about these classes, pleases refer to the Leap API reference documentation.

Creating a Controller object

The Leap::Controller class provides the main interface between the Leap and your application. When you create a Controller object, it connects to the Leap software running on the computer and makes hand tracking data available through Leap::Frame objects. You can access these Frame objects by instantiating a Controller object and calling the Controller::frame function.

If your application has a natural update loop or frame rate, then you can call Controller::frame as part of this update. Otherwise, you can add a listener to the controller object. The controller object invokes the callback functions defined in your Leap::Listener subclass whenever a new frame of tracking data is available (and also for a few other Leap events).

The sample application creates a Controller object in its main function and adds an instance of a Listener subclass to the Controller with the Controller::addListener function:

int main() {
  // Create a sample listener and controller
  SampleListener listener;
  Controller controller;

  // Have the sample listener receive events from the controller
  controller.addListener(listener);

  // Keep this process running until Enter is pressed
  std::cout << "Press Enter to quit..." << std::endl;
  std::cin.get();

  // Remove the sample listener when done
  controller.removeListener(listener);

  return 0;
}

Now this snippet won't quite compile yet. You need to create a subclass of Leap::Listener named SampleListener. The Listener subclass defines callback functions which the controller calls when a Leap event occurs, including when a frame of tracking data is ready.

Subclassing the Listener class

The sample application defines a Leap::Listener subclass, SampleListener, which implements callback functions to handle events dispatched by the Leap. The events include:

For three lifecycle event callbacks, onInit, onDisconnect, and onExit the sample application simply prints a message to standard output. For the onConnect and onFrame event, the listener callback does a bit more work. When the controller calls the onConnect callback, the function enables recognition for all the gesture types. When the controller calls onFrame, the function gets the latest frame of motion tracking data and prints information about the detected objects to standard output.

Getting a Frame of data

The Controller calls the onFrame callback function when the Leap generates a new frame of motion tracking data. You can access the new data by calling the Controller::frame function, which returns the newest Frame object. (A reference to the Controller object is passed to the callback as a parameter.) A Frame object contains an ID, a timestamp, and a list containing a Hand object for each physical hand in view.

The following code from the sample application's onFrame implementation gets the most recent Frame object from the controller, retrieves the list of Hands from the Frame and then prints out the Frame ID, timestamp, and the number of hands, fingers, and tools detected in the frame:

// Get the most recent frame and report some basic information
const Frame frame = controller.frame();
std::cout << "Frame id: " << frame.id()
          << ", timestamp: " << frame.timestamp()
          << ", hands: " << frame.hands().count()
          << ", fingers: " << frame.fingers().count()
          << ", tools: " << frame.tools().count() << std::endl;

The function goes on to examine the first Hand in the list:

if (!frame.hands().empty()) {
  // Get the first hand
  const Hand hand = frame.hands()[0];

A Hand object contains an ID, properties representing the hand's physical characteristics, and a list of Finger objects. Each Finger object contains an ID and properties representing the characteristic of the finger.

Once it has retrieved a hand, the function checks it for fingersa and then averages the finger tip positions, printing the result with the number of fingers:

// Check if the hand has any fingers
const FingerList fingers = hand.fingers();
if (!fingers.empty()) {
  // Calculate the hand's average finger tip position
  Vector avgPos;
  for (int i = 0; i < fingers.count(); ++i) {
    avgPos += fingers[i].tipPosition();
  }
  avgPos /= (float)fingers.count();
  std::cout << "Hand has " << fingers.count()
            << " fingers, average finger tip position" << avgPos << std::endl;
}

Next, the function prints the radius of a sphere fit to the hand's curvature, along with the hand's palm position:

// Get the hand's sphere radius and palm position
std::cout << "Hand sphere radius: " << hand.sphereRadius()
          << " mm, palm position: " << hand.palmPosition() << std::endl;

Finally, the onFrame function uses Vector functions to calculate the hand's pitch, roll, and yaw angles from the hand's normal vector and the direction vector. The angles are converted from radians to degrees:

// Get the hand's normal vector and direction
const Vector normal = hand.palmNormal();
const Vector direction = hand.direction();

// Calculate the hand's pitch, roll, and yaw angles
std::cout << "Hand pitch: " << direction.pitch() * RAD_TO_DEG << " degrees, "
          << "roll: " << normal.roll() * RAD_TO_DEG << " degrees, "
          << "yaw: " << direction.yaw() * RAD_TO_DEG << " degrees" << std::endl << std::endl;

Getting Gestures

To receive gestures from the Leap, you first have to enable recognition for each type of gesture you are interested in. You can enable gesture recognition any time after the controller connects to the Leap (isConnected() is true). In the sample program, all gestures are enabled in the onConnect() callback function using the enableGesture() methods defined in the Controller class:

void SampleListener::onConnect(const Controller& controller) {
    std::cout << "Connected" << std::endl;
    controller.enableGesture(Gesture::TYPE_CIRCLE);
    controller.enableGesture(Gesture::TYPE_KEY_TAP);
    controller.enableGesture(Gesture::TYPE_SCREEN_TAP);
    controller.enableGesture(Gesture::TYPE_SWIPE);
}

The Leap adds Gesture objects representing each recognized movement pattern to the gestures list in the Frame object. In the onFrame() callback, the sample application loops through the gesture list and prints information about each one to the standard output. This operation is performed with a standard for-loop and switch statement.

The Gesture API uses a base Gesture class which is extended by classes representing the individual gestures. The objects in the gesture list are Gesture instances, so you must convert the Gesture instance to an instance of the correct subclass. Casting is not supported, instead each subclass provides a constructor that performs the conversion. For example, a Gesture instance representing a circle gesture can by converted into a CircleGesture instance with the following code:

CircleGesture circle = CircleGesture(gesture);

If you try to convert a Gesture instance into the wrong subclass, the constructor functions return invalid Gesture objects.

It is sometimes useful to compare the properties of a gesture in the current frame to those from an earlier frame. For example, the circle gesture has a progress attribute that describes how many times the finger has traversed the circle. This is the total progress, however; if you want the progress between frames, you must subtract the progress value of the gesture in the previous frame. You can do this by looking up the gesture in the previous frame using the gesture ID. The following code calculates the progress since the previous frame to derive the angle in radians:

float sweptAngle = 0;
if (circle.state() != Gesture::STATE_START) {
  CircleGesture previousUpdate = CircleGesture(controller.frame(1).gesture(circle.id()));
  sweptAngle = (circle.progress() - previousUpdate.progress()) * 2 * PI;
}

The full code for the gesture loop is:

// Get gestures
const GestureList gestures = frame.gestures();
for (int g = 0; g < gestures.count(); ++g) {
    Gesture gesture = gestures[g];

    switch (gesture.type()) {
        case Gesture::TYPE_CIRCLE:
        {
            CircleGesture circle = gesture;
            std::string clockwiseness;

            if (circle.pointable().direction().angleTo(circle.normal()) <= PI/4) {
              clockwiseness = "clockwise";
            } else {
              clockwiseness = "counterclockwise";
            }

            // Calculate angle swept since last frame
            float sweptAngle = 0;
            if (circle.state() != Gesture::STATE_START) {
              CircleGesture previousUpdate = CircleGesture(controller.frame(1).gesture(circle.id()));
              sweptAngle = (circle.progress() - previousUpdate.progress()) * 2 * PI;
            }
            std::cout << "Circle id: " << gesture.id()
                      << ", state: " << gesture.state()
                      << ", progress: " << circle.progress()
                      << ", radius: " << circle.radius()
                      << ", angle " << sweptAngle * RAD_TO_DEG
                      <<  ", " << clockwiseness << std::endl;
            break;
        }
        case Gesture::TYPE_SWIPE:
        {
            SwipeGesture swipe = gesture;
            std::cout << "Swipe id: " << gesture.id()
              << ", state: " << gesture.state()
              << ", direction: " << swipe.direction()
              << ", speed: " << swipe.speed() << std::endl;
            break;
        }
        case Gesture::TYPE_KEY_TAP:
        {
        KeyTapGesture tap = gesture;
            std::cout << "Key Tap id: " << gesture.id()
              << ", state: " << gesture.state()
              << ", position: " << tap.position()
              << ", direction: " << tap.direction()<< std::endl;
            break;
        }
        case Gesture::TYPE_SCREEN_TAP:
        {
            ScreenTapGesture screentap = gesture;
            std::cout << "Screen Tap id: " << gesture.id()
            << ", state: " << gesture.state()
            << ", position: " << screentap.position()
            << ", direction: " << screentap.direction()<< std::endl;
            break;
        }
        default:
            std::cout << "Unknown gesture type." << std::endl;
            break;
    }
}

Running the sample

To run the sample application:

  1. Compile and link the sample application:
  • On Windows, compile and link against Leap.h and LeapMath.h (in the SDK include directory) and Leap.lib (in the SDK lib directory). Use the libraries in the lib\x86 directory for 32-bit projects. Use the libraries in the lib\x64 directory for 64-bit projects.

Note: To compile an application in debug mode, link against the Leapd.lib library.

  • On Mac, compile and link against Leap.h and LeapMath.h (in the SDK include directory) and libLeap.dylib (in the SDK lib directory).
  1. Plug the Leap device into a USB port and place it in front of you.
  2. If you haven't already, install the Leap software.
  3. Start the Leap software. If prompted, enter your registered email address and password when prompted. The Leap icon appears in the notification area of the task bar (on Windows) or finder bar (on Mac) and turns green when ready.
  4. Run the sample application:
  • On Windows, make sure that Sample.exe and Leap.dll are in the current directory or a directory in your dynamic library search path (Path). Use the libraries in the lib\x86 directory for 32-bit projects. Use the libraries in the lib\x64 directory for 64-bit projects. Run the following command in a command-line prompt:

    Sample.exe

Note: To run an application in debug mode, make sure that Leapd.dll is in your DLL search path.

  • On Mac, make sure that Sample and libLeap.dylib are in the current directory or a directory in your dynamic library search path (DYLD_LIBRARY_PATH). Run the following command in a terminal window:

    ./Sample

You should see the messages "Initialized" and "Connected" printed to standard output when the application initializes and connects to the Leap. You should then see frame information printed each time the Leap dispatches the onFrame event. When you place a hand above the Leap, you should also see finger and palm position information printed.

Now that you have seen how to access motion tracking data from the Leap, you can begin developing your own C++ applications that integrate the Leap.


Copyright © 2012-2013 Leap Motion, Inc. All rights reserved.

Leap Motion proprietary and confidential. Not for distribution. Use subject to the terms of the Leap Motion SDK Agreement available at https://developer.leapmotion.com/sdk_agreement, or another agreement between Leap Motion and you, your company or other organization.