Understanding the Python Sample Application

This article discusses the Python sample application included with the Leap Motion SDK. After reading this article, you should be ready to access Leap Motion hand tracking data from your own Python applications.

Topics:

In the Leap Motion SDK folder, you can find the following files used for this article:

Windows:

Mac:

Also note that you can find the Leap Motion API reference documentation at LeapSDK/docs/API_Reference/annotated.html.

Overview

In a nutshell, the Leap Motion controller detects and tracks hands and fingers placed within its field of view. The Leap Motion captures this data one frame at a time. Your applications can use the Leap Motion API to access this data.

The sample application demonstrates how to use the Leap Motion API to listen for frame events dispatched by the Leap Motion and how to access the hand and finger data in each frame. The application is a small command-line program that prints information about detected hands and fingers to standard output. The application is contained in a single file, Sample.py.

The sample application uses most of the key classes in the Leap Motion API, including:

For more detailed information about these classes, pleases refer to the Leap Motion API reference documentation.

Creating a Controller object

The Controller class provides the main interface between the Leap Motion software and your application. When you create a Controller object, it connects to the Leap Motion software running on the computer and makes hand tracking data available through Frame objects. You can access these Frame objects by instantiating a Controller object and calling the Controller.frame function.

If your application has a natural update loop or frame rate, then you can call Controller.frame as part of this update. Otherwise, you can add a listener to the controller object. The controller object invokes the callback functions defined in your Listener subclass whenever a new frame of tracking data is available (and also for a few other Leap Motion events).

The sample application creates a Controller object in its main function and adds an instance of a Listener subclass to the Controller with the Controller.add_listener function:

def main():
    # Create a sample listener and controller
    listener = SampleListener()
    controller = Leap.Controller()

    # Have the sample listener receive events from the controller
    controller.add_listener(listener)

    # Keep this process running until Enter is pressed
    print "Press Enter to quit..."
    sys.stdin.readline()

    # Remove the sample listener when done
    controller.remove_listener(listener)

The application then runs until you press the Enter key. In the meantime, the controller calls the appropriate listener callback functions when events occur. Before running the program, though, you must create your own subclass of the Leap.Listener class.

Subclassing the Listener class

The sample application defines a Listener subclass, SampleListener, which implements callback functions to handle events dispatched by the Leap. The events include:

For three lifecycle event callbacks, on_init, on_disconnect, and on_exit the sample application simply prints a message to standard output. For the on_connect and on_frame event, the listener callback does a bit more work. When the controller calls the on_connect callback, the function enables recognition for all the gesture types. When the controller calls on_frame, the function gets the latest frame of motion tracking data and prints information about the detected objects to standard output.

Getting a Frame of data

The Controller calls the on_frame callback function when the Leap generates a new frame of motion tracking data. You can access the new data by calling the Controller.frame function, which returns the newest Frame object. (A reference to the Controller object is passed to the callback as a parameter.) A Frame object contains an ID, a timestamp, and a list containing a Hand object for each physical hand in view.

The following code from the sample application's on_frame implementation gets the most recent Frame object from the controller, retrieves the list of Hands from the Frame and then prints out the Frame ID, timestamp, and the number of Hands in the list:

# Get the most recent frame and report some basic information
frame = controller.frame()
print "Frame id: %d, timestamp: %d, hands: %d, fingers: %d, tools: %d" % (
      frame.id, frame.timestamp, len(frame.hands), len(frame.fingers), len(frame.tools))

The function goes on to examine the first Hand in the list:

if not frame.hands.empty:
    # Get the first hand
    hand = frame.hands[0]

A Hand object contains an ID, properties representing the hand's physical characteristics, and a list of Finger objects. Each Finger object contains an ID and properties representing the characteristic of the finger.

Once it has retrieved a hand, the function checks it for fingers and then averages the finger tip positions, printing the result and the number of fingers:

# Check if the hand has any fingers
fingers = hand.fingers
if not fingers.empty:
    # Calculate the hand's average finger tip position
    avg_pos = Leap.Vector()
    for finger in fingers:
        avg_pos += finger.tip_position
    avg_pos /= len(fingers)
    print "Hand has %d fingers, average finger tip position: %s" % (
          len(fingers), avg_pos)

Next, the function prints the radius of a sphere fit to the hand's curvature, along with the hand's palm position:

# Get the hand's sphere radius and palm position
print "Hand sphere radius: %f mm, palm position: %s" % (
      hand.sphere_radius, hand.palm_position)

Finally, the on_frame function uses Vector functions to calculate the hand's pitch, roll, and yaw angles from the hand's normal vector and the direction vector. The angles are converted from radians to degrees:

# Get the hand's normal vector and direction
normal = hand.palm_normal
direction = hand.direction

# Calculate the hand's pitch, roll, and yaw angles
print "Hand pitch: %f degrees, roll: %f degrees, yaw: %f degrees\n" % (
    direction.pitch * Leap.RAD_TO_DEG,
    normal.roll * Leap.RAD_TO_DEG,
    direction.yaw * Leap.RAD_TO_DEG)

Getting Gestures

To receive gestures from the Leap, you first have to enable recognition for each type of gesture you are interested in. You can enable gesture recognition any time after the controller connects to the Leap (is_connected is true). In the sample program, all gestures are enabled in the on_connect() callback function using the enable_gesture() methods defined in the Controller class:

def on_connect(self, controller):
    print "Connected"

    # Enable gestures
    controller.enable_gesture(Leap.Gesture.TYPE_CIRCLE);
    controller.enable_gesture(Leap.Gesture.TYPE_KEY_TAP);
    controller.enable_gesture(Leap.Gesture.TYPE_SCREEN_TAP);
    controller.enable_gesture(Leap.Gesture.TYPE_SWIPE);

The Leap adds Gesture objects representing each recognized movement pattern to the gestures list in the Frame object. In the on_frame() callback, the sample application loops through the gesture list and prints information about each one to the standard output. This operation is performed with a standard for-loop and switch statement.

The Gesture API uses a base Gesture class which is extended by classes representing the individual gestures. The objects in the gesture list are Gesture instances, but you cannot simple access the subclass properties. You must convert the Gesture instance to an instance of the correct subclass. Each Gesture subclass provides a constructor that performs the conversion. For example, a Gesture instance representing a circle gesture can by converted into a CircleGesture instance with the following code:

circle = CircleGesture(gesture)

If you try to convert a Gesture instance into the wrong subclass, the constructor functions return invalid Gesture objects.

It is sometimes useful to compare the properties of a gesture in the current frame to those from an earlier frame. For example, the circle gesture has a progress attribute that describes how many times the finger has traversed the circle. This is the total progress, however; if you want the progress between frames, you must subtract the progress value of the gesture in the previous frame. You can do this by looking up the gesture in the previous frame using the gesture ID. The following code calculates the progress since the previous frame to derive the angle in radians:

# Calculate the angle swept since the last frame
swept_angle = 0
if circle.state != Leap.Gesture.STATE_START:
    previous_update = CircleGesture(controller.frame(1).gesture(circle.id))
    swept_angle =  (circle.progress - previous_update.progress) * 2 * Leap.PI

The full code for the gesture loop is:

# Gestures
for gesture in frame.gestures():
    if gesture.type == Leap.Gesture.TYPE_CIRCLE:
        circle = CircleGesture(gesture)

        # Determine clock direction using the angle between the pointable and the circle normal
        if circle.pointable.direction.angle_to(circle.normal) <= Leap.PI/4:
            clockwiseness = "clockwise"
        else:
            clockwiseness = "counterclockwise"

        # Calculate the angle swept since the last frame
        swept_angle = 0
        if circle.state != Leap.Gesture.STATE_START:
            previous_update = CircleGesture(controller.frame(1).gesture(circle.id))
            swept_angle =  (circle.progress - previous_update.progress) * 2 * Leap.PI

        print "Circle id: %d, %s, progress: %f, radius: %f, angle: %f degrees, %s" % (
                gesture.id, self.state_string(gesture.state),
                circle.progress, circle.radius, swept_angle * Leap.RAD_TO_DEG, clockwiseness)

    if gesture.type == Leap.Gesture.TYPE_SWIPE:
        swipe = SwipeGesture(gesture)
        print "Swipe id: %d, state: %s, position: %s, direction: %s, speed: %f" % (
                gesture.id, self.state_string(gesture.state),
                swipe.position, swipe.direction, swipe.speed)

    if gesture.type == Leap.Gesture.TYPE_KEY_TAP:
        keytap = KeyTapGesture(gesture)
        print "Key Tap id: %d, %s, position: %s, direction: %s" % (
                gesture.id, self.state_string(gesture.state),
                keytap.position, keytap.direction )

    if gesture.type == Leap.Gesture.TYPE_SCREEN_TAP:
        screentap = ScreenTapGesture(gesture)
        print "Screen Tap id: %d, %s, position: %s, direction: %s" % (
                gesture.id, self.state_string(gesture.state),
                screentap.position, screentap.direction )

Running the sample

To run the sample application:

  1. Plug the Leap device into a USB port and place it in front of you.
  2. If you haven't already, install the Leap software.
  3. Start the Leap software. If prompted, enter your registered email address and password when prompted. The Leap icon appears in the notification area of the task bar (on Windows) or finder bar (on Mac) and turns green when ready.
  4. Run the sample application:
  • On Windows, make sure that Sample.py, Leap.py, LeapPython.pyd, and Leap.dll are in the current directory. If you are using a 32-bit version of Python, use the .pyd and .dll files from the lib\x86 folder of the SDK. Use the .pyd and .dll files from lib\x64 with 64-bit Python. Run the following command in a command-line prompt:

    python Sample.py

  • On Mac, make sure that Sample.py, Leap.py, LeapPython.so, and libLeap.dylib are in the current directory and run the following command in a terminal window:

    python Sample.py

You should see the messages "Initialized" and "Connected" printed to standard output when the application initializes and connects to the Leap. You should then see frame information printed each time the Leap dispatches the on_frame event. When you place a hand above the Leap, you should also see finger and palm position information printed.

Now that you have seen how to access motion tracking data from the Leap, you can begin developing your own Python applications that integrate the Leap.


Copyright © 2012-2013 Leap Motion, Inc. All rights reserved.

Leap Motion proprietary and confidential. Not for distribution. Use subject to the terms of the Leap Motion SDK Agreement available at https://developer.leapmotion.com/sdk_agreement, or another agreement between Leap Motion and you, your company or other organization.