Understanding the Objective-C Sample Application
This article discusses the Objective-C sample application included with
the Leap SDK. After reading this article, you should be ready to access
Leap hand tracking data from your own Objective-C applications.
In the Leap SDK folder, you can find the following files used for this
article:
LeapSDK/samples/SampleObjectiveC.xcodeproj
— Xcode project file.LeapSDK/samples/ObjectiveC/Sample.h
— Header file for the sample
code which interacts with the Leap library.LeapSDK/samples/ObjectiveC/Sample.m
— Sample code that interacts
with the Leap library to print out tracking data.LeapSDK/samples/ObjectiveC/*
— Miscellaneous standard OS X
application files.LeapSDK/include/LeapObjectiveC.h
— Header file for the Leap
Objective C++ wrapper.LeapSDK/include/LeapObjectiveC.mm
— Objective C++ wrapper for the
Leap library.LeapSDK/include/Leap.h
— Leap C++ API class and struct
definitions.LeapSDK/include/LeapMath.h
— Leap C++ API Vector and Matrix class
and struct definitions.LeapSDK/lib/libLeap.dylib
— Leap library for Mac.
All the files needed for the sample application are already referenced
in the SampleObjectiveC Xcode project file.
Overview
In a nutshell, the Leap motion tracking device detects and tracks hands
and fingers placed within its field of view. The Leap captures this data
one frame at a time. Your applications can use the Leap API to access
this data.
The sample application demonstrates how to use the Leap API to listen
for frame events dispatched by the Leap and how to access the hand and
finger data in each frame. The application is a small program that
prints information about detected hands and fingers to standard output.
The sample application uses most of the key classes in the Leap API,
including:
- LeapController — the interface between the Leap and your
application - LeapFrame — contains a set of hand and finger tracking data
- LeapHand — contains tracking data for a detected hand
- LeapFinger — contains tracking data for a detected finger
- LeapVector — represents a 3D position or directional vector
- LeapGesture — represents a recognized gesture.
For more detailed information about these classes, pleases refer to the
Leap API reference documentation.
Using the Leap Objective-C Wrapper
The Objective-C wrapper translates the classes in the standard Leap C++
API into classes that can be used directly from Objective-C.
The main difference between the Objective-C API and the Leap C++ API is
the Listener callback mechanism. In Objective-C, you can use either an
NSNotification-based listener or the LeapDelegate protocol to observe
Leap events dispatched by a LeapController object. Another difference in
the two APIs is that the class names of the Objective-C wrapper use the
prefix Leap
. Thus, the C++ Controller
class is named
LeapController
in Objective-C.
Creating a LeapController object
The LeapController class provides the main interface between the Leap
and your application. When you create a LeapController object, it
connects to the Leap software running on the computer and makes hand
tracking data available through
these Frame objects by instantiating a LeapController object and calling
the [LeapController frame:] method.
If your application has a natural update loop or frame rate, then you
can call the LeapController frame:
method as part of this
update. Otherwise, you can either receive NSNotifications when a new
Frame of data is ready or add a delegate to the controller object.
The sample application creates a LeapController object in the run
method of the Sample class. The Sample class adopts the LeapListener
protocol and adds itself to the LeapController object so that it can
receive the Leap events:
- (void)run
{
controller = [[LeapController alloc] init];
[controller addListener:self];
NSLog(@"running");
}
The [LeapController addListener:] method automatically
subscribes the listener object to receive NSNotifications for the
members of the LeapListener protocol which it implements.
Note that the Leap library is multi-threaded. Using the
NSNotification-based listener avoids most threading issues since all of
the Leap callback functions are invoked on the main thread. If you use a
delegate instead, be aware that the delegate functions are invoked from
threads created by the Leap library, not the main, UI thread.
Adopting the LeapListener protocol
To add a listener object to a controller, create an object adopting the
LeapListener protocol. All the protocol methods are optional. The Sample
class implements the methods in the LeapListener protocol to handle
notifications dispatched by the Leap. The methods include:
- [LeapListener onInit:] — dispatched once, when the controller to
which the delegate is registered is initialized. - [LeapListener onConnect:] — dispatched when the controller connects
to the Leap and is ready to begin sending frames of motion tracking
data. - [LeapListener onDisconnect:] — dispatched if the controller
disconnects from the Leap (for example, if you unplug the Leap
device or shut down the Leap software). - [LeapListener onExit:] — Not implemented for notifications; will
never be dispatched. - [LeapListener onFrame:] — dispatched when a new frame of motion
tracking data is available.
In the lifecycle notification handlers, onInit
, onConnect
, and
onDisconnect
, the sample application simply prints a message to the
NSLog output. For the onFrame
notification, the handler implementation
does a bit more work. When the controller dispatches
onFrame:
, the method gets the latest frame of motion
tracking data and prints information about the detected objects to the
NSLog output.
You must add a listener to the LeapController to use NSNotifications.
While any object can subscribe to the Leap notifications, the controller
does not dispatch notifications unless there is at least one Listener
explicitly assigned.
Adopting the LeapDelgate Protocol
If you prefer to use a delegate rather than NSNotifications, you can
create an object adopting the LeapDelegate protocol. The methods of the
LeapDelegate protocol are the same as the LeapListener protocol except
that they take a LeapController object as a parameter instead of an
NSNotification object. The controller object invokes the methods defined
in your LeapDelegate implementation whenever a new frame of tracking
data is available (and also for a few other Leap events).
You can assign either a LeapListener object or a LeapDelegate object to
a controller, but not both.
Getting a Frame of data
The LeapController object dispatches an NSNotification object when the
Leap generates a new frame of motion tracking data. The notification
object includes a reference to the controller. You can access the new
frame of data by calling the controller object’s frame:
method.
The LeapFrame object returned by frame:
is the top level of the
Leap model heirarchy and provides such information as a frame ID, a
timestamp, and lists containing the hands, fingers, and tools in view.
The following code defined in the Sample class’s implementation of the
onFrame:
method gets the most recent LeapFrame object from
the controller, retrieves the list of LeapHand objects from the LeapFrame and
then logs the frame ID, timestamp, and the number of hands, fingers, and
tools detected in the frame:
- (void)onFrame:(NSNotification *)notification;
{
LeapController *aController = (LeapController *)[notification object];
// Get the most recent frame and report some basic information
LeapFrame *frame = [aController frame:0];
NSLog(@"Frame id: %lld, timestamp: %lld, hands: %ld, fingers: %ld, tools: %ld",
[frame id], [frame timestamp], [[frame hands] count],
[[frame fingers] count], [[frame tools] count]);
The method goes on to examine the first hand in the list of hands:
if ([[frame hands] count] != 0) {
// Get the first hand
LeapHand *hand = [[frame hands] objectAtIndex:0];
A LeapHand object contains an ID, properties representing the hand's
physical characteristics, and a list of LeapFinger objects. Each
LeapFinger object contains an ID and properties representing the
characteristic of the finger.
Once it has retrieved a hand, the method checks it for fingers and then
averages the finger tip positions, printing the result with the number
of fingers:
// Check if the hand has any fingers
NSArray *fingers = [hand fingers];
if ([fingers count] != 0) {
// Calculate the hand's average finger tip position
LeapVector *avgPos = [[LeapVector alloc] init];
for (int i = 0; i < [fingers count]; i++) {
LeapFinger *finger = [fingers objectAtIndex:i];
avgPos = [avgPos plus:[finger tipPosition]];
}
avgPos = [avgPos divide:[fingers count]];
NSLog(@"Hand has %ld fingers, average finger tip position %@",
[fingers count], avgPos);
}
Next, the method prints the radius of a sphere fit to the hand's
curvature, along with the hand’s palm position:
// Get the hand's sphere radius and palm position
NSLog(@"Hand sphere radius: %f mm, palm position: %@",
[hand sphereRadius], [hand palmPosition]);
Finally, the onFrame:
method uses LeapVector class methods
to get the hand’s pitch, roll, and yaw angles from the hand’s normal
vector and the direction vectors. The angles are converted from radians
to degrees:
// Get the hand's normal vector and direction
const LeapVector *normal = [hand palmNormal];
const LeapVector *direction = [hand direction];
// Calculate the hand's pitch, roll, and yaw angles
NSLog(@"Hand pitch: %f degrees, roll: %f degrees, yaw: %f degrees\n",
[direction pitch] * LEAP_RAD_TO_DEG,
[normal roll] * LEAP_RAD_TO_DEG,
[direction yaw] * LEAP_RAD_TO_DEG);
Getting Gestures
To receive gestures from the Leap, you first have to enable recognition
for each type of gesture you are interested in. You can enable gesture
recognition any time after the controller connects to the Leap. In the
sample program, all gestures are enabled in the onConnect:
handler using the [LeapController enableGesture:enable:] method:
- (void)onConnect:(NSNotification *)notification;
{
NSLog(@"Connected");
LeapController *aController = (LeapController *)[notification object];
[aController enableGesture:LEAP_GESTURE_TYPE_CIRCLE enable:YES];
[aController enableGesture:LEAP_GESTURE_TYPE_KEY_TAP enable:YES];
[aController enableGesture:LEAP_GESTURE_TYPE_SCREEN_TAP enable:YES];
[aController enableGesture:LEAP_GESTURE_TYPE_SWIPE enable:YES];
}
The Leap adds LeapGesture objects representing each recognized movement
pattern to the gestures list in the Frame object. In the
onFrame:
handler, the sample application loops through the
gesture list and prints information about each one to the standard
output. This operation is performed with a standard for-loop and switch
statement.
It is sometimes useful to compare the properties of a gesture in the
current frame to those from an earlier frame. For example, the circle
gesture has a progress attribute that describes how many times the
finger has traversed the circle. This is the total progress, however; if
you want the progress between frames, you must subtract the progress
value of the gesture in the previous frame. You can do this by looking
up the gesture in the previous frame using the gesture ID. The following
code calculates the progress since the previous frame to derive the
angle in radians:
// Calculate the angle swept since the last frame
float sweptAngle = 0;
if(gesture.state != LEAP_GESTURE_STATE_START) {
LeapGesture *previousUpdate = [[aController frame:1] gesture:gesture.id];
sweptAngle = (gesture.progress - previousUpdate.progress) * 2 * LEAP_PI;
}
The full code for the gesture loop is:
NSArray *gestures = [frame gestures:nil];
for (int g = 0; g < [gestures count]; g++) {
LeapGesture *gesture = [gestures objectAtIndex:g];
switch (gesture.type) {
case LEAP_GESTURE_TYPE_CIRCLE: {
LeapCircleGesture *circleGesture = (LeapCircleGesture *)gesture;
// Calculate the angle swept since the last frame
float sweptAngle = 0;
if(circleGesture.state != LEAP_GESTURE_STATE_START) {
LeapCircleGesture *previousUpdate =
(LeapCircleGesture *)[[aController frame:1] gesture:gesture.id];
sweptAngle = (circleGesture.progress - previousUpdate.progress) * 2 * LEAP_PI;
}
NSLog(@"Circle id: %d, %@, progress: %f, radius %f, angle: %f degrees",
circleGesture.id, [Sample stringForState:gesture.state],
circleGesture.progress, circleGesture.radius, sweptAngle * LEAP_RAD_TO_DEG);
break;
}
case LEAP_GESTURE_TYPE_SWIPE: {
LeapSwipeGesture *swipeGesture = (LeapSwipeGesture *)gesture;
NSLog(@"Swipe id: %d, %@, position: %@, direction: %@, speed: %f",
swipeGesture.id, [Sample stringForState:swipeGesture.state],
swipeGesture.position, swipeGesture.direction, swipeGesture.speed);
break;
}
case LEAP_GESTURE_TYPE_KEY_TAP: {
LeapKeyTapGesture *keyTapGesture = (LeapKeyTapGesture *)gesture;
NSLog(@"Key Tap id: %d, %@, position: %@, direction: %@",
keyTapGesture.id, [Sample stringForState:keyTapGesture.state],
keyTapGesture.position, keyTapGesture.direction);
break;
}
case LEAP_GESTURE_TYPE_SCREEN_TAP: {
LeapScreenTapGesture *screenTapGesture = (LeapScreenTapGesture *)gesture;
NSLog(@"Screen Tap id: %d, %@, position: %@, direction: %@",
screenTapGesture.id, [Sample stringForState:screenTapGesture.state],
screenTapGesture.position, screenTapGesture.direction);
break;
}
default:
NSLog(@"Unknown gesture type");
break;
}
}
Running the sample
To run the sample application:
Plug the Leap device into a USB port and place it in front of you.
If you haven’t already, install the Leap software.
Start the Leap software. If prompted, enter your registered email
address and password. The Leap icon appears on the finder bar and
turns green when ready.Open the SampleObjectiveC project file in Xcode.
Press the Xcode Run button.
You should see the messages “Initialized” and “Connected” printed to
Debug Output area in Xcode when the application initializes and connects
to the Leap. You should then see frame information printed to the Debug
Output area each time the Leap dispatches the onFrame
event. When you
place a hand above the Leap, you should also see finger and palm
position information printed. Note that the application window is
intentionally left blank. All output appears in the Xcode Debug Output
area.
Now that you have seen how to access motion tracking data from the Leap,
you can begin developing your own Objective-C applications that
integrate the Leap.
Copyright © 2012-2013 Leap Motion, Inc. All rights reserved.
Leap Motion proprietary and confidential. Not for distribution. Use
subject to the terms of the Leap Motion SDK Agreement available at
https://developer.leapmotion.com/sdk_agreement, or another agreement
between Leap Motion and you, your company or other organization.