This article discusses the JavaScript sample application included with the Leap SDK. After reading this article, you should be ready to access Leap hand tracking data from your own Web applications.
Topics:
In the Leap SDK folder, you can find the following file used for this article:
You will also need the Leap JavaScript client library available from Leap Motion's LeapJS repository on GitHub.
Also note that you can find the Leap JavaScript API reference documentation on GitHub.
In a nutshell, the Leap motion tracking device detects and tracks hands and fingers placed within its field of view. The Leap captures this data one frame at a time. Web applications can use the Leap application WebSocket server and the JavaScript API to access this data. The Leap sends tracking information through the socket connection as a JSON formated message. The JavaScript API takes the JSON message and evaluates it into proper objects.
The sample application demonstrates how to use the Leap JavaScript API. The example is contained in a single Web page, Sample.html. The application displays several properties from the key tracking data objects in the Leap API, including:
For more detailed information about these classes, pleases refer to the Leap JavaScript API reference documentation on GitHub.
The Leap JavaScript client library is available from Leap Motion's LeapJS repository on GitHub. The library handles the connection to the Leap application and passes frames of tracking data to your application. LeapJS is an open-source project and distributed separately from the main Leap SDK. We encourage community contributions to this library.
To use the JavaScript API, you must include the leap.js file (or the minified version, leap.min.js). In the Sample.html file, I use a simple HTML structure to display the Leap data and include the leap.js file in the usual way using script tags:
<html> <head> <title>Leap JavaScript Sample</title> <script src="leap.js"></script> <script>// JavaScript code goes here</script> </head> <body onload="checkLibrary()"> <h1>Leap JavaScript Sample</h1> <div id="main"> <h3>Frame data:</h3> <div id="frameData"></div> <div style="clear:both;"></div> <h3>Hand data:</h3> <div id="handData"></div> <div style="clear:both;"></div> <h3>Finger and tool data:</h3> <div id="pointableData"></div> <div style="clear:both;"></div> <h3>Gesture data:</h3> <div id="gestureData"></div> </div> </body> </html>
To keep things simple, all the example code in the example is in the main HTML file.
Since the Leap provides data continuously, you need to set up an event loop to handle individual frames of data. You can either set up a loop that handles the data as the Leap produces each frame, or you can set up a loop that handles the data when your application is ready to do something with it. The Leap JavaScript client library supports both options.
In this example, I demonstrate the second method, since there is no need in this app to process the data faster than the browser can draw it to the screen. The Leap API provides the loop() function, which invokes a callback function you provide whenever the browser is ready to draw. In this sample, the callback is an anonymous function that prints out the tracking data to the body of the web page. The first parameter of the loop() function contains optional parameters that are passed to the Controller objet. This is the skeleton of the function:
// Setup Leap loop with frame callback function var controllerOptions = {enableGestures: true}; Leap.loop(controllerOptions, function(frame) { // Body of callback function })
Internally, Leap.loop() uses the browser's requestAnimationFrame() feature and passes the latest frame of Leap data on each browser animation cycle.
The Frame object passed to the loop() callback function as the frame parameter is root of the tracking data model and provides access to all the tracked objects, like hands, fingers, and tools. To display the data, I create a string of HTML containing the object properties and assign the string to a div element on the page.
Displaying the basic frame data is a simple matter of constructing a string using the Frame object properties:
var frameString = "Frame ID: " + frame.id + "<br />" + "Timestamp: " + frame.timestamp + " µs<br />" + "Hands: " + frame.hands.length + "<br />" + "Fingers: " + frame.fingers.length + "<br />" + "Tools: " + frame.tools.length + "<br />" + "Gestures: " + frame.gestures.length + "<br />";
Motion data, which the Leap provides for the scene as a whole and for each hand individually, requires a frame to serve as the basis for comparison. In this example, I store a reference to the previous frame (in a variable named previousFrame) and pass that to each motion function. The example displays the Frame translation, rotation axis, rotation angle, and scale factor; there are a few other methods available as well. The motion functions of the Frame object describe the motion of the overall scene. The example displays the motion data as follows:
// Frame motion factors if (previousFrame) { var translation = frame.translation(previousFrame); frameString += "Translation: " + vectorToString(translation) + " mm <br />"; var rotationAxis = frame.rotationAxis(previousFrame); var rotationAngle = frame.rotationAngle(previousFrame); frameString += "Rotation axis: " + vectorToString(rotationAxis, 2) + "<br />"; frameString += "Rotation angle: " + rotationAngle.toFixed(2) + " radians<br />"; var scaleFactor = frame.scaleFactor(previousFrame); frameString += "Scale factor: " + scaleFactor.toFixed(2) + "<br />"; }
When you access a value that represents a vector, the Leap JavaScript API expresses the vector as a standard Array object containing three elements. This is the case with the result of the translation() function, above. The element, translation[0], for example represents the x-coordinate of the vector. For convenience, I have defined the function vectorToString() to convert a vector to a human-readable format.
At the end of the callback function, remember to assign the current frame to the previousFrame variable to serve as the point of reference for motions in the next frame.
Hands can be accessed from the Frame object using the hands property, which is an array. The example code iterates through the list of hands and prints out the hand properties, IDs for associated fingers and tools, and the motion data for each hand.
// Display Hand object data var handString = ""; if (frame.hands.length > 0) { for (var i = 0; i < frame.hands.length; i++) { var hand = frame.hands[i]; handString += "Hand ID: " + hand.id + "<br />"; handString += "Direction: " + vectorToString(hand.direction, 2) + "<br />"; handString += "Palm normal: " + vectorToString(hand.palmNormal, 2) + "<br />"; handString += "Palm position: " + vectorToString(hand.palmPosition) + " mm<br />"; handString += "Palm velocity: " + vectorToString(hand.palmVelocity) + " mm/s<br />"; handString += "Sphere center: " + vectorToString(hand.sphereCenter) + " mm<br />"; handString += "Sphere radius: " + hand.sphereRadius.toFixed(1) + " mm<br />"; // And so on... } }
Fingers and tools are represented by Pointable objects in the JavaScript API. (You can use the Pointable.tool property to tell which type an object represents.) The Frame object provides a pointables property, which is an array containing all the fingers and tools detected in that frame. Hand objects also contain a pointables list, which contains only the pointable objects associated with that hand.
// Display Pointable (finger and tool) object data var pointableString = ""; if (frame.pointables.length > 0) { for (var i = 0; i < frame.pointables.length; i++) { var pointable = frame.pointables[i]; pointableString += "Pointable ID: " + pointable.id + "<br />"; pointableString += "Belongs to hand with ID: " + pointable.handId + "<br />"; if (pointable.tool) { pointableString += "Classified as a tool <br />"; pointableString += "Length: " + pointable.length.toFixed(1) + " mm<br />"; pointableString += "Width: " + pointable.width.toFixed(1) + " mm<br />"; } else { pointableString += "Classified as a finger<br />"; pointableString += "Length: " + pointable.length.toFixed(1) + " mm<br />"; } pointableString += "Direction: " + vectorToString(pointable.direction, 2) + "<br />"; pointableString += "Tip position: " + vectorToString(pointable.tipPosition) + " mm<br />"; pointableString += "Tip velocity: " + vectorToString(pointable.tipVelocity) + " mm/s<br />"; } }
To receive gestures from the Leap, you first have to enable gesture recognition. Enable gestures when you call the Leap.loop() function by passing in an options object:
// Setup Leap loop with frame callback function var controllerOptions = {enableGestures: true}; Leap.loop(controllerOptions, function(frame) { // Body of callback function })
The Leap adds Gesture objects representing each recognized movement pattern to the gestures list in the Frame object. The sample application loops through the gesture list and prints information about each one to the standard output. This operation is performed with a standard for-loop and switch statement.
The full code for the gesture loop is:
// Display Gesture object data var gestureString = ""; if (frame.gestures.length > 0) { for (var i = 0; i < frame.gestures.length; i++) { var gesture = frame.gestures[i]; gestureString += "Gesture ID: " + gesture.id + ", " + "type: " + gesture.type + ", " + "state: " + gesture.state + ", " + "hand IDs: " + gesture.handIds.join(", ") + ", " + "pointable IDs: " + gesture.pointableIds.join(", ") + ", " + "duration: " + gesture.duration + " µs, "; switch (gesture.type) { case "circle": gestureString += "center: " + vectorToString(gesture.center) + " mm, " + "normal: " + vectorToString(gesture.normal, 2) + ", " + "radius: " + gesture.radius.toFixed(1) + " mm, " + "progress: " + gesture.progress.toFixed(2) + " rotations"; break; case "swipe": gestureString += "start position: " + vectorToString(gesture.startPosition) + " mm, " + "current position: " + vectorToString(gesture.position) + " mm, " + "direction: " + vectorToString(gesture.direction, 2) + ", " + "speed: " + gesture.speed.toFixed(1) + " mm/s"; break; case "screenTap": case "keyTap": gestureString += "position: " + vectorToString(gesture.position) + " mm, " + "direction: " + vectorToString(gesture.direction, 2); break; default: gestureString += "unkown gesture type"; } gestureString += "<br />"; } }
To run the sample application:
When you place a hand above the Leap, you should see the Frame, Hand, and Finger information printed out on the page. When you make a gesture, you should also see the appropriate Gesture information.
Now that you have seen how to access motion tracking data from the Leap, you can begin developing your own JavaScript applications that integrate the Leap.
Copyright © 2012-2013 Leap Motion, Inc. All rights reserved.
Leap Motion proprietary and confidential. Not for distribution. Use subject to the terms of the Leap Motion SDK Agreement available at https://developer.leapmotion.com/sdk_agreement, or another agreement between Leap Motion and you, your company or other organization.