A user can register the position and orientation of their monitor screens using the Leap Motion Settings dialog. When screen locations are known, you can use the Leap Motion API Screen class to get the physical position and size of that screen in terms of the Leap Motion coordinate system and to find where on the screen a finger or tool is pointing.
Registering the screen location with the Screen Locator utility is not needed for most functions of the Leap Motion system to work properly. Registration is required only when your application uses the Leap::Screen class.
Note: The Screen Locator tool and Screen location API are not currently supported on Linux.
Topics:
The Screen class provides the position and orientation of a display screen within the Leap Motion frame of reference. The Screen::intersect() function calculates the position of a point at the intersection between the screen plane and a ray emanating from a finger or tool. Additionally, the Screen::project() function calculates the position of a point projected onto the screen plane.
Before using the Screen class, however, consider that registering the screen location takes a certain amount of effort by the user and that the registered location is only valid until the user moves the monitor or the Leap. Your application's use of the API must be sufficiently valuable that the user feels their effort is worthwhile. We recommend that you make it clear to your users why you need them to register their screen locations and that you provide a quick test for the user to check whether the screen location is still accurate.
For instructions on using the Screen Locator utility, see Screen location.
For each known screen, the Screen class provides an origin point and vectors representing the horizontal and vertical extents of the screen. A normal vector is also provided.
The origin and orientation vectors of a located screen.
The screen origin is placed at the bottom, left-hand corner of the screen. The axis vectors are parallel to the corresponding side of the screen. The length of these vectors represent the length of the screen in millimeters. The normal vector is a unit direction vector perpendicular to the surface of the screen.
The Screen class provides an intersect() function for calculating the point on the screen where a finger or tool is pointing. The function projects a ray from the Pointable object and determines where that ray intersects the plane of the screen. It returns a valid value as long as the Pointable object is pointing toward the plane in which the screen lies. If the pointer is parallel or pointing away from the screen, then the coordinates contained in the intersection vector are all NaN (not-a-number).
The Screen class also provides a similar intersect() function that accepts a position and direction vector and calculates the screen intersection point of the specified ray, which does not necessarily represent a Pointable object.
Three pointers with corresponding intersection points.
Additionally, the Screen class provides a project() function for calculating the location on the screen where a point is projected onto its plane. Unlike the intersect() function, the project() function always returns a valid value, as a point can always be projected onto a plane.
Two points with corresponding projection points.
You can request the intersection and projection points in terms of the Leap Motion coordinate system. The coordinates returned form a 3D Vector representing the point in space measured in millimeters from the Leap Motion origin. Alternately, you can request 2D, normalized coordinates, which define the intersection and projection point relative to the bottom, left-hand corner of the screen and within the plane of the screen.
When using normalized coordinates, the origin point is (0, 0, 0) and the opposite, top, right-hand corner is (1, 1, 0). The x and y values for points within the screen plane are always between 0 and 1 and represent the ratio of that point between one side of the screen and another (z values are always zero when using normalized coordinates). For example, the point at the center of the screen has normalized coordinates (0.5, 0.5, 0). Normalized coordinates for intersection and projection points outside the screen border can be less than 0 or greater than 1.
By default, the intersect() and project() functions clamp the normalized coordinates to the range (0..1, 0..1, 0). This means that any intersection or projection points outside the screen border are effectively moved onto the border. You can change the default clamping area by setting the clampRatio parameter when calling the intersect() or project() functions. The default clamp ratio is 1.0. Use a smaller value to decrease the clamping area, and a larger value to increase the area. For example, setting clampRatio to 0.5 limits the interaction area to the center 50% of the screen and clamps all point coordinates between 0.25 and 0.75. Likewise, setting clampRatio to 2.0 doubles the interaction area and clamps point coordinates between -0.5 and 1.5. The resulting interaction area is always centered on the screen's center.
You can get a list of all screens with registered locations from a Controller instance:
Controller controller; ScreenList screens = controller.calibratedScreens();
Note that the screen list always contains at least one entry, even if the user has never registered the location of a screen. This entry represents the user's default screen.
You can get the closest screen that a ray projected along a finger or tool will intersect from the ScreenList object:
Controller controller; Frame frame = controller.frame(); if (frame.pointables().count() > 0) { Pointable pointable = frame.pointables()[0]; ScreenList screens = controller.calibratedScreens(); Screen screen = screens.closestScreenHit(pointable); }
Similarly, you can get the closest screen to a point (such as a finger or tool's tip position) from the ScreenList object:
Controller controller; Frame frame = controller.frame(); if (frame.pointables().count() > 0) { Pointable pointable = frame.pointables()[0]; ScreenList screens = controller.calibratedScreens(); Screen screen = screens.closestScreen(pointable.tipPosition()); }
You can request normalized coordinates from the intersect() function to help calculate the on-screen, pixel coordinates of an intersection point:
Controller controller; Frame frame = controller.frame(); if (frame.pointables().count() > 0) { Pointable pointable = frame.pointables()[0]; ScreenList screens = controller.calibratedScreens(); Screen screen = screens.closestScreenHit(pointable); Vector normalizedCoordinates = screen.intersect(pointable, true); int xPixel = (int)(normalizedCoordinates.x * screen.widthPixels()); int yPixel = screen.heightPixels() - (int)(normalizedCoordinates.y * screen.heightPixels()); }
You can find the coordinates of a point projected onto the screen plane using the project() function. For example, you may want to project a finger or tool's position onto the closest screen, ignoring its direction:
Controller controller; Frame frame = controller.frame(); if (frame.pointables().count() > 0) { Pointable pointable = frame.pointables()[0]; ScreenList screens = controller.calibratedScreens(); Screen screen = screens.closestScreen(pointable.tipPosition()); Vector screenProjection = screen.project(pointable.tipPosition(), false); }
When using normalized coordinates, you make the area in which the user can point larger or smaller by setting the clampRatio parameter when calling the intersect() function. By default, the clampRatio parameter is 1.0, which corresponds to the bounds of the screen. Any points outside these bounds will be clamped to the border when the point coordinates are returned. (Points within the bounds of the interaction area are unaffected by clamping.)
To halve the interaction area, set the clampRatio parameter to 0.5 when you call the intersect() function:
Vector normalizedCoordinates = screen.intersect(pointable, true, 0.5);
To double the area, double the clampRatio:
Vector normalizedCoordinates = screen.intersect(pointable, true, 2.0);
To get the distance between the pointing finger or tool and the screen intersection point, simply subtract the two Vector objects representing these points and get the magnitude of the result. When calling the intersect() function to get the intersection point, set the normalized parameter to false:
Controller controller; Frame frame = controller.frame(); if (frame.pointables().count() > 0) { Pointable pointable = frame.pointables()[0]; ScreenList screens = controller.calibratedScreens(); Screen screen = screens.closestScreenHit(pointable); Vector intersection = screen.intersect(pointable, false); Vector tipToScreen = intersection - pointable.tipPosition(); float pointingDistance = tipToScreen.magnitude(); }
To get the shortest distance between the pointing finger and tool and the screen plane, that is, the distance of a perpendicular line segment from the plane to the pointable tip, use the distanceToPoint() function passing in the tip position:
Controller controller; Frame frame = controller.frame(); if (frame.pointables().count() > 0) { Pointable pointable = frame.pointables()[0]; ScreenList screens = controller.calibratedScreens(); Screen screen = screens.closestScreenHit(pointable); float perpendicularDistance = screen.distanceToPoint(pointable.tipPosition()); }
Copyright © 2012-2013 Leap Motion, Inc. All rights reserved.
Leap Motion proprietary and confidential. Not for distribution. Use subject to the terms of the Leap Motion SDK Agreement available at https://developer.leapmotion.com/sdk_agreement, or another agreement between Leap Motion and you, your company or other organization.