LeapScreen Location
A user can register the position and orientation of their monitor
screens using the Leap application. When screen locations are known, you
can use the LeapScreen class to get the physical position and size
of that screen in terms of the Leap coordinate system and to find where
on the screen a finger or tool is pointing.
Registering the screen location with the Screen Locator utility is not
needed for most functions of the Leap to work properly. Registration is
required only when your application uses the LeapScreen class.
Overview
The
screen within the Leap frame of reference. The
[LeapScreen intersect:normalize:clampRatio:]
function calculates the position of a point at the intersection between
the screen plane and a ray emanating from a finger or tool. Likewise, the
[LeapScreen intersect:direction:normalize:clampRatio:]
function calculates the position of a point at the intersection between
the screen plane and a a ray projected in an arbitrary direction from an arbitrary point.
Additionally, the [LeapScreen project:normalize:clampRatio:] function calculates the position
of a point projected orthogonally onto the screen plane.
Before using the LeapScreen class, however, consider that registering the
screen location takes a certain amount of effort by the user and that
the registered location is only valid until the user moves the monitor
or the Leap. Your application’s use of the API must be sufficiently
valuable that the user feels their effort is worthwhile. We recommend
that you make it clear to your users why you need them to register their
screen locations and that you provide a quick test for the user to check
whether the screen location is still accurate.
For instructions on using the Screen Locator utility, see Screen
location.
Screen coordinates
For each known screen, the LeapScreen class provides an origin point and
vectors representing the horizontal and vertical extents of the screen.
A normal vector is also provided.
The screen origin is placed at the bottom, left-hand corner of the
screen. The axis vectors are parallel to the corresponding side of the
screen. The length of these vectors represent the length of the screen
in millimeters. The normal vector is a unit direction vector
perpendicular to the surface of the screen.
Intersection and projection points
The LeapScreen class provides two intersection functions for calculating the
point on the screen given either a pointable object or an arbitrary 3D point
and direction. The functions projects a ray along the given direction and determine
where that ray intersects the plane of the screen. It returns a valid value as long as
the ray points toward the plane in which the screen lies. If the pointer is parallel
or pointing away from the screen, then the coordinates contained in the intersection
vector are all NaN (not-a-number).
- Pointer A intersects the screen plane outside the physical screen
bounds - Pointer B intersects the screen directly
- Pointer C does not intersect the screen plane
Additionally, the LeapScreen class provides a projection function for
calculating the location on the screen where a point is projected onto
its plane. Unlike the intersection functions, the projection function
always returns a valid value, as a point can always be projected onto a
plane.
- Point A is projected onto the screen directly
- Point B is projected onto the screen plane outside the physical
screen bounds
You can request the intersection and projection points in terms of the
Leap coordinate system. The coordinates returned form a 3D Vector
representing the point in space measured in millimeters from the Leap
origin. Alternately, you can request 2D, normalized coordinates, which
define the intersection and projection point relative to the bottom,
left-hand corner of the screen and within the plane of the screen.
When using normalized coordinates, the origin point is (0, 0, 0) and the
opposite, top, right-hand corner is (1, 1, 0). The x and y values for
points within the screen plane are always between 0 and 1 and represent
the ratio of that point between one side of the screen and another (z
values are always zero when using normalized coordinates). For example,
the point at the center of the screen has normalized coordinates (0.5,
0.5, 0). Normalized coordinates for intersection and projection points
outside the screen border can be less than 0 or greater than 1.
By default, the intersection and projection functions clamp the
normalized coordinates to the range (0..1, 0..1, 0). This means that any
intersection or projection points outside the screen border are
effectively moved onto the border. You can change the default clamping
area by setting the clampRatio
parameter when calling one of the
intersection or projection functions. The default clamp ratio is 1.0.
Use a smaller value to decrease the clamping area, and a larger value to
increase the area. For example, setting clampRatio
to 0.5 limits the
interaction area to the center 50% of the screen and clamps all point
coordinates between 0.25 and 0.75. Likewise, setting clampRatio
to 2.0
doubles the interaction area and clamps point coordinates between -0.5
and 1.5. The resulting interaction area is always centered on the
screen’s center.
Find the closest screen
You can get a list of all screens with registered locations from a
LeapController instance:
- (void)onFrame:(NSNotification *)notification
{
LeapController *controller = (LeapController *)[notification object];
NSArray *screens = [controller calibratedScreens];
}
Note that the screen list always contains at least one entry, even if
the user has never registered the location of a screen. This entry
represents the user’s default screen.
You can get the closest screen that a ray projected along a finger or
tool will intersect from the NSArray containing the calibrated screen list
using NSArray functions defined in the NSArray(LeapScreenList) category:
LeapFrame *frame = [controller frame:0];
if ([frame.pointables count] > 0) {
LeapPointable *pointable = [frame.pointables objectAtIndex:0];
NSArray *screens = controller.calibratedScreens;
LeapScreen *screen = [screens closestScreenHit:pointable];
NSLog(@"ClosestScreenHit ID: %d; Size: %d, %d", screen.id, screen.widthPixels, screen.heightPixels);
}
Similarly, you can get the closest screen to a point (such as a finger
or tool’s tip position):
LeapFrame *frame = [controller frame:0];
if ([frame.pointables count] > 0) {
LeapPointable *pointable = [frame.pointables objectAtIndex:0];
NSArray *screens = controller.calibratedScreens;
LeapScreen *screen = [screens closestScreen:pointable.tipPosition];
NSLog(@"ClosestScreen ID: %d; Size: %d, %d", screen.id, screen.widthPixels, screen.heightPixels);
}
Find the pixel coordinates of an intersection point
You can request normalized coordinates from the [LeapScreen intersect:normalize:clampRatio:]
and [LeapScreen intersect:direction:normalize:clampRatio:] functions
to help calculate the on-screen, pixel coordinates of an intersection point:
LeapFrame *frame = [controller frame:0];
if ([frame.pointables count] > 0) {
LeapPointable *pointable = [frame.pointables objectAtIndex:0];
NSArray *screens = controller.calibratedScreens;
LeapScreen *screen = [screens closestScreenHit:pointable];
LeapVector *normalizedCoordinates = [screen intersect:pointable normalize:YES clampRatio:2.0];
int xPixel = normalizedCoordinates.x * screen.widthPixels;
int yPixel = screen.heightPixels - normalizedCoordinates.y * screen.heightPixels;
NSLog(@"Normalized coordinates: (%d, %d)", xPixel, yPixel);
}
Find the Leap coordinates of a projection point
You can find the coordinates of a point projected onto the screen plane
using the [LeapScreen project:normalize:clampRatio:] function. For example, you may want to project a
finger or tool’s position onto the closest screen, ignoring its
direction:
LeapFrame *frame = [controller frame:0];
if ([frame.pointables count] > 0) {
LeapPointable *pointable = [frame.pointables objectAtIndex:0];
NSArray *screens = controller.calibratedScreens;
LeapScreen *screen = [screens closestScreen:pointable.tipPosition];
LeapVector *leapCoordinates = [screen project:pointable.tipPosition normalize:NO clampRatio:1.0];
NSLog(@"Leap coordinates: (%f, %f, %f)", leapCoordinates.x, leapCoordinates.y, leapCoordinates.z);
}
Set the interaction area larger or smaller
When using normalized coordinates, you make the area in which the user
can point larger or smaller by setting the clampRatio
parameter when
calling the [LeapScreen intersect:normalize:clampRatio:] or
[LeapScreen intersect:direction:normalize:clampRatio:] functions. By
default, the clampRatio
parameter is 1.0, which corresponds to the
bounds of the screen. Any points outside these bounds will be clamped to
the border when the point coordinates are returned. (Points within the
bounds of the interaction area are unaffected by clamping.)
To halve the interaction area, set the clampRatio
parameter to 0.5
when you call the intersect()
function:
LeapVector *normalizedCoordinates = [screen intersect:pointable normalize:YES clampRatio:0.5];
To double the area, double the clampRatio
:
LeapVector *normalizedCoordinates = [screen intersect:pointable normalize:YES clampRatio:2.0];
Find the distance between the pointing finger or tool and the screen
To get the distance between the pointing finger or tool and the screen
intersection point, simply subtract the two Vector objects representing
these points and get the magnitude of the result. When calling the
[LeapScreen intersect:normalize:clampRatio:] or [LeapScreen intersect:direction:normalize:clampRatio:]
functions to get the intersection point, set the normalized
parameter to false:
LeapFrame *frame = [controller frame:0];
if ([frame.pointables count] > 0) {
LeapPointable *pointable = [frame.pointables objectAtIndex:0];
NSArray *screens = controller.calibratedScreens;
LeapScreen *screen = [screens closestScreenHit:pointable];
LeapVector *leapCoordinates = [screen intersect:pointable normalize:NO clampRatio:0.0];
LeapVector *tipToScreen = [leapCoordinates minus:pointable.tipPosition];
float pointingDistance = tipToScreen.magnitude;
NSLog(@"Distance: %f", pointingDistance);
}
To get the shortest distance between the pointing finger and tool and
the screen plane, that is, the distance of a perpendicular line segment
from the plane to the pointable tip, use the [LeapScreen distanceToPoint:]
function passing in the tip position:
LeapFrame *frame = [controller frame:0];
if ([frame.pointables count] > 0) {
LeapPointable *pointable = [[frame pointables] objectAtIndex:0];
NSArray *screens = [controller calibratedScreens];
LeapScreen *screen = [screens closestScreenHit:pointable];
float normalDistance = [screen distanceToPoint:pointable.tipPosition];
NSLog(@"Normal distance: %f", normalDistance);
}
Copyright © 2012-2013 Leap Motion, Inc. All rights reserved.
Leap Motion proprietary and confidential. Not for distribution. Use
subject to the terms of the Leap Motion SDK Agreement available at
https://developer.leapmotion.com/sdk_agreement, or another agreement
between Leap Motion and you, your company or other organization.