Remote-controlled pointing

Patent metadata

Field	Value
Publication number	US-9563293-B2
Application number	US-93689908-A
Country	US
Kind code	B2
Filing date	Apr 9, 2008
Priority date	Apr 9, 2008
Publication date	Feb 7, 2017
Grant date	Feb 7, 2017

Abstract

A remote-controlled pointing system ( 32 ) includes a projector ( 36 ), an image sensor ( 34 ), and a controller ( 38 ). The projector ( 36 ) projects a beam ( 45 ) of light from a location in a projection plane ( 46 ) onto a scene ( 18 ). The image sensor ( 34 ) captures an image ( 54 ) of the scene ( 18 ) in a capture plane ( 50 ). The controller ( 38 ) performs operations that include transmitting the captured image ( 54 ) to a remote destination ( 14 ), receiving from the remote destination ( 14 ) a target beam location ( 44 ) in the capture plane ( 50 ), determining a location of the beam ( 45 ) in the capture plane ( 50 ), and changing the location in the projection plane ( 46 ) from which the beam ( 45 ) is projected based on the determined beam location and the target beam location ( 44 ).

First claim

What is claimed is:

1. Apparatus, comprising:
a projector operable to project a beam of light from a location in a projection plane onto a scene;
an image sensor operable to capture an image of the scene in a capture plane;
a controller operable to perform operations comprising transmitting the captured image to a remote destination, receiving from the remote destination a target beam location in the capture plane, determining a location of the beam in the capture plane, and changing the location in the projection plane from which the beam is projected based on the determined beam location and the target beam location.

2. The apparatus of claim 1 , wherein the controller additionally is operable to direct the projector to intermittently project the beam onto the scene, and in the determining the controller is operable to determine the location of the beam in the capture plane based on an analysis of a set of images of the scene captured by the image sensor.

3. The apparatus of claim 2 , wherein in the determining of the location of the beam in the capture plane the controller performs operations comprising: spatially filtering the images in the set; identifying one or more locations in the capture plane in which differences between the spatially filtered images meet a detection threshold predicate; and ascertaining a central point in the capture plane about which the identified locations are distributed.

4. The apparatus of claim 3 , wherein in the determining of the location of the beam in the capture plane the controller repeats the filtering, the identifying, and the ascertaining for one or more additional sets of images of the scene captured by the image sensor; and the controller determines the location of the beam in the capture plane based on a temporal filtering of the ascertained central points.

5. The apparatus of claim 1 , wherein in the changing the controller changes the location in the projection plane from which the beam is projected in response to a difference between the determined beam location and the target beam location.

6. The apparatus of claim 1 , wherein in the changing the controller performs operations comprising incrementally adjusting the location in the projection plane from which the beam is projected until a difference between the determined beam location and the target beam location meets an adjustment threshold predicate.

7. The apparatus of claim 6 , wherein in the adjusting the controller calculates a new location (x proj (t+1),y proj (t+1)) in the projection plane from which to project the beam as follows
x proj ( t+ 1)= x proj ( t )−α·( x cap detected ( t )− x cap target ( t ))

y proj ( t+ 1)= y proj ( t )−α·( y cap detected ( t )− y cap target ( t ))

where (x proj (t),y proj (t)) is the location in the projection plane from which the beam currently is being projected, (x cap detected (t),y cap detected (t)) is the determined beam location, (x cap target (t),y cap target (t)) is the target beam location, and α is an adjustment factor.

8. The apparatus of claim 7 , wherein the adjustment factor α(t) is given by
α( t )= L ·√{square root over (( x proj ( t )− x cap detected ( t )) 2 +( y proj ( t )− y cap detected ( t )) 2 )}

where L is a constant.

9. The apparatus of claim 1 , wherein the controller performs the transmitting and the receiving in a first process loop, and the controller performs the determining and the changing in a second process loop.

10. The apparatus of claim 9 , wherein the controller performs the transmitting, the receiving, the determining, and the changing as asynchronous events.

11. The apparatus of claim 1 , wherein the image sensor, the projector, and the controller are contained in a handheld device body.

12. A machine-implemented method, comprising:
projecting a beam of light from a location in a projection plane onto a scene;
capturing an image of the scene in a capture plane;
transmitting the captured image to a remote destination;
receiving from the remote destination a target beam location in the capture plane;
determining a location of the beam in the capture plane; and
changing the location in the projection plane from which the beam is projected based on the determined beam location and the target beam location.

13. The method of claim 12 , further comprising intermittently projecting the beam onto the scene, and wherein the determining comprises determining the location of the beam in the capture plane based on an analysis of a set of images of the scene captured by the image sensor.

14. The method of claim 13 , wherein the determining comprises: spatially filtering the images in the set; identifying one or more locations in the capture plane in which differences between the spatially filtered images meet a detection threshold predicate; and ascertaining a central point in the capture plane about which the identified locations are distributed.

15. The method of claim 14 , wherein the determining comprises repeating the filtering, the identifying, and the ascertaining for one or more additional sets of images of the scene captured by the image sensor; and the determining additionally comprises determining the location of the beam in the capture plane based on a temporal filtering of the ascertained central points.

16. The method of claim 12 , wherein the changing comprises changing the location in the projection plane from which the beam is projected in response to a difference between the determined beam location and the target beam location.

17. The method of claim 12 , wherein the changing comprises incrementally adjusting the location in the projection plane from which the beam is projected until a difference between the determined beam location and the target beam location meets an adjustment threshold predicate.

18. The method of claim 17 , wherein the adjusting comprises calculating a new location (x proj (t+1),y proj (t+1)) in the projection plane from which to project the beam as follows
x proj ( t+ 1)= x proj ( t )−α·( x cap detected ( t )− x cap detected ( t ))

y proj ( t+ 1)= y proj ( t )−α·( y cap detected ( t )− y cap detected ( t ))

where (x proj (t),y proj (t)) is the location in the projection plane from which the beam currently is being projected, (x cap detected (t),y cap detected (t)) is the determined beam location, (x cap target (t),y cap target (t)) is the target beam location, and α is an adjustment factor.

19. The method of claim 18 , wherein the changing comprises dynamically determining a value of the adjustment factor α(t).

20. The method of claim 12 , further comprising performing the transmitting and the receiving in a first process loop, and performing the determining and the changing in a second process loop.

21. The method of claim 20 , wherein the transmitting, the receiving, the determining, and the changing are performed as asynchronous events.

22. A non-transitory computer-readable medium storing computer-readable instructions causing a computer to perform operations comprising:
projecting a beam of light from a location in a projection plane onto a scene;
capturing an image of the scene in a capture plane;
transmitting the captured image to a remote destination;
receiving from the remote destination a target beam location in the capture plane;
determining a location of the beam in the capture plane; and
changing the location in the

Assignees

• Ulichney Robert Alan
• Gaubatz Matthew D
• Hewlett Packard Development Co Lp

Inventors

• Ulichney Robert Alan
• Gaubatz Matthew D

Classifications

• G06F3/0386 Primary

  for light pen · CPC title

• G06F3/0304 Primary

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title