Three-dimensional interface

Patent metadata

Field	Value
Publication number	US-9684427-B2
Application number	US-201414322952-A
Country	US
Kind code	B2
Filing date	Jul 3, 2014
Priority date	Dec 9, 2005
Publication date	Jun 20, 2017
Grant date	Jun 20, 2017

Abstract

A three-dimensional virtual-touch human-machine interface system ( 20 ) and a method ( 100 ) of operating the system ( 20 ) are presented. The system ( 20 ) incorporates a three-dimensional time-of-flight sensor ( 22 ), a three-dimensional autostereoscopic display ( 24 ), and a computer ( 26 ) coupled to the sensor ( 22 ) and the display ( 24 ). The sensor ( 22 ) detects a user object ( 40 ) within a three-dimensional sensor space ( 28 ). The display ( 24 ) displays an image ( 42 ) within a three-dimensional display space ( 32 ). The computer ( 26 ) maps a position of the user object ( 40 ) within an interactive volumetric field ( 36 ) mutually within the sensor space ( 28 ) and the display space ( 32 ), and determines when the positions of the user object ( 40 ) and the image ( 42 ) are substantially coincident. Upon detection of coincidence, the computer ( 26 ) executes a function programmed for the image ( 42 ).

First claim

The invention claimed is:

1. A three-dimensional interface system, comprising:
one or more three-dimensional time-of-flight object sensors configured to detect a position of a user object within a first three-dimensional space;
a three-dimensional display configured to display one or more images within a second three-dimensional space bounded by display limits, individual images of the one or more images being displayed as if in three-dimensional space, an individual images of the one or more images having a breadth corresponding to a size in an x-dimension, a height corresponding to a size in a y-dimension, and a depth corresponding to a size in a z-dimension, and the individual images of the one or more images having at least one corresponding active region being defined by x, y and z coordinates programmed to be interactive with a user object; and
a computer coupled to said sensor and said display, configured to map said position of said user object within a third three-dimensional space including x, y and z axis coordinates wherein the third three-dimensional space is common to both the first three-dimensional space and the second three-dimensional space, configured to map a position of said one or more active regions of said corresponding one or more images within said third three-dimensional space, and configured to determine whether said positions of said user object and one or more of said active regions of said corresponding one or more images are substantially coincident within said third three-dimensional space, so that the computer can differentiate between two or more of said active regions differing only in z axis coordinates in said third three-dimensional space.

2. The system as claimed in claim 1 wherein said three-dimensional display is a three-dimensional autostereoscopic display.

3. The system as claimed in claim 1 wherein said three-dimensional display is a three-dimensional autostereoscopic liquid crystal display.

4. The system as claimed in claim 1 wherein said three-dimensional display is a three-dimensional autostereoscopic plasma-screen display.

5. The system of claim 1 wherein the three-dimensional display provides a different image for each eye of a viewer.

6. The system as claimed in claim 1 wherein said three-dimensional display is configured to display said one or more active regions of said one or more images in said first three-dimensional space.

7. The system as claimed in claim 1 wherein said third three-dimensional space is mutually within said first and second three-dimensional spaces.

8. The system as claimed in claim 1 wherein said first three-dimensional space is bound by first limits; said second three-dimensional space is bound by second limits; said third three-dimensional space is bound by said first and second limits.

9. The system as claimed in claim 8 wherein said third three-dimensional space is unbounded in a direction away from said display.

10. The system as claimed in claim 1 wherein said one or more images is one of a plurality of images; and said display is configured to display said plurality of images within said second three-dimensional space so that a user interacting with the three-dimensional display is able to view the plurality of images as if they were in three-dimensional space.

11. The system as claimed in claim 1 wherein said user object is one of a plurality of user objects; and said sensor is configured to detect said one user object within said first three-dimensional space.

12. The system as claimed in claim 1 wherein said user object is a secondary object.

13. The system as claimed in claim 12 , wherein said secondary object is separate from a portion of a user.

14. A method of determining virtual touch within a three-dimensional interface system, comprising the operation of:
displaying, through the use of a three-dimensional display, one or more images within a first three-dimensional space so that the one or more images are displayed as if positioned in the first three-dimensional space, wherein individual images of the one or more images have a breadth corresponding to a size in an x-dimension, a height corresponding to a size in a y-dimension, and a depth corresponding to a size in a z-dimension, and the individual images of the one or more images having at least one corresponding active region defined by x, y and z coordinates programmed to be interactive with a user object;
detecting a presence of a user object including x, y and z axis coordinates, within a second three-dimensional space through the use of one or more three-dimensional time-of-flight sensors;
mapping, through a computer, within a third three-dimensional space common to both the first three-dimensional space and the second three-dimensional space, a position of the user object in the second three-dimensional space and a position of the at least one corresponding active region of the one or more images within the first three-dimensional space; and
determining, by the computer, whether the position of the user object is substantially coincident with the position of the at least one corresponding active region of the one or more images along each of the x, y and z axis coordinates within the third three-dimensional space so that the computer can differentiate between two or more of said active regions differing only in z axis coordinates in said third three-dimensional space.

15. The method as claimed in claim 14 , further comprising:
returning to said detecting activity following said determining activity.

16. The method as claimed in claim 14 wherein said mapping and determining activities are executed responsive to detecting said presence of said user object; and said detecting activity is iterated responsive to detecting no presence of said user object.

17. The method as claimed in claim 14 , further comprising returning to said detecting activity responsive to determining said user object is not substantially coincident with said active region.

18. The method as claimed in claim 14 , further comprising:
responsive to determining that the user object is substantially coincident with the position of the at least one corresponding active region of the one or more images, determining whether a function programmed for the at least one corresponding active region of an individual image of the one or more images is to be executed;
responsive to determining that the function programmed for the at least one corresponding active region of the one or more images is to be executed, executing the function; and
responsive to executing the function, updating the individual image to display an updated image in the first three-dimensional space.

19. The method as claimed in claim 18 , further comprising: returning to said detecting activity responsive to ascertaining said function is not to be executed.

20. The method as claimed in claim 14 , further comprising establishing said position of said user object prior to said mapping activity.

21. The method as claimed in claim 14 , further comprising:
displaying the one or more images containing at least one image, wherein a plurality of active regions of said at least one image is in said first three-dimensional space;
detecting a presence of one or more user objects within said second three-dimensional space;
mapping said position of each of said one or more user objects within said third three-dimensional space; and
determining whether one or more of said one or more user objects is substantially coinci

Assignees

• Microsoft Technology Licensing Llc

Inventors

• El Dokor Tarek
• King Joshua T
• Holmes James E
• Glomski William E
• Ngomba Maria N

Classifications

• G06F3/04845

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title

• G06F3/04815 Primary

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title