Wearable gestural interface

Patent metadata

Field	Value
Publication number	US-9569001-B2
Application number	US-69980310-A
Country	US
Kind code	B2
Filing date	Feb 3, 2010
Priority date	Feb 3, 2009
Publication date	Feb 14, 2017
Grant date	Feb 14, 2017

Abstract

This invention may be implemented as a wearable apparatus comprised of a camera, a projector, a mirror, a microphone and a digital computer. The camera captures visual data. This data is analyzed by the digital computer to recognize objects and hand gestures, using color tracking and edge detection techniques. The projector is used, along with a mirror to adjust the direction of the projected light, to project images on objects in the user's environment. For example, the images may be projected on surfaces such as a wall, table, or piece of paper. The projected images may contain information relevant to the object being augmented. Indeed, the information may include current data obtained from the Internet. Also, the projected images may comprise graphical interfaces, with which a user may interact by making hand gestures.

First claim

What is claimed is:

1. Apparatus comprising:
(a) a camera;
(b) a projector for projecting images on a surface, which surface is separate from the apparatus and is not dedicated to electronic visual display;
(c) a transceiver for connecting to a global network; and
(d) a digital computer for
(i) performing analysis of visual data captured by the camera (A) to identify a solid object or a feature of the solid object, neither the object nor the feature being an image projected by the projector, and (B) to recognize a gesture made by a human in the air while the human is neither in physical contact with, nor holding an object that is in physical contact with, a solid surface, and
(ii) based at least in part on the analysis:
outputting instructions to obtain, from the global network, information about the object or feature;
receiving and processing data indicative of the information, and
outputting instructions to make changes to the images, in a way that the changes depend at least in part on the information and the gesture;

wherein
(A) the global network comprises the Internet, the World-Wide Web, or any other global system of interconnected computer networks, and

(B) the apparatus is wearable in a manner in which weight of the apparatus is not supported by a human user's hands.

2. Apparatus as set forth in claim 1 , wherein the projector is adapted for projecting images which convey the information.

3. Apparatus as set forth in claim 1 , wherein the computer is adapted for outputting signals indicative of audio output of a transducer, in such a way that the transducer outputs sound that conveys the information audibly to a user.

4. Apparatus as set forth in claim 1 , wherein:
(a) the feature is the surface of a product or of the product's packaging;
(b) the information is about the product; and
(c) the projector is adapted for projecting, on the surface, images that convey the information.

5. Apparatus as set forth in claim 1 , wherein:
(a) the feature comprises text or a graphical pattern, which text or graphical patttern is not an image projected by the projector, and which text or graphical pattern has a subject;
(b) the information is about the subject; and
(c) the projector is adapted for projecting, on the surface, images that convey the information.

6. Apparatus as set forth in claim 1 , wherein the computer is adapted for analyzing the visual data to identify a specific person, and further adapted for outputting instructions for altering audio or visual output of a transducer in such a way as to convey the information, and the information is about the specific person.

7. Apparatus as set forth in claim 1 , wherein the computer is adapted for outputting instructions to project an image of a graphical user interface, and further adapted for analyzing the visual data captured by the camera to detect movements comprising a user's interaction with the projected image of the graphical user interface, wherein the graphical user interface displays a plurality of graphical icons or other visual symbols, each of which represent a different input that may be selected by a gesture made by the user.

8. Apparatus as set forth in claim 1 , wherein the computer is adapted for outputting instructions to project images that display an email message obtained through the global network.

9. Apparatus comprising:
(a) a projector for projecting images on at least one surface, which surface is not part of the apparatus or any other electronic visual display device; and
(b) a camera for
(i) capturing visual data, and
(ii) outputting signals indicative of the visual data, for transmission to one or more digital processors adapted for (A) analyzing the visual data captured by the camera to recognize a gesture made by a human in the air while the human is neither touching, nor holding an object that touches, a solid surface, and (B) outputting instructions to change the projected images, in such a way as to alter the projected images in response to the gesture;

wherein the apparatus is wearable by a person in a manner in which weight of the apparatus is not supported by the person's hands.

10. Apparatus as set forth in claim 9 , wherein at least one said digital processor is adapted for
outputting, in response to the gesture, instructions to project unto the surface a visual pattern captured by the camera, which visual pattern is on a different surface when captured by the camera, which different surface is not part of an electronic visual display device.

11. Apparatus as set forth in claim 9 , wherein at least one said-digital processor is adapted for outputting, in response to the gesture, instructions for the camera to take a photograph.

12. Apparatus as set forth in claim 9 , wherein at least one said digital processor is adapted for outputting, in response to the gesture, instructions to project, on the user, an image of a timepiece.

13. Apparatus as set forth in claim 9 , wherein at least one said digital processor is adapted for analyzing said visual data to detect changes in position of a body part of a user or of an object held by or affixed to the user, and further adapted for outputting instructions to alter the projected images in response to the changes in position.

14. Apparatus as set forth in claim 9 , wherein at least one said digital processor is adapted for analyzing the visual data to recognize a tangible object and for outputting instructions to project on the object information about the object.

Assignees

• Mistry Pranav
• Maes Patricia
• Massachusetts Inst Technology

Inventors

• Mistry Pranav
• Maes Patricia

Classifications

• G06F1/1639

  the display being based on projection · CPC title

• H04M1/0272

  for a projector or beamer module assembly (constructional features of projectors per se G03B21/00; constructional details of projectors using electronic spatial light modulators per se H04N9/31) · CPC title

• G06F17/30879

  Physics · mapped topic

• G06F1/1696

  the I/O peripheral being a printing or scanning device · CPC title

• G06F3/0426

  tracking fingers with respect to a virtual keyboard projected or printed on the surface (virtual keyboards on touch screens G06F3/04886) · CPC title