Identifying projection surfaces in augmented reality environments

Patent metadata

Field	Value
Publication number	US-9723293-B1
Application number	US-201113165543-A
Country	US
Kind code	B1
Filing date	Jun 21, 2011
Priority date	Jun 21, 2011
Publication date	Aug 1, 2017
Grant date	Aug 1, 2017

Abstract

Techniques for identifying usable projection areas within an environment and projecting content onto these areas are described herein. The systems described herein may include one or more projectors, one or more cameras, and one or more computing devices. In some instances, the described techniques may utilize the one or more cameras to identify a flat projection surface. After identifying this surface, the techniques may use depth detection, edge detection, or other techniques to determine whether the surface includes any objects protruding there from. If so, then the techniques may identify the size and location of these objects and may store an indication that these objects occlude a portion of the flat projection surface and, therefore, that the locations associated with these objects are not usable projection areas. The techniques may then deduct the locations of the objects from the initially identified projection surface to determine the usable projection areas.

First claim

What is claimed is:

1. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:
identifying, within an environment, a projection surface for projecting content thereon;
identifying one or more objects protruding from the projection surface;
determining a portion of the projection surface that is free from the one or more protruding objects based on the identifying of the one or more objects protruding from the projection surface;
receiving a request to project a particular piece of content within the environment;
at least partly in response to the determining, altering a field of view of a projector to include the determined portion of the projection surface that is free from the one or more protruding objects; and
at least partly in response to the receiving of the request, projecting, by the projector, the particular piece of content onto the determined portion of the projection surface that is free from the one or more protruding objects.

2. One or more non-transitory computer-readable media as recited in claim 1 , wherein the identifying of the projection surface comprises:
projecting a structured light pattern onto the projection surface; and
measuring distortion of the structured light pattern on the projection surface.

3. One or more non-transitory computer-readable media as recited in claim 1 , wherein the identifying of the one or more protruding objects comprises determining a depth of the projection surface and a depth of the one or more protruding objects.

4. One or more non-transitory computer-readable media as recited in claim 1 , wherein the identifying of the one or more protruding objects comprises detecting edges of the one or more protruding objects.

5. One or more non-transitory computer-readable media as recited in claim 4 , wherein the detecting of the edges comprises:
identifying a predominant color of the projection surface; and
identifying changes in the predominant color, at least a portion of the changes representing respective edges of the one or more protruding objects.

6. One or more non-transitory computer-readable media as recited in claim 1 , wherein the identifying of the one or more protruding objects comprises comparing a first image of the environment with a second, later image of the environment.

7. One or more non-transitory computer-readable media as recited in claim 1 , wherein the request is received from a user within the environment and the particular piece of content comprises a menu, the acts further comprising:
identifying a gesture from the user selecting one or more particular icons on the menu; and
at least partly in response to the identifying of the gesture:
identifying a different piece of content associated with the gesture; and
projecting the different piece of content.

8. One or more non-transitory computer-readable media as recited in claim 1 , wherein the identifying of the one or more protruding objects comprises identifying at least two protruding objects, and wherein the projecting comprises projecting the particular piece of content on a portion of the projection surface that is spaced approximately between the at least two protruding objects.

9. A method comprising:
under control of one or more computer systems configured with specific executable instructions,
identifying a layout of an environment, the layout including at least one projection surface having one or more exposed portions and one or more occluded portions, wherein at least one of the one or more occluded portions of the at least one projection surface is at least partly covered by one or more objects that reside on, or protrude from, the at least one of the one or more occluded portions of the at least one projection surface, and wherein the identifying of the layout comprises identifying the one or more objects that reside on, or protrude from, the at least one projection surface;
receiving a request to project content within a particular area of the environment;
at least partly in response to the identifying, altering a field of view of a projector to include a particular exposed portion of the projection surface that is free from the one or more objects that reside on, or protrude from, the at least one projection surface; and
at least partly in response to the receiving of the request, projecting, by the projector, the content on the particular exposed portion of the at least one projection surface within the particular area of the environment.

10. A method as recited in claim 9 , wherein the identifying of the layout comprises detecting depths of objects within the environment.

11. A method as recited in claim 9 , wherein the identifying of the layout comprises detecting color variations within the environment to identify objects therein.

12. A method as recited in claim 11 , wherein the detected color variations within the environment identify edges of the objects within the environment.

13. A method as recited in claim 9 , wherein the at least one projection surface comprises a wall, a floor, a countertop, a ceiling, a desk, or a table.

14. A method as recited in claim 9 , wherein the request is received from a user making a gesture within the environment or the user audibly uttering the request within the environment.

Assignees

• Cederlof Colter E
• Amazon Tech Inc

Inventors

• Cederlof Colter E

Classifications

• H04N23/60

  Control of cameras or camera modules · CPC title

• H04N13/0239 Primary

  Electricity · mapped topic

• H04N13/239 Primary

  using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) · CPC title

• H04N9/12

  Picture reproducers (H04N9/11 takes precedence) · CPC title

• G01B11/2513

  with several lines being projected in more than one direction, e.g. grids, patterns · CPC title