Perspective-aware projected user interfaces

Patent metadata

Field	Value
Publication number	US-9723248-B1
Application number	US-201514612180-A
Country	US
Kind code	B1
Filing date	Feb 2, 2015
Priority date	Feb 2, 2015
Publication date	Aug 1, 2017
Grant date	Aug 1, 2017

Abstract

Information regarding actions or activities to be performed at a workstation may be projected upon a portion of the workstation using one or more projectors. The information may include one or more arrows or other indicators referencing specific tools, materials or objects that may be used to perform one or more of the actions or activities. Such arrows or indicators may be rendered in a manner that simulates a three-dimensional or floating appearance thereof from a perspective of a user that may be adjusted or modified based on changes in the perspective of the user, and with respect to one or more physical or virtual sources of light.

First claim

What is claimed is:

1. A workstation comprising:
a working surface;
a projector configured to project light upon at least a portion of the working surface;
an imaging device; and
a computing device in communication with the imaging device and the projector,
wherein the computing device is configured to implement one or more services, and
wherein the one or more services are configured to:
capture imaging data of at least a portion of the workstation;
determine a vantage point of a user based at least in part on the imaging data;
identify information regarding a task to be performed by the user, wherein the information relates to at least one object maintained in a storage area;
determine a position of at least one light source provided within the workstation;
determine a position of the storage area in which the at least one object is maintained based at least in part on the imaging data;
determine a location on the portion of the working surface for projecting an indicator associated with the at least one object based at least in part on the imaging data;
select a rendering for the indicator based at least in part on at least one of the vantage point of the user, the position of the at least one light source or the location on the portion of the working surface, wherein the selected rendering for the indicator comprises a synthetic three-dimensional view of the indicator;
define a projection to be projected upon the portion of the working surface at the location in accordance with the selected rendering of the indicator; and
project the projection upon the portion of the working surface at the location using the projector.

2. The workstation of claim 1 , wherein the one or more services are further configured to:
select a virtual plane based at least in part on the vantage point of the user and the location on the portion of the working surface for projecting the indicator;
wherein the rendering for the indicator is selected such that the indicator appears in the synthetic three-dimensional view within the selected virtual plane.

3. The workstation of claim 1 , wherein the one or more services are further configured to:
determine at least one of a location or an orientation of a shadow element associated with the indicator based at least in part on the at least one of the vantage point of the user, the position of the at least one light source or the location on the portion of the working surface,
wherein the projection further comprises the shadow element.

4. A computer-implemented method comprising:
identifying information regarding at least one action associated with a task to be performed on a surface of a workstation;
identifying a position of at least one light source associated with the workstation;
identifying at least one aspect of the workstation associated with the at least one action;
identifying an indicator of the at least one aspect of the workstation;
defining a first manner in which the indicator is to be projected upon the surface with three-dimensionality, wherein the first manner is defined based at least in part on the position of the at least one light source; and
causing, by at least one computer processor, at least one projector to project at least the indicator upon the surface in accordance with the first manner.

5. The computer-implemented method of claim 4 , further comprising:
determining a first perspective of a user at the workstation; and
determining a location of the at least one aspect of the workstation,
wherein the first manner is defined based at least in part on the first perspective of the user and the location of the at least one aspect of the workstation.

6. The computer-implemented method of claim 5 , wherein determining the first perspective of the user at the workstation further comprises:
determining a position of at least one eye of the user,
wherein the first perspective is determined based at least in part on the position of the at least one eye of the user.

7. The computer-implemented method of claim 6 , wherein determining the position of the at least one eye of the user comprises:
determining a position of a head of the user,
wherein the position of the at least one eye of the user is determined based at least in part on the position of the head of the user.

8. The computer-implemented method of claim 5 , wherein defining the first manner in which the indicator is to be projected upon the surface with three-dimensionality further comprises:
defining at least one shadow element corresponding to the indicator of the at least one aspect of the workstation,
selecting at least one of a location of the at least one shadow element or an orientation of the at least one shadow element based at least in part on at least one of:
at least one attribute of the indicator; or
the first perspective of the user at the workstation,
wherein the first manner presents the at least one shadow element in the selected location or the selected orientation with respect to the indicator.

9. The computer-implemented method of claim 5 , wherein defining the first manner in which the indicator is to be projected upon the surface with three-dimensionality further comprises:
determining at least one attribute of the indicator; and
selecting a location on the surface for the indicator based at least in part on the at least one attribute and the location of the at least one aspect of the workstation.

10. The computer-implemented method of claim 9 , wherein defining the first manner in which the indicator is to be projected upon the surface with three-dimensionality further comprises:
selecting a virtual plane within which the indicator of the at least one aspect of the workstation is to appear to the user based at least in part on the first perspective of the user and the selected location on the surface for the indicator; and
defining a projection onto the surface, wherein the projection onto the surface comprises the indicator, and wherein the projection onto the surface corresponds to the virtual plane,
wherein causing the at least one projector to project at least the indicator upon the surface in accordance with the first manner comprises:
causing the at least one projector to project the projection onto the surface.

11. The computer-implemented method of claim 10 , further comprising:
deriving a depth profile of at least a portion of the workstation using imaging data captured by at least one imaging device; and
selecting at least one of the location of the at least one aspect of the workstation or the location on the surface for the indicator based at least in part on the depth profile.

12. The computer-implemented method of claim 9 , wherein the location of the at least one aspect of the workstation is within at least one region beneath the surface.

13. The computer-implemented method of claim 12 , wherein defining the first manner in which the indicator is to be projected upon the surface with three-dimensionality further comprises:
identifying an image of the at least one region beneath the surface, and
wherein causing the at least one projector to project at least the indicator upon the surface in accordance with the first manner further comprises:
causing, by the at least one computer processor, the at least one projector to project at least the image of the at least one region onto at least a portion of the surface.

14. The computer-implemented method of claim 5 , further comprising:
determining a second perspective of the user at the workstation; and
defining a second manner in which the indicator is to be projected upon the s

Assignees

• Amazon Tech Inc

Inventors

• Colburn Robert Alexander
• Curlander James Christopher
• Gorumkonda Gurunandan Krishnan
• De Greef Lilian

Classifications

• H04N7/18

  Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast · CPC title

• H04N9/3194

  including sensor feedback · CPC title

• G06T15/205

  Image-based rendering · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• H04N5/7458 Primary

  the modulator being an array of deformable mirrors, e.g. digital micromirror device [DMD] · CPC title