Methods and apparatus for real-time interactive anamorphosis projection via face detection and tracking

Patent metadata

Field	Value
Publication number	US-10867164-B2
Application number	US-201816023967-A
Country	US
Kind code	B2
Filing date	Jun 29, 2018
Priority date	Jun 29, 2018
Publication date	Dec 15, 2020
Grant date	Dec 15, 2020

Abstract

Methods, apparatus, systems, and articles of manufacture for real-time interactive anamorphosis projection via face detection and tracking are disclosed. An example system includes a sensor to capture an image of a face of a user. An augmented reality controller is to access the image from the sensor, determine a position of the face of the user relative to a display surface, and apply a perspective correction to an anamorphic camera representing a vantage point of the active user. A user application is to generate a scene based on the position of the anamorphic camera. A display is to present, at the display surface, the scene based on the vantage point of the active user.

First claim

What is claimed is:

1. An apparatus for interactive projection, the apparatus comprising:
a buffer to receive image data from a sensor;
a buffer controller to move a stack top from a first portion of the image data to a second portion of the image data, and in response to determining that the first portion of the image data is not being processed, release the first portion of the image data from the buffer;
a face detector to detect a face of a user in an image from the second portion of the image data captured by the sensor in proximity to a display surface;
a position tracker to determine a face position of the detected face;
a camera position calculator to calculate a position of a virtual camera in relation to the display surface based on the position of the detected face; and
a perspective transform generator to apply a perspective correction to the virtual camera to cause the virtual camera to replicate a vantage point of the user, the perspective transform generator to cause a display to render, on the display surface, a scene based on the vantage point of the user.

2. The apparatus of claim 1 , wherein the face detector is further to detect a face bounding rectangle, and the position tracker is to determine an estimated eye position within the face bounding rectangle.

3. The apparatus of claim 2 , further including a position normalizer to normalize the estimated eye position.

4. The apparatus of claim 3 , wherein the position normalizer is to determine whether distortion correction has been applied to the image, and in response to determining that distortion correction has been applied to the image, normalize the eye position using a spline interpolation.

5. The apparatus of claim 1 , wherein the virtual camera is a first virtual camera, the user is a first user, the vantage point is a first vantage point, the scene is a first scene, and the perspective transform generator is further to apply a second perspective correction to a second virtual camera to cause the second virtual camera to replicate a second vantage point of a second user, and cause the display to render, on the display surface, a second scene based on the vantage point of the second user.

6. The apparatus of claim 5 , further including a display controller to synchronize a display with a first pair of lenses and a second pair of lenses, the first pair of lenses to enable display of the first scene to the first user, the second pair of lenses to enable display of the second scene to the second user.

7. The apparatus of claim 6 , wherein the first pair of lenses are actively shuttered lenses.

8. At least one non-transitory computer-readable medium comprising instructions which, when executed, cause at least one processor to at least:
receive image data from a sensor;
move a stack top from a first portion of the image data to a second portion of the image data, and in response to determining that the first portion of the image data is not being processed, release the first portion of the image data;
detect a face of a user in an image, the image from the second portion of the image data captured by the sensor in proximity to a display surface;
determine a face position of the detected face;
calculate a position of an anamorphic camera in relation to the display surface based on the position of the detected face;
apply a perspective correction to the anamorphic camera to cause the anamorphic camera to replicate a vantage point of the user; and
cause a display to render, on the display surface, a scene based on the vantage point of the user.

9. The least one non-transitory computer-readable medium of claim 8 , wherein the instructions cause the at least one processor to determine the face position by at least:
detecting a face bounding rectangle; and
determining an estimated eye position within the face bounding rectangle.

10. The least one non-transitory computer-readable medium of claim 9 , wherein the instructions, when executed, further cause the at least one processor to normalize the estimated eye position.

11. The least one non-transitory computer-readable medium of claim 10 , wherein the instructions, when executed, cause the processor to normalize the eye position by at least:
determining whether distortion correction has been applied to the image; and
in response to determining that distortion correction has been applied to the image, normalizing the eye position using a spline interpolation.

12. The least one non-transitory computer-readable medium of claim 8 , wherein the face of the user corresponds to a first face of a first user, the position is a first position, the anamorphic camera is a first anamorphic camera, the perspective correction is a first perspective correction, the vantage point is a first vantage point, the scene is a first scene, and the instructions, when executed, cause the at least one processor to at least:
detect a second face of a second user in the image;
determine a second face position of the second face;
calculate a second position of a second anamorphic camera in relation to the display surface based on the second face position;
apply a second perspective correction to the second anamorphic camera to cause the second anamorphic camera to replicate a second vantage point of the second user; and
cause the display to render, on the display surface, a second scene based on the vantage point of the second user.

13. The least one non-transitory computer-readable medium of claim 12 , wherein the instructions, when executed, cause the at least one processor to at least synchronize a display with a first pair of lenses and a second pair of lenses, the first pair of lenses to enable display of the first scene to the first user, the second pair of lenses to enable display of the second scene to the second user.

14. The least one non-transitory computer-readable medium of claim 13 , wherein the first pair of lenses are actively shuttered lenses.

15. The least one non-transitory computer-readable medium of claim 13 , wherein the first pair of lenses is to enable display of the first scene to the first user and the second pair of lenses is to enable display of the second scene to the second user during alternating time periods.

16. An apparatus for interactive projection, the apparatus comprising:
means for receiving image data from a sensor;
means for moving a stack top from a first portion of the image data to a second portion of the image data, and in response to determining that the first portion of the image data is not being processed, releasing the first portion of the image data;
means for detecting a face of a user in an image, the image from the second portion of the image data captured by the sensor in proximity to a display surface;
means for determining a face position of the detected face;
means for identifying whether a user associated with the identified face is an active user;
means for calculating, in response to determining that the user associated with the identified face is an active user, a position of an anamorphic camera in relation to the display surface based on the position of the detected face; and
means for applying a perspective correction to the anamorphic camera to cause the anamorphic camera to replicate a vantage point of the active user, the means for applying to cause a display to render, on the display surface, a scene based on the vantage point of the active user.

17. The apparatus of claim 16 , wherein the means for determining is further to detect a face bounding rectangle, and determine an estimated eye pos

Assignees

• Intel Corp

Inventors

• Gavino Brandon
• Dharmadhikari Abhay
• Panneer Selvakumar
• Marshall Carl
• Nooji Vinay
• Chen Fan
• Warlen Alexandra

Classifications

• G06V10/82 Primary

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V40/172

  Classification, e.g. identification · CPC title

• G06F18/2413

  based on distances to training or reference patterns · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title