Relative position-inclusive device interfaces

Patent metadata

Field	Value
Publication number	US-9274744-B2
Application number	US-90298610-A
Country	US
Kind code	B2
Filing date	Oct 12, 2010
Priority date	Sep 10, 2010
Publication date	Mar 1, 2016
Grant date	Mar 1, 2016

Abstract

A computing device can analyze image or video information to determine a relative position of an active user. The computing device can optimize audio or video data capture based at least in part upon the relative location. The device can capture audio using one or more microphones pointing toward the relative location of the active user, and can use other microphones to determine audio from other sources to be removed from the captured audio. If video data is being captured, a video capture element can be adjusted to focus primarily on the active user. The position of the user can be monitored so the audio and video data capture can be adjusted accordingly.

First claim

What is claimed is:

1. A method of modifying user input to a computing device, comprising:
capturing first image data of a user using a first image capture element of the computing device;
capturing second image data of the user using a second image capture element, the first image capture element and the second image capture element being positioned on a same side of the computing device;
analyzing the first image data and the second image data to detect at least one action performed by the user;
determining, using a triangulation determination based at least in part on the first image data and the second image data, a relative position of the user with respect to the computing device;
selecting, based at least in part on the relative position of the user, a first audio capture element from a plurality of audio capture elements, the first audio capture element being closer to the user than a remainder of the plurality of audio capture elements;
capturing audio data using the plurality of audio capture elements of the computing device;
determining background audio data included in the audio data based at least in part on the relative position of the user, the background audio data corresponding to sound captured by the remainder of the plurality of audio capture elements;
generating modified audio data by removing data corresponding to at least a portion of the background audio data included in the audio data; and
providing the modified audio data as input to the computing device.

2. The method of claim 1 , further comprising:
updating the relative position of the user based on the first image data and the second image data.

3. The method of claim 1 , further comprising:
capturing video data of the user using at least one of the first image capture element or the second image capture element, wherein the at least one of the first image capture element or the second image capture element is oriented to capture the video data from a viewable area at least partially including the relative position of the user.

4. The method of claim 1 , wherein generating the modified audio data includes removing substantially all of the background audio data from the audio data.

5. A method of modifying input to a computing device, comprising:
capturing first image data and second image data using at least two image capture elements of the computing device, the at least two image capture elements being positioned on a same side of the computing device, the first image data and the second image data including respective image representations of an active audio source;
determining a relative position of the active audio source with respect to the computing device, based, at least in part, on a triangulation determination based at least in part on the first image data and the second image data;
selecting, based at least in part on the relative position of the active audio source, a first audio capture element from a plurality of audio capture elements, the first audio capture element being closer to the active audio source than a remainder of the plurality of audio capture elements;
capturing active audio data using the plurality of audio capture elements;
updating the relative position of the active audio source based at least in part upon third image data of the active audio source captured using the at least two image capture elements to obtain an updated relative position of the active audio source;
adjusting the first audio capture element to capture the active audio data from a direction at least partially towards the updated relative position of the active audio source;
determining background audio data within the active audio data based at least in part on the updated relative position of the active audio source, the background audio data corresponding to sound emitted from locations other than the updated relative position of the active audio source; and
modifying the active audio data to remove data corresponding to at least a portion of the background audio data.

6. The method of claim 5 , further comprising:
analyzing the first image data to detect at least one action corresponding to the active audio source.

7. The method of claim 6 , wherein analyzing the first image data comprises executing at least one image analysis algorithm for determining a motion of a user, a gesture of the user, speech formation by the user, or a state of a sound-producing device.

8. The method of claim 6 , wherein determining the relative position of the active audio source with respect to the computing device further comprises:
validating an identity of the active audio source.

9. The method of claim 5 , further comprising:
determining a second relative position of a secondary audio source with respect to the computing device;
capturing secondary audio data using at least one audio capture element of the computing device, the at least one audio capture element being configured to capture the secondary audio data from a direction at least partially towards the second relative position of the secondary audio source; and
filtering the secondary audio data from the active audio data.

10. The method of claim 5 , further comprising:
analyzing the active audio data and second audio data captured from any other audio source; and
filtering of the active audio data based on the second audio data.

11. The method of claim 5 , wherein capturing the active audio data is based at least in part upon an identity of the active audio source.

12. The method of claim 5 , further comprising:
capturing video data of the active audio source using at least one image capture element of the computing device currently oriented to capture the video data from a viewable area at least partially including the relative position of the active audio source.

13. The method of claim 5 , wherein the at least two image capture elements include at least one of an image capture element, an ultrasonic transceiver, an infrared transceiver, a laser, or a radio transceiver.

14. The method of claim 5 , wherein the relative position of the active audio source is updated based upon at least one of movement of the active audio source or movement of the computing device.

15. The method of claim 5 , wherein determining the relative position of the active audio source comprises identifying the active audio source within a defined area with respect to the computing device.

16. The method of claim 5 , further comprising transmitting a response signal from the computing device in the direction at least partially towards the updated relative position of the active audio source.

17. The method of claim 5 , wherein the active audio source is a primary user of the computing device or another person in closest proximity to the computing device.

18. The method of claim 5 , further comprising:
determining a person speaking out of a plurality of people within a proximity of the computing device to be the active audio source.

19. A computing device, comprising:
a processor;
a memory device including instructions operable to be executed by the processor to perform a set of actions, enabling the processor to:
capture first image data and second image data using at least two image capture elements of the computing device, the at least two image capture elements being positioned on a same side of the computing device, the first image data and second image data including respective image representations of an active audio source;
determine a relative position of the active audio source with r

Assignees

• Hart Gregory M
• Bezos Jeffrey P
• Kwee Frances Mhh
• Brown James Samuel
• Amazon Tech Inc

Inventors

• Hart Gregory M
• Bezos Jeffrey P
• Kwee Frances Mhh
• Brown James Samuel

Classifications

• G06F1/1686

  the I/O peripheral being an integrated camera · CPC title

• G06F2200/1637

  Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• H04N7/142

  Constructional details of the terminal equipment, e.g. arrangements of the camera and the display · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title