Devices with enhanced audio

Patent metadata

Field	Value
Publication number	US-12112094-B2
Application number	US-202318194926-A
Country	US
Kind code	B2
Filing date	Apr 3, 2023
Priority date	Jul 28, 2011
Publication date	Oct 8, 2024
Grant date	Oct 8, 2024

Abstract

A system for enhancing audio including a plurality of sensors, an output device, and a processor in communication with the plurality of sensors and the output device. The processor is configured to process data captured by the plurality of sensors, and based on that, modify an output of the output device. The processor also is configured to determine whether there are a plurality of users associated with a video conferencing session, determine which user of the plurality of users is speaking, and enhance the audio or video output of the speaking user on the output device.

First claim

What is claimed is:

1. A first computing device comprising a processor that is configured to, by a video conferencing application:
communicate with a camera that captures images of a first user, and produce a video output of the first user and track movement of the first user;
communicate with a plurality of microphones that pick-up voice of the first user and produce an audio output of the first user; and
communicate with a communication interface to transmit the video output and the audio output via a network to a second computing device that is in a video conferencing session with the first computing device, the second computing displays the video output and produces sound of the audio output,
wherein the processor is configured to automatically adjust the video output of the first user as the first user moves around based on the processor using facial recognition to determine where on a display the first user is located and by tracking movement of the first user on the display.

2. The first computing device of claim 1 wherein the processor is configured to adjust the audio output in response to determining that the first user is speaking.

3. The first computing device of claim 2 wherein the processor adjusts the audio output by beam steering the plurality of microphones.

4. The first computing device of claim 3 wherein the processor is configured to focus or zoom the camera on the first user.

5. The first computing device of claim 4 wherein the video conferencing application has a setting for the processor to automatically enhance the video output or audio output of the first user when the first user is speaking.

6. The first computing device of claim 1 wherein the processor is configured to automatically adjust the video output of the first user, based on determining that there are multiple persons in the video conferencing session, wherein the processor determines there are multiple persons by performing facial recognition or voice recognition.

7. The first computing device of claim 1 wherein the video conferencing application has a setting that instructs the processor to automatically adjust the video output or the audio output of the first user when the first user is speaking.

8. The first computing device of claim 7 wherein the processor is configured to adjust the video output by focusing or zooming the camera on the first user.

9. The first computing device of claim 1 wherein the processor is configured to determine whether the first user is located offset from a middle of a display.

10. The first computing device of claim 1 wherein the processor is configured to adjust the audio output based on size of a room in which the first user is located or based on reverberation of the room.

11. A method for providing a video conferencing session having video output and audio output, the method comprising the following operations performed by a first computing device executing a video conferencing application:
communicating with a camera that captures images of a first user, and produce a video output of the first user and track movement of the first user;
communicating with a plurality of microphones that pick-up voice of the first user and produce an audio output of the first user; and
communicating with a communication interface to transmit the video output and the audio output via a network to a second computing device that is in the video conferencing session with the first computing device, the second computing displays the video output and produces sound of the audio output, and the first computing device automatically adjusts the video output of the first user as the first user moves around based on the first computing device using facial recognition to determine where on a display the first user is located and by tracking movement of the first user on the display.

12. The method of claim 11 further comprising adjusting the audio output in response to determining that the first user is speaking.

13. The method of claim 12 wherein adjusting the audio output comprises beam steering the plurality of microphones.

14. The method of claim 13 wherein adjusting the video output comprises focusing or zooming the camera on the first user.

15. The method of claim 14 wherein the video conferencing application has a setting to automatically enhance the video output or the audio output of the first user when the first user is speaking.

16. The method of claim 11 further comprising automatically adjusting the video output of the first user based on determining that there are multiple persons in the video conferencing session by performing facial recognition or voice recognition.

17. The method of claim 16 wherein the video conferencing application has a setting to automatically adjust the video output or the audio output of the first user when the first user is speaking.

18. The method of claim 17 wherein adjusting the video output comprises focusing or zooming the camera on the first user.

19. The method of claim 11 further comprising determining whether the first user is located offset from a middle of a display.

20. The method of claim 11 wherein adjusting the audio output is based on size of a room in which the first user is located or based on reverberation of the room.

Assignees

• Apple Inc

Inventors

• Pance Aleksandar
• Bilbrey Brett
• Hadley Darby E
• Johnson Martin E
• Isaac Ronald Nadim

Classifications

• H04S7/305

  Electronic adaptation of stereophonic audio signals to reverberation of the listening space (H04S7/301 takes precedence) · CPC title

• H04R2430/01

  Aspects of volume control, not necessarily automatic, in sound systems · CPC title

• H04R29/001

  for loudspeakers (H04R29/007 takes precedence) · CPC title

• H04R3/04

  for correcting frequency response · CPC title

• H04N7/15

  Conference systems · CPC title