Devices with enhanced audio

Patent metadata

Field	Value
Publication number	US-11061643-B2
Application number	US-202017002653-A
Country	US
Kind code	B2
Filing date	Aug 25, 2020
Priority date	Jul 28, 2011
Publication date	Jul 13, 2021
Grant date	Jul 13, 2021

Abstract

A system for enhancing audio including a plurality of sensors, an output device, and a processor in communication with the plurality of sensors and the output device. The processor is configured to process data captured by the plurality of sensors, and based on that, modify an output of the output device. The processor also is configured to determine whether there are a plurality of users associated with a video conferencing session, determine which user of the plurality of users is speaking, and enhance the audio or video output of the speaking user on the output device.

First claim

What is claimed is:

1. A viewing user's computing device, comprising:
a processor that is to be in communication with an audio output device and a display screen and configured to
receive first images from first a video camera of a first location, and first audio from a first plurality of microphones of the first location, that capture a first user,
receive second images from a second video camera of the second location, and second audio from a second plurality of microphones of the second location, that capture a second user,
present the received first images in a first chat window, and the received second images in a second chat window simultaneously with the first chat window to a viewing user through the display screen of the viewing user's computing device and wherein each of the first chat window and the second chat window has its own audio as the first audio and the second audio, respectively, being simultaneously output through the audio output device,
select one of the first user or the second user, by detecting the viewing user's tap on an image of the first user in the first chat window or the second user in the second chat window on the display screen,
determine when the selected user is speaking, and
focus on the selected user, by transmitting a signal to the first location or the second location of the selected user to process the first audio from the first plurality of microphones or the second audio from the second plurality of microphones for beam steering to better capture the selected user.

2. The computing device of claim 1 wherein the processor is further configured to focus on the selected user by digitally enhancing the first audio or the second audio by filtering different channel signals to enhance or to reduce particular frequencies.

3. The computing device of claim 2 wherein the processor is configured to present an option to the viewing user on whether to focus on a selected user.

4. The computing device of claim 3 wherein the processor is further configured to digitally enhance the image of the selected user according to the option presented to the viewing user.

5. The computing device of claim 1 wherein the processor is further configured to present an option to the viewing user on whether to focus on a selected user.

6. The computing device of claim 5 wherein the processor is configured to digitally enhance the first images or the second images from the video camera according to the option presented to the viewing user.

7. The computing device of claim 6 wherein an image of the first user as presented in the first images through the display screen is smaller than an image of the second user as presented in the second images through the display screen, until the focus on the selected user being the first user digitally enhances the image of the first user.

8. The computing device of claim 1 wherein the first audio as presented through the audio output device is quieter than the second audio as presented through the audio output device until the focus on the selected user being the first user digitally enhances the first audio.

9. The computing device of claim 1 wherein the processor is further configured to automatically enhance an image of the first user or an image of the second user being presented on the display screen, according to a setting of a video conferencing application.

10. The computing device of claim 9 wherein the processor is to enhance the image of the first user or the second user by changing a color of a chat window containing the image or by pulling the chat window to a front of the display screen.

11. A method performed by a processor executing a video conferencing application in a viewing user's computing device, the method comprising:
receiving first images from a first video camera of a first location and first audio from a plurality of microphones of the first location, that capture a first user;
receiving second images from second a video camera of a second location and a second audio from a plurality of microphones of the second location, that capture a second user;
presenting the received first images in a first chat window, and the received second images in a second chat window simultaneously with the first chat window through a display screen of the viewing user's computing device, wherein each of the first chat window and the second chat window has its own audio as the first audio and the second audio, respectively, being simultaneously output through an audio output device of the viewing user's computing device;
selecting one of the first user or the second user, by detecting the viewing user's tap on an image of the first user in the first chat window or the second user in the second chat window on the display screen;
determining when the selected user is speaking; and
focusing on the selected user, by transmitting a signal to the first location or the second location of the selected user to process the first audio from the plurality of microphones or the second audio from the second plurality of microphones from beam steering to better capture the selected user.

12. The method of claim 11 wherein the first plurality of microphones are spaced around a perimeter of a display screen at the first location.

13. The method of claim 12 wherein focusing on the selected user further comprises digitally enhancing the first audio or the second audio by filtering different channel signals to enhance or to reduce particular frequencies.

14. The method of claim 12 further comprising
presenting an option to the viewing user on whether to focus on a selected user.

15. The method of claim 14 further comprising
digitally enhancing the first images or the second images according to the option presented to the viewing user.

16. The method of claim 12 further comprising
automatically enhancing the first images or the second images, according to a setting of a video conferencing application.

17. The method of claim 11 wherein focusing on the selected user further comprises digitally enhancing the first audio or the second audio by filtering different channel signals to enhance or to reduce particular frequencies.

18. The method of claim 11 further comprising
presenting an option to the viewing user on whether to focus on a selected user.

19. The method of claim 18 further comprising
digitally enhancing the first images or the second images according to the option presented to the viewing user.

20. The method of claim 11 further comprising
automatically enhancing the first images or the second images, according to a setting of a video conferencing application.

21. A viewing user's computing device, comprising:
an audio output device;
a display screen; and
a processor that is to be in communication with the audio output device and the display screen and configured to
receive first images from a first video camera of a first location, and first audio from a first plurality of microphones of the first location, that capture a first user,
receive second images from a second video camera of a second location, and second audio from a second plurality of microphones of the second location, that capture a second user,
present the received first images in a first chat window and the received second images in a second chat window simultaneously with the first chat window through the display screen, and wherein each of the first chat window and the second chat window has its own audio as the first audio and the second au

Assignees

• Apple Inc

Inventors

• Pance Aleksandar
• Bilbrey Brett
• Hadley Darby E
• Johnson Martin E
• Isaac Ronald Nadim

Classifications

• H04N23/69

  Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming · CPC title

• G06F3/167 Primary

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• H04N21/42201

  biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user (input arrangements for interaction with the human body based on nervous system activity detection G06F3/015) · CPC title

• H04N21/4852

  for modifying audio parameters, e.g. switching between mono and stereo · CPC title

• H04N21/42203

  sound input device, e.g. microphone · CPC title