Devices with enhanced audio

Patent metadata

Field	Value
Publication number	US-2025068383-A1
Application number	US-202418827398-A
Country	US
Kind code	A1
Filing date	Sep 6, 2024
Priority date	Jul 28, 2011
Publication date	Feb 27, 2025
Grant date	—

Abstract

A system for enhancing audio including a plurality of sensors, an output device, and a processor in communication with the plurality of sensors and the output device. The processor is configured to process data captured by the plurality of sensors, and based on that, modify an output of the output device. The processor also is configured to determine whether there are a plurality of users associated with a video conferencing session, determine which user of the plurality of users is speaking, and enhance the audio or video output of the speaking user on the output device.

First claim

1 .- 20 . (canceled)

21 . A computing device, comprising:
a processor that is configured to communicate with an audio output device and a display to provide a video conferencing session having a video output and a sound output, the processor to:
receive i) streaming audio from a first microphone and video from a first camera of a first computing device, and ii) streaming audio from a second microphone and video from a second camera of a second computing device;
produce during the video conferencing session
i) the video, on the display, that includes simultaneous images of a plurality of people, respectively, that are grouped together on the display to present a single interface of the video conferencing session, wherein the plurality of people include a first user of the first camera and the first computing device and a second user of the second camera and the second computing device, and
ii) the sound output, by the audio output device, which includes audio of the plurality of people including audio from the first computing device and audio from the second computing device;

detect that one of the plurality of people is speaking, as a speaking person, and determine which one of the simultaneous images on the display correlates to the audio of the speaking person;
detect that a location, on the display, of said one of the simultaneous images of the speaking person is offset from a middle of the display, and in response enhance the audio of the speaking person relative to the audio of others in the plurality of people so that the audio of the speaking person is angled towards a side of the display on which the image of the speaking person is located.

22 . The computing device of claim 21 wherein the processor is further configured to, in response to detecting the speaking person, enhance the video by reducing the images of the others in the plurality of people.

23 . The computing device of claim 21 wherein the processor is further configured to, in response to detecting the speaking person, enhance the video by enlarging the image of the speaking person relative to the images of the others in the plurality of people.

24 . The computing device of claim 21 wherein the processor is further configured to, in response to detecting the speaking person, enhance the video by changing color in in the image of the speaking person, or pulling the image of the speaking person to front on the display.

25 . The computing device of claim 21 wherein the processor detects the speaking person by performing facial recognition on the video from the first camera and on the video from the second camera, or by performing voice recognition on the streaming audio from the first computing device and on the streaming audio from the second computing device.

26 . The computing device of claim 21 wherein the processor is further configured to select one of the plurality of people in the video conferencing session by detecting a viewing user's tap on the image of the selected person and transmitting a signal to a computing device at a location of the selected person to process the audio of the selected person to better capture speech of the selected person.

27 . A method for providing a video conferencing session having video and sound output, the method comprising:
receiving in a viewing user's computing device i) streaming audio from a first microphone and video from a first camera of a first computing device, and ii) streaming audio from a second microphone and video from a second camera of a second computing device; producing, during the video conferencing session by a single video chat program running in the viewing user's computing device:
i) the video, on a display, that includes a plurality of simultaneous images of a plurality of people, respectively, wherein the plurality of simultaneous images are grouped together on the display to present a single interface of the video conferencing session, wherein the plurality of people include a first user of the first camera and the first computing device and a second user of the second camera and the second computing device;
ii) the sound output, by an audio output device, which includes the streaming audio from the first computing device and the streaming audio from the second computing device;
detecting that one of the plurality of people is speaking, as a speaking person, and determining which one of the plurality of simultaneous images on the display correlates to the streaming audio of the speaking person; and detecting that a location, on the display, of said one of the plurality of simultaneous images that correlates to the speaking person is offset from a middle of the display, and in response enhancing the streaming audio of the speaking person in the sound output, relative to the streaming audio of others in the plurality of people so that the viewing user hears the streaming audio of the speaking person from where the image of the speaking person is located.

28 . The method of claim 27 further comprising, in response to detecting the speaking person, enhancing the video by reducing the images of the others in the plurality of people.

29 . The method of claim 27 further comprising, in response to detecting the speaking person, enhancing the video by enlarging the image of the speaking person relative to the images of the others in the plurality of people.

30 . The method of claim 27 further comprising, in response to detecting the speaking person, enhancing the video by changing color in in the image of the speaking person, or pulling the image of the speaking person to front on the display.

31 . The method of claim 27 wherein detecting the speaking person comprises performing facial recognition on the video from the first camera and on the video from the second camera or performing voice recognition on the streaming audio from the first microphone and on the streaming audio from the second microphone.

32 . The method of claim 27 further comprising selecting one of the plurality of people in the video conferencing session by:
detecting a viewing user's tap on the image of the selected person; and
transmitting a signal to a computing device at a location of the selected person to process the streaming audio of the selected person so as to better capture speech of the selected person.

33 . A memory comprising instructions stored therein that configure a processor in a viewing user's computing device to provide a video conferencing session having video and sound output, by the processor:
receiving i) streaming audio from a first microphone and video from a first camera of a first computing device, and ii) streaming audio from a second microphone and video from a second camera of a second computing device; producing, during the video conferencing session by a single video chat program running in the viewing user's computing device, i) the video, on a display, that includes a plurality of simultaneous images of a plurality of people, respectively, that are grouped together on the display to present a single interface of the video conferencing session, wherein the plurality of people includes a first user of the first camera and the first computing device and a second user of the second camera and the second computing device, and ii) the sound output, by an audio output device, that includes the streaming audio from the first microphone and the streaming audio from the second microphone; detecting that one of the plurality of people is speaking, as a speaking person, and determining which one of the plurality of simultaneous images on the display correlates to the streaming audio of th

Assignees

• Apple Inc

Inventors

• Pance Aleksandar
• Bilbrey Brett
• Hadley Darby E
• Johnson Martin E
• Isaac Ronald Nadim

Classifications

• H04S7/305

  Electronic adaptation of stereophonic audio signals to reverberation of the listening space (H04S7/301 takes precedence) · CPC title

• H04R2430/01

  Aspects of volume control, not necessarily automatic, in sound systems · CPC title

• H04R29/001

  for loudspeakers (H04R29/007 takes precedence) · CPC title

• H04R3/04

  for correcting frequency response · CPC title

• H04N7/15

  Conference systems · CPC title