Field of view based audio selection

Patent metadata

Field	Value
Publication number	US-12137329-B2
Application number	US-202218049234-A
Country	US
Kind code	B2
Filing date	Oct 24, 2022
Priority date	Oct 24, 2022
Publication date	Nov 5, 2024
Grant date	Nov 5, 2024

Abstract

Systems and methods are described for improving audio quality. Sensor data is received from an accelerometer of a camera. Based on measurements captured by the sensor data, a position of a lens of the camera and a current field of view (FOV) of the camera is determined. Audio beamforming is performed based on the current FOV of the camera by selecting a subset of microphones disposed on the camera to record audio based on the position of the lens and the current FOV of the camera; and activating the subset of microphones disposed on the camera, where microphones outside the selected subset of microphones are excluded from activation so that audio originating outside of the current FOV is removed.

First claim

What is claimed is:

1. A method for improving audio quality comprising:
receiving sensor data from an accelerometer of a camera;
determining a position of a lens of the camera based on measurements captured by the sensor data;
determining a current field of view (FOV) of the camera based on the measurements captured by the sensor data; and
performing audio beamforming based on the current FOV of the camera by:
selecting a subset of microphones disposed on the camera to record audio based on the position of the lens and the current FOV of the camera; and
activating the subset of microphones disposed on the camera, wherein microphones outside the selected subset of microphones are excluded from activation so that audio originating outside of the current FOV is removed.

2. The method of claim 1 , the method further comprising:
analyzing audio from one or more microphones on the camera to determine signal amplitudes within the audio; and
selecting the subset of microphones to be activated based on the one or more microphones with a highest signal amplitude or lowest average level of noise.

3. The method of claim 1 , the method further comprising:
determining, based on the sensor data from the accelerometer and a gyroscope of the camera, a direction of incident light received by a lens of the camera;
selecting the subset of microphones disposed on the camera to record audio based on the direction of incident light received by the lens; and
activating the subset of microphones to record audio within the current FOV of the camera.

4. The method of claim 1 , the method further comprising:
receiving first accelerometer data from a first accelerometer disposed on the base of the camera;
receiving second accelerometer data from a second accelerometer disposed on a sensor board of the camera, wherein the sensor board of the camera is moveable to point in a range of directions; and
detecting a direction the lens of the camera is pointed towards in relation to a body of the camera based on a combination of at least the first accelerometer data and the second accelerometer data;
wherein the detected direction of the lens of the camera provides dynamic information about tilt and movement of the lens in relation to the body of the camera, and wherein the camera body is a point of reference for the movement of the lens.

5. The method of claim 1 , the method further comprising:
arranging the audio beamforming to match an area covered by the current FOV of the camera, wherein the audio beamforming matches a direction the lens is pointed and sound outside the current FOV is attenuated.

6. The method of claim 1 , the method further comprising:
switching from a first mode of matching the audio beamforming to the current FOV to a second mode of receiving omnidirectional audio.

7. The method of claim 1 , the method further comprising:
determining that one or more microphones are occluded by a portion of the camera body, preventing audio detection for a region of interest within the current FOV; and
dynamically selecting the subset of microphones based on the determination that each microphone within the subset is not occluded by the portion of the camera body.

8. A computing apparatus comprising:
a processor; and
a memory storing instructions that, when executed by the processor, configure the apparatus to:
receive sensor data from an accelerometer of a camera;
determine a position of a lens of the camera based on measurements captured by the sensor data;
determine a current field of view (FOV) of the camera based on the measurements captured by the sensor data; and
perform audio beamforming based on the current FOV of the camera by:
select a subset of microphones disposed on the camera to record audio based on the position of the lens and the current FOV of the camera; and
activate the subset of microphones disposed on the camera, wherein microphones outside the selected subset of microphones are excluded from activation so that audio originating outside of the current FOV is removed.

9. The computing apparatus of claim 8 , wherein the instructions further configure the apparatus to:
analyze audio from one or more microphones on the camera to determine signal amplitudes within the audio; and
select the subset of microphones to be activated based on the one or more microphones with a highest signal amplitude or lowest average level of noise.

10. The computing apparatus of claim 8 , wherein the instructions further configure the apparatus to:
determine, based on the sensor data from the accelerometer and a gyroscope of the camera, a direction of incident light received by a lens of the camera;
select the subset of microphones disposed on the camera to record audio based on the direction of incident light received by the lens; and
activate the subset of microphones to record audio within the current FOV of the camera.

11. The computing apparatus of claim 8 , wherein the instructions further configure the apparatus to:
receive first accelerometer data from a first accelerometer disposed on the base of the camera;
receive second accelerometer data from a second accelerometer disposed on a sensor board of the camera, wherein the sensor board of the camera is moveable to point in a range of directions; and
detect a direction the lens of the camera is pointed towards in relation to a body of the camera based on a combination of at least the first accelerometer data and the second accelerometer data;
wherein the detected direction of the lens of the camera provides dynamic information about tilt and movement of the lens in relation to the body of the camera, and wherein the camera body is a point of reference for the movement of the lens.

12. The computing apparatus of claim 8 , wherein the instructions further configure the apparatus to:
arrange the audio beamforming to match an area covered by the current FOV of the camera, wherein the audio beamforming matches a direction the lens is pointed and sound outside the current FOV is attenuated.

13. The computing apparatus of claim 8 , wherein the instructions further configure the apparatus to:
switch from a first mode of matching the audio beamforming to the current FOV to a second mode of receiving omnidirectional audio.

14. The computing apparatus of claim 8 , wherein the instructions further configure the apparatus to:
determine that one or more microphones are occluded by a portion of the camera body, preventing audio detection for a region of interest within the current FOV; and
dynamically select the subset of microphones based on the determination that each microphone within the subset is not occluded by the portion of the camera body.

15. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to:
receive sensor data from an accelerometer of a camera;
determine a position of a lens of the camera based on measurements captured by the sensor data;
determine a current field of view (FOV) of the camera based on the measurements captured by the sensor data; and
perform audio beamforming based on the current FOV of the camera by:
select a subset of microphones disposed on the camera to record audio based on the position of the lens and the current FOV of the camera; and
activate the subset of microphones disposed on the camera, wherein microphones outside the selected subset of microphones are excluded from activation so that audio originating outside of the current FOV is removed.

16. The comput

Assignees

• Cisco Tech Inc

Inventors

• Snyder Ian Matthew
• Swamy Ramkrishna

Classifications

• H04R1/028 Primary

  associated with devices performing functions other than acoustics, e.g. electric candles · CPC title

• H04R29/005

  Microphone arrays · CPC title

• H04R1/406

  microphones · CPC title

• H04R2499/11

  Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title