Device and rendering environment tracking

Patent metadata

Field	Value
Publication number	US-12382235-B2
Application number	US-202318161645-A
Country	US
Kind code	B2
Filing date	Jan 30, 2023
Priority date	Feb 1, 2022
Publication date	Aug 5, 2025
Grant date	Aug 5, 2025

Abstract

Images of an actual rendering environment are acquired through image sensors operating in conjunction with a media consumption system. The acquired images of the actual rendering environment are used to predict audio characteristics of objects present in the actual rendering environment. Spatial audio rendered, to a user in the actual rendering environment, by audio speakers operating in conjunction with the media consumption system is adjusted or modified based at least in part on the audio characteristics of the objects present in the actual rendering environment.

First claim

What is claimed is:

1. A method comprising:
acquiring a plurality of images of an actual rendering environment through one or more image sensors operating in conjunction with a media consumption system;
using the plurality of images of the actual rendering environment to predict audio characteristics of objects present in the actual rendering environment;
adjusting spatial audio rendered, to a user in the actual rendering environment, by a plurality of audio speakers operating in conjunction with the media consumption system based at least in part on the audio characteristics of the objects present in the actual rendering environment;
wherein the spatial audio is derived from received audio data by the media consumption system; wherein the spatial audio as adjusted reproduces a recorded rendering environment for which the received audio data is intended to be rendered; wherein the recorded rendering environment comprises a reference audio speaker configuration that is different from an actual audio speaker configuration deployed in the actual rendering environment.

2. The method of claim 1 , wherein the plurality of images of the actual rendering environment is used to identify a make and a model of headphones, among the objects present in the actual rendering environment, worn by the user.

3. The method of claim 1 , wherein the plurality of images of the actual rendering environment is used to identify a make and a model of a non-headphone audio speaker among the objects present in the actual rendering environment.

4. The method of claim 1 , wherein the plurality of images of the actual rendering environment is used to predict audio characteristics of a non-audio-speaker object among the objects present in the actual rendering environment.

5. The method of claim 1 , wherein the plurality of images of the actual rendering environment is acquired over a time duration; wherein the media consumption system selects default audio characteristics of at least a subset of objects, among the objects present in the actual rendering environment, in rendering audio to the user when the time duration begins.

6. The method of claim 1 , wherein the audio characteristics are predicted by an audio characteristics predictive model trained with training images of a plurality of objects of different types in a plurality of different rendering environments and ground truths indicating respective audio characteristics for the plurality of objects of different types in the plurality of different rendering environments.

7. The method of claim 1 , wherein a subset of audio characteristics among the audio characteristics comprises one of: a reverberation audio characteristic or an echo audio characteristic, of an object present in the actual rendering environment.

8. The method of claim 1 , wherein the recorded rendering environment is specified in audio metadata in a media data signal that carries the received audio data to the media consumption system.

9. The method of claim 1 , wherein the audio characteristics are used to select specific personalized equalization operational parameters for rendering audio to the user in the actual rendering environment.

10. The method of claim 1 , wherein a new audio portion is created and rendered by a first audio speaker to reduce an audio leakage caused by a second audio speaker in an ear of the user.

11. An apparatus performing the method of claim 1 .

12. A non-transitory computer readable storage medium, storing software instructions, which when executed by one or more processors cause performance of the method recited in claim 1 .

13. A computing device comprising one or more processors and one or more storage media, storing a set of instructions, which when executed by one or more processors cause performance of the method recited in claim 1 .

Assignees

• Dolby Laboratories Licensing Corp

Inventors

• Ninan Ajit
• Rozzi William Anthony

Classifications

• H04S7/305

  Electronic adaptation of stereophonic audio signals to reverberation of the listening space (H04S7/301 takes precedence) · CPC title

• H04S7/304 Primary

  For headphones · CPC title

• H04S2400/13

  Aspects of volume control, not necessarily automatic, in stereophonic sound systems · CPC title

• H04R27/00

  Public address systems (circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; {monitoring or testing arrangements for public address systems H04R29/007}; amplifiers H03F) · CPC title

• H04S2420/01

  Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD] · CPC title