Apparatus, systems, and methods for audio and video filtering for electronic user devices

Patent metadata

Field	Value
Publication number	US-12457302-B2
Application number	US-202117483552-A
Country	US
Kind code	B2
Filing date	Sep 23, 2021
Priority date	Sep 23, 2021
Publication date	Oct 28, 2025
Grant date	Oct 28, 2025

Abstract

Apparatus, systems, and methods for audio and video filtering for electronic user devices are disclosed. An example apparatus includes at least one memory, instructions in the apparatus, and processor circuitry to execute instructions to detect a visual event based on image data, the visual event representative of an activity associated with a likelihood of noise, the image data associated with a video stream output by a camera associated with a user device, and in response to the detection of the visual event, apply an audio filter to a portion of an audio stream corresponding to the image data in the video stream.

First claim

What is claimed is:

1. An apparatus comprising:
at least one memory;
machine-readable instructions; and
at least one processor circuit to be programmed by the machine-readable instructions to:
detect a first visual event associated with an activity in first image data of a video stream output by a camera associated with a user device, the activity associated with a likelihood of noise, the first image data associated with a first time;
invoke a neural network to process the first image data to classify the first visual event as one of a first stage of the activity or a second stage of the activity, the first stage preceding generation of the noise, the second stage associated with the generation of the noise;
cause application of an audio filter to a first portion of an audio stream corresponding to the first image data based on classification of the first visual event as the first stage or the second stage;
detect a second visual event associated with the activity based on second image data of the video stream, the second image data associated with a second time, the second time after the first time;
invoke the neural network to process the second image data to classify the second visual event as a third stage of the activity, the third stage different than the first stage and the second stage; and
cause presentation of the second image data without audio filtering based on classification of the second visual event as the third stage.

2. The apparatus of claim 1 , wherein one or more of the at least one processor circuit is to cause application of the audio filter for a duration of time based on the activity.

3. The apparatus of claim 1 , wherein one or more of the at least one processor circuit is to cause application of the audio filter by causing the first portion of the audio stream to be muted.

4. The apparatus of claim 1 , wherein one or more of the at least one processor circuit is to cause application of the audio filter by filtering the noise from the first portion of the audio stream.

5. The apparatus of claim 1 , wherein the audio filter is a first filter, and one or more of the at least one processor circuit is to cause application of a second filter to a third portion of the video stream including the first visual event.

6. The apparatus of claim 5 , wherein one or more of the at least one processor circuit is to cause application of the second filter by causing the third portion of the video stream to be blurred or concealed.

7. The apparatus of claim 1 , wherein the activity includes an activity performed by a user of the user device.

8. The apparatus of claim 1 , wherein the activity includes an event in an environment in which the user device is located.

9. The apparatus of claim 1 , wherein the first stage of the activity includes a hand of a user moving toward another portion of a body of the user, and the third stage of the activity includes the hand of the user moving away from the portion of the body of the user.

10. The apparatus of claim 1 , wherein one or more of the at least one processor circuit is to cause application of the audio filter by causing a volume of the first portion of the audio stream to be reduced.

11. At least one non-transitory computer-readable storage medium comprising machine-readable instructions to cause at least one processor circuit to at least:
detect a first visual event associated with a user activity in a first frame of a video stream generated by a camera associated with a user device, the user activity associated with a likelihood of noise, the first frame associated with a first time;
invoke a neural network to process the first frame to classify the first visual event as one of a first stage of the user activity or a second stage of the user activity, the first stage preceding generation of the noise, the second stage associated with the generation of the noise;
cause application of a filter to a first portion of an audio stream corresponding to the first frame to generate a filtered audio stream based on classification of the first visual event as the first stage or the second stage;
cause the filtered audio stream to be output for transmission;
detect a second visual event associated with the user activity in a second frame of the video stream, the second frame associated with a second time, the second time after the first time;
invoke the neural network to process the second frame to classify the second visual event as a third stage of the user activity, the third stage different than the first stage and the second stage; and
cause presentation of a second portion of the audio stream corresponding to the second frame without filtering based on classification of the second visual event as the third stage.

12. The at least one non-transitory computer-readable medium of claim 11 , wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to:
detect a third visual event associated with the user activity in a third frame of the video stream, the third frame associated with a third time, the third time between the first time and the second time; and
cause the application of the filter to be maintained in response to the detection of the third visual event.

13. The at least one non-transitory computer-readable medium of claim 11 , wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to cause application of the filter for a duration of time based on the user activity.

14. The at least one non-transitory computer-readable medium of claim 11 , wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to cause application of the filter to mute the first portion of the audio stream.

15. The at least one non-transitory computer-readable medium of claim 11 , wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to cause application of the filter to filter the noise from the first portion of the audio stream.

16. The at least one non-transitory computer-readable medium of claim 11 , wherein the filter is a first filter, and the machine-readable instructions are to cause one or more of the at least one processor circuit to cause application of a second filter to a third portion of the video stream including the first visual event.

17. The at least one non-transitory computer-readable medium of claim 16 , wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to cause application of the second filter to conceal the third portion of the video stream.

18. A method comprising:
detecting, by at least one processor circuit programmed by at least one instruction, a first visual event associated with an activity in first image data of a video stream output by a camera associated with a user device, the activity associated with a likelihood of noise, the first image data associated with a first time;
invoking a neural network to process the first image data to classify the first visual event as one of a first stage of the activity or a second stage of the activity, the first stage preceding generation of the noise, the second stage associated with the generation of the noise;
causing, by one or more of the at least one processor circuit, application of an audio filter to a first portion of an audio stream corresponding to the first image data based on classification of the first visual event as the first stage or the second stage;
detecting, by one or more of t

Assignees

• Intel Corp

Inventors

• Browning David
• Rosenzweig Michael

Classifications

• H04N21/4318

  by altering the content in the rendering process, e.g. blanking, blurring or masking an image region (image enhancement or restoration in general G06T5/00) · CPC title

• H04N21/4394

  involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58) · CPC title

• H04N21/45457

  applied to a time segment · CPC title

• H04N21/4396

  by muting the audio signal · CPC title

• H04N7/15 Primary

  Conference systems · CPC title