Removing image overlays

Patent metadata

Field	Value
Publication number	US-2025117994-A1
Application number	US-202418426758-A
Country	US
Kind code	A1
Filing date	Jan 30, 2024
Priority date	Oct 9, 2023
Publication date	Apr 10, 2025
Grant date	—

Abstract

In implementation of techniques for removing image overlays, a computing device implements a reflection removal system to receive an input RAW digital image, the input RAW digital image including both a base image and an overlay image. Using a machine learning model, the reflection removal system segments the base image from the overlay image. The reflection removal system generates an output RAW digital image that includes the base image and displays the output RAW digital image in a user interface.

First claim

What is claimed is:

1 . A method comprising:
receiving, by a processing device, an input RAW digital image, the input RAW digital image including both a base image and an overlay image; segmenting, by the processing device, the base image from the overlay image from the input RAW digital image using a machine learning model; generating, by the processing device, an output RAW digital image that includes the base image; and displaying, by the processing device, the output RAW digital image in a user interface.

2 . The method of claim 1 , wherein the overlay image depicts a reflection and is layered over the base image in the input RAW digital image.

3 . The method of claim 1 , further comprising receiving an additional RAW input digital image captured from an angle that is different from an angle used to capture the input RAW digital image.

4 . The method of claim 3 , wherein the input RAW digital image and the additional RAW input digital image are captured simultaneously by different cameras.

5 . The method of claim 3 , wherein the additional RAW input digital image includes information about light in a physical environment captured by the input RAW digital image.

6 . The method of claim 5 , wherein the machine learning model segments the base image from the overlay image based on the information about the light in the physical environment from the input RAW digital image.

7 . The method of claim 1 , wherein the machine learning model is trained on RAW digital images formed by combining two RAW digital images.

8 . The method of claim 1 , further comprising generating digital content using the machine learning model to replace the overlay image in the output RAW digital image.

9 . The method of claim 1 , further comprising generating an additional output RAW digital image that includes the overlay image.

10 . A system comprising:
a memory component; and a processing device coupled to the memory component, the processing device to perform operations comprising:
receiving an input digital image captured by a rear facing camera, the input digital image including both a base image and an overlay image;
receiving a second digital image captured by a forward facing camera;
separating the base image from the overlay image from the input digital image using a machine learning model based on content of the second digital image; and
generating an output digital image that includes the base image for display in a user interface.

11 . The system of claim 10 , wherein the overlay image depicts a reflection and is layered over the base image in the input digital image.

12 . The system of claim 10 , wherein the second digital image is captured from an angle that is different from an angle used to capture the input digital image.

13 . The system of claim 12 , wherein the second digital image includes information about light in a physical environment captured by the input digital image.

14 . The system of claim 13 , wherein the machine learning model segments the base image from the overlay image based on the information about the light in the physical environment from the input digital image.

15 . The system of claim 10 , wherein the machine learning model is trained on digital images formed by combining two digital images.

16 . The system of claim 10 , further comprising generating digital content using the machine learning model to replace the overlay image in the output digital image.

17 . A non-transitory computer-readable storage medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising:
receiving an input RAW digital image, the input RAW digital image including both a base image and an overlay image; segmenting the base image from the overlay image from the input RAW digital image using a machine learning model; generating an output RAW digital image that includes the base image; and displaying the output RAW digital image in a user interface.

18 . The non-transitory computer-readable storage medium of claim 17 , wherein the input RAW digital image is formed by aligning and merging a burst of initial RAW digital images, and the overlay image depicts a reflection and is layered over the base image in the input RAW digital image.

19 . The non-transitory computer-readable storage medium of claim 17 , further comprising receiving an additional RAW input digital image captured from an angle that is different from an angle used to capture the input RAW digital image and includes information about light in a physical environment captured by the input RAW digital image.

20 . The non-transitory computer-readable storage medium of claim 19 , wherein the machine learning model segments the base image from the overlay image based on information about the light in the physical environment from the input RAW digital image.

Assignees

• Adobe Inc

Inventors

• Kee Eric Randall
• Pikielny Adam Ahmed
• Levoy Marc Stewart

Classifications

• G06T2207/20081

  Training; Learning · CPC title

• G06T5/77

  Retouching; Inpainting; Scratch removal · CPC title

• G06T5/60

  using machine learning, e.g. neural networks · CPC title

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title

• G06V10/24 Primary

  Aligning, centring, orientation detection or correction of the image · CPC title