Systems and methods for anonymizing images

Patent metadata

Field	Value
Publication number	US-12488147-B2
Application number	US-202318103249-A
Country	US
Kind code	B2
Filing date	Jan 30, 2023
Priority date	Jan 30, 2023
Publication date	Dec 2, 2025
Grant date	Dec 2, 2025

Abstract

A person's privacy is protected by the law in many settings and disclosed herein are systems, methods, and instrumentalities associated with anonymizing an image of a person while still preserving the visual saliency and/or utility of the image for one or more downstream tasks. These objectives may be accomplished using various machine-learning (ML) techniques such as ML models trained for extracting identifying and residual features from the input image as well as ML models trained for transforming the identifying features into identity-concealing features and for preserving the utility features of the image. An output image may be generated based on the various ML models, wherein the identity of the person may be substantially disguised in the output image while the background and utility attributes of the original image may be substantially maintained in the output image.

First claim

What is claimed is:

1 . An apparatus, comprising:
a processor configured to:
obtain an input image of a person;
extract, based on an identity concealing machine-learning (ML) model, a set of identifying features from the input image, wherein the set of identifying features indicates, at least partially, an identity of the person;
extract, based on a residual preservation ML model, a set of residual features from the input image, wherein the set of residual features is associated with a background of the input image;
extract, based on a utility preservation ML model, a set of utility features from the input image, wherein the set of utility features is associated with a utility attribute of the input image, and wherein the utility preservation ML model has been pre-trained based on one or more tasks that use the utility attribute of the input image;
transform, based on the identity concealing ML model, the set of identifying features into a set of identity-concealing features, wherein the identity concealing ML model has been pre-trained to perform the transformation in an irreversible manner and based on a first loss designed to disentangle the set of identifying features from the set of residual features and the set of utility features; and
generate an output image of the person based on the set of identity-concealing features, the set of residual features, and the set of utility features, wherein the identity concealing ML model has been pre-trained further based on a second loss designed to maximize a difference between the set of identifying features and the set of identity-concealing features, and wherein the utility preservation ML model has been pre-trained based on a third loss designed to preserve the set of utility features in the output image.

2 . The apparatus of claim 1 , wherein the set of identity-concealing features includes randomly added noise.

3 . The apparatus of claim 1 , wherein the identity concealing ML model is implemented through an artificial neural network comprising an encoder and a decoder, wherein the encoder is trained for predicting a latent space representation of the set of identifying features, and wherein the decoder is trained for generating the set of identity-concealing features based on the latent space representation and by distancing the set of identity-concealing features from the set of identifying features.

4 . The apparatus of claim 1 , wherein the identity concealing ML model and the utility preservation ML model are trained together through a training process that comprises:
generating a preliminary output image based on an input training image; extracting, based on present parameters of the utility preservation ML model, a first plurality of utility features from the preliminary output image, wherein the first plurality of utility features is associated with the utility attribute used by the one or more tasks; extracting, based on the present parameters of the utility preservation ML model, a second plurality of utility features from the input training image, wherein the second plurality of utility features is associated with the utility attribute used by the one or more tasks; and adjusting the present parameters of the utility preservation ML model to reduce a difference between the first plurality of utility features and the second plurality of utility features.

5 . The apparatus of claim 4 , wherein the training process further comprises:
extracting, based on present parameters of the identity concealing ML model, identifying features from the input training image; generating corresponding identity-concealing features based on the identifying features extracted from the input training image; and adjusting the present parameters of the identity concealing ML model to maximize a difference between the identifying features extracted from the input training image and the identity-concealing features generated by the identity concealing ML model.

6 . The apparatus of claim 5 , wherein the utility preservation ML model and the identity concealing ML model are trained together with the residual preservation ML model, wherein the residual preservation ML model is used during the training process to extract respective residual features from the input training image and the preliminary output image, and wherein the respective present parameters of the utility preservation ML model and the identity concealing ML model are adjusted during the training process to minimize a difference between the respective residual features extracted from the input training image and the preliminary output image.

7 . The apparatus of claim 1 , wherein at least one of the utility preservation ML model or the identity concealing ML model is derived through ensemble learning.

8 . The apparatus of claim 1 , wherein the processor being configured to generate the output image of the person based on the set of identity-concealing features, the set of residual features, and the set of utility features comprises the processor being configured to combine the set of identity-concealing features, the set of residual features, and the set of utility features, and generate the output image based on the combined features.

9 . The apparatus of claim 1 , wherein the utility attribute of the input image for the one or more tasks is associated with at least one of a gaze of the person as depicted in the input image, a facial expression of the person as depicted in the input image, or a facial landmark of the person as depicted in the input image.

10 . A method of anonymizing images, the method comprising:
obtaining an input image of a person; extracting, based on an identity concealing machine-learning (ML) model, a set of identifying features from the input image, wherein the set of identifying features indicates, at least partially, an identity of the person; extracting, based on a residual preservation ML model, a set of residual features from the input image, wherein the set of residual features is associated with a background of the input image; extracting, based on a utility preservation ML model, a set of utility features from the input image, wherein the set of utility features is associated with a utility attribute of the input image, and wherein the utility preservation ML model has been pre-trained based on one or more tasks that use the utility attribute of the input image; transforming, based on the identity concealing ML model, the set of identifying features into a set of identity-concealing features, wherein the identity concealing ML model has been pre-trained to perform the transformation in an irreversible manner and based on a first loss designed to disentangle the set of identifying features from the set of residual features and the set of utility features; and generating an output image of the person based on the set of identity-concealing features, the set of residual features, and the set of utility features, wherein the identity concealing ML model has been pre-trained further based on a second loss designed to maximize a difference between the set of identifying features and the set of identity-concealing features, and wherein the utility preservation ML model has been pre-trained based on a third loss designed to preserve the set of utility features in the output image.

11 . The method of claim 10 , wherein the set of identity-concealing features includes randomly added noise.

12 . The method of claim 10 , wherein the identity concealing ML model is implemented through an artificial neural network comprising an encoder and a decoder, wherein the encoder is trained for predicting a latent space representation

Assignees

• Shanghai United Imaging Intelligence Co Ltd

Inventors

• Planche Benjamin
• Cai Zikui
• Gao Zhongpai
• Wu Ziyan
• Zheng Meng
• Chen Terrence

Classifications

• G06V10/7715 Primary

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06V40/171

  Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships · CPC title

• G06V40/172

  Classification, e.g. identification · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V40/174

  Facial expression recognition · CPC title