Method of image adjustment and apparatus

Patent metadata

Field	Value
Publication number	US-2025272781-A1
Application number	US-202519058684-A
Country	US
Kind code	A1
Filing date	Feb 20, 2025
Priority date	Feb 22, 2024
Publication date	Aug 28, 2025
Grant date	—

Abstract

A method of image adjustment for an image sequence, comprises the steps of generating a first estimate, of the gaze positions of multiple viewers of an image, modifying a distribution of image quality for an image, responsive to the first estimate, and providing the image to the multiple viewers, generating a second estimate, of the effect of modifying the distribution of image quality on the gaze positions of at least a subset of viewers, and modifying the distribution of image quality for a subsequent image, responsive to the second estimate.

First claim

1 . A method of image adjustment for an image sequence, comprising the steps of:
generating a first estimate, of gaze positions of multiple viewers of an image; modifying a distribution of image quality for an image, responsive to the first estimate, and providing the image to the multiple viewers; generating a second estimate, of an effect of modifying the distribution of image quality on gaze positions of at least a subset of viewers; and modifying the distribution of image quality for a subsequent image, responsive to the second estimate.

2 . A method according to claim 1 , in which the step of modifying a distribution of image quality comprises
modifying the distribution of image quality responsive to one or more metrics selected from a list consisting of:
a relative density of gaze positions; and
a degree of overlap of regions corresponding to gaze positions of multiple viewers of the image.

3 . A method according to claim 2 , in which the distribution of image quality is in one or more discrete steps, responsive to one or more corresponding predetermined thresholds of the, or each, metric.

4 . A method according to claim 3 , in which the one or more predetermined thresholds are set responsive to a predetermined budget for a provision of a modified image.

5 . A method according to claim 1 , in which the step of generating a second estimate of the effect of modifying the distribution of image quality on the gaze positions of at least a subset of viewers comprises the steps of:
generating a third estimate, of the gaze positions of multiple viewers of the subsequent image; evaluating a change of the gaze positions responsive to the distribution of image quality in the image presented to the viewers; and generating the second estimate responsive to the evaluation of the change.

6 . A method according to claim 1 , wherein, if a convergence of the gaze positions of multiple viewers of an image exceeds a first predetermined threshold, the method further comprises the step of modifying the distribution of image quality for a subsequent image, to reduce an image quality in an area of convergence.

7 . A method according to claim 1 , wherein if a convergence of the gaze positions of multiple viewers of an image is below a second predetermined threshold for a predetermined period, the method further comprises the step of modifying the distribution of image quality for a subsequent image, to an image quality in multiple areas, responsive to the gaze positions.

8 . A method according to claim 1 , wherein
if modifying the distribution of image quality for a subsequent image reduces image quality in an area of an image, or for an object in an image, that is being viewed by a threshold number of viewers for a threshold period of time; and the second estimate indicates a reduction in a number of viewers in that area, then the method further comprises the step of:
modifying the distribution of image quality for a subsequent image, to increase an image quality in that area.

9 . A method according to claim 8 in the step of modifying the distribution of image quality for a subsequent image, to increase an image quality in that area comprises estimating whether viewers previously in that area are moving to or have moved to a higher quality area, and if so, not modifying the distribution of image quality in that area.

10 . A method according to claim 1 , comprising the steps of:
based on successive estimates of the gaze positions of multiple viewers of successive images; estimating an area of future convergence of at least a subset of viewers; and modifying the distribution of image quality for a subsequent image, to increase an image quality in that area of future convergence.

11 . A method according to claim 1 , comprising the steps of:
based on successive estimates of the gaze positions of multiple viewers of successive images; estimating a pair of areas that at least a subset of viewers switch between over time; and modifying the distribution of image quality for a subsequent image, to increase an image quality in both areas.

12 . A method according to claim 1 , in which the step of modifying the distribution of image quality for a subsequent image comprises one or more selected from a list consisting of:
modifying the distribution of image quality during generation of the subsequent image; and modifying the distribution of image quality when streaming the subsequent image.

13 . A computer program comprising computer executable instructions adapted to cause a computer system to perform a method of method of image adjustment for an image sequence, comprising the steps of:
generating a first estimate, of gaze positions of multiple viewers of an image; modifying a distribution of image quality for an image, responsive to the first estimate, and providing the image to the multiple viewers; generating a second estimate, of an effect of modifying the distribution of image quality on gaze positions of at least a subset of viewers; and modifying the distribution of image quality for a subsequent image, responsive to the second estimate.

14 . A computer program according to claim 13 , in which the step of modifying a distribution of image quality comprises
modifying the distribution of image quality responsive to one or more metrics selected from a list consisting of:
a relative density of gaze positions; and
a degree of overlap of regions corresponding to gaze positions, of multiple viewers of the image.

15 . A computer program according to claim 14 , in which the distribution of image quality is in one or more discrete steps, responsive to one or more corresponding predetermined thresholds of the, or each, metric.

16 . A computer program according to claim 15 , in which the one or more predetermined thresholds are set responsive to a predetermined budget for a provision of a modified image.

17 . A computer program according to claim 13 , in which the step of generating a second estimate of the effect of modifying the distribution of image quality on the gaze positions of at least a subset of viewers comprises the steps of:
generating a third estimate, of the gaze positions of multiple viewers of the subsequent image; evaluating a change of the gaze positions responsive to the distribution of image quality in the image presented to the viewers; and generating the second estimate responsive to the evaluation of the change.

18 . An image adjustment system for use with an image sequence, comprising:
a gaze estimation processor configured to generate a first estimate, of gaze positions of multiple viewers of an image; an image processor configured to modify a distribution of image quality for an image, responsive to the first estimate, and provide the image to the multiple viewers; an effect estimation processor configured to generate a second estimate, of an effect of modifying the distribution of image quality on gaze positions of at least a subset of viewers; and the image processor configured to modify the distribution of image quality for a subsequent image, responsive to the second estimate.

19 . An image adjustment system according to claim 18 , in which:
an effect estimation processor is configured, when generating the second estimate, to receive from the gaze estimation processor a third estimate, of the gaze positions of multiple viewers of the subsequent image; evaluate a change of the gaze positions responsive to the distribution of image quality in the image presented to

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Leung Jun Yen
• Connor Patrick John
• Monti Maria Chiara
• Green Lawrence
• Cerrato Maurizio
• Gupta Rajeev

Classifications

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• H04N21/44218

  Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45) · CPC title

• H04N21/258

  Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data {(arrangements for services using the result on the distributing side of broadcast systems H04H60/66; profiles in network data switching protocols H04L67/30)} · CPC title

• H04N13/383

  for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes · CPC title

• H04N13/368

  for two or more viewers · CPC title