Image display system and image display method

Patent metadata

Field	Value
Publication number	US-10291848-B2
Application number	US-201615562562-A
Country	US
Kind code	B2
Filing date	Mar 30, 2016
Priority date	Mar 31, 2015
Publication date	May 14, 2019
Grant date	May 14, 2019

Abstract

When a user viewing an image of a conversation partner displayed on a display moves, a displayed image is more easily switched to an image reflecting a position of the user after the movement. When an image of a first user is displayed on a display of a second user, the image of the first user taken by an imaging device and an image of a background taken by the imaging device are acquired, and the acquired images are composited with each other, thereby displaying on the display a composite image in which the first user is positioned in front of the background. Moreover, when a movement of the face of the second user with respect to the display is detected, transition processing of causing the composite image displayed on the display to transition from a state before the detection of the movement of the face is carried out.

First claim

The invention claimed is:

1. An image display system for displaying an image of a first user on a display of a second user, comprising:
a server programmed to:
acquire an image of the first user taken by a camera and an image of a background taken by the camera;
compose the image of the first user and the image of the background with each other, and display on the display a composite image in which the first user is positioned in front of the background; and
detect a movement of a face of the second user with respect to the display, wherein:
when the movement of the face is detected, transition processing is carried out causing the composite image displayed on the display to transition from a state before detecting the movement of the face,
the server causing the composite image to transition to a state where one of a display position of the image of the first user in the composite image and a range included in the composite image out of the image of the background acquired by the server is shifted along a movement direction of the face by a shift amount larger than a shift amount of the other in the transition processing, and either
two directions, which are directions opposite to each other in a widthwise direction of the display, are set to a first direction and a second direction, and when detecting that the face moves toward the first direction, the server causes the composite image to transition to a state where the range is shifted toward the second direction by a shift amount larger than a shift amount of the display position in the transition processing, or
two directions, which are directions opposite to each other in a widthwise direction of the display, are set to a first direction and a second direction, and when detecting that the face moves toward the first direction, the server causes the composite image to transition to a state where the display position is shifted toward the second direction by a shift amount larger than a shift amount of the range in the transition processing.

2. The image display system according to claim 1 , wherein the server is further programmed to identify a distance from a reference surface having a direction extending through the display as a normal direction, wherein, when detecting that the face moves toward the first direction, the server causes the composite image to transition to a state where the display position is shifted toward the second direction by a shift amount corresponding to a distance between the reference surface and the first user identified, and the range is shifted toward the second direction by a shift amount corresponding to a distance between the reference surface and the background identified in the transition processing.

3. The image display system according to claim 1 , wherein the server is further programmed to identify a distance from a reference surface having a direction extending through the display as a normal direction, wherein, when the position of the first user changes in the normal direction while the camera is taking the image of the first user, the server carries out adjustment processing of adjusting a display size of the image of the first user in the composite image so as to be the display size before the position of the first user changes based on the distance between the reference surface and the first user identified.

4. The image display system according to claim 1 , wherein:
the display is installed in a room where the second user is present, and presents an appearance as furniture or an architectural material while the composite image is not displayed, the furniture or the architectural material installed in the room and having a mirror surface portion; and
the server displays the composite image on the mirror surface portion of the furniture or the architectural material forming the display when the composite image is displayed.

5. The image display system according to claim 4 , wherein the server displays on the display the composite image in which at least one of a predetermined design and a figure appears in front of the first user.

6. The image display system according to claim 1 , wherein the server acquires as the image of the background an image of a space corresponding to the background taken by the camera when the first user is not present in the space.

7. An image display method of displaying an image of a first user on a display of a second user, comprising using a computer to perform the steps of:
acquiring an image of the first user taken by a camera and an image of a background taken by the camera;
compositing the acquired image of the first user and the acquired image of the background with each other, and displaying on the display a composite image in which the first user is positioned in front of the background; and
detecting a state where a face of the second user moves with respect to the display,
wherein:
when movement of the face is detected, the computer carries out transition processing of causing the composite image displayed on the display to transition from a state before the detection of the movement of the face,
the composite image is caused to transition to a state where one of a display position of the image of the first user in the composite image and a range included in the composite image out of the image of the background is shifted along a movement direction of the face by a shift amount larger than a shift amount of the other in the transition processing, and either
two directions, which are directions opposite to each other in a widthwise direction of the display, are set to a first direction and a second direction, and when detecting that the face moves toward the first direction, causing the composite image to transition to a state where the range is shifted toward the second direction by a shift amount larger than a shift amount of the display position in the transition processing, or
two directions, which are directions opposite to each other in a widthwise direction of the display, are set to a first direction and a second direction, and when detecting that the face moves toward the first direction, causing the composite image to transition to a state where the display position is shifted toward the second direction by a shift amount larger than a shift amount of the range in the transition processing.

8. An image display system for displaying an image of a first user on a display of a second user, comprising:
a server programmed to:
acquire an image of the first user taken by a camera and an image of a background taken by the camera;
compose the image of the first user and the image of the background with each other, and display on the display a composite image in which the first user is positioned in front of the background;
detect a movement of a face of the second user with respect to the display; and
identify a distance from a reference surface having a direction extending through the display as a normal direction, wherein:
when the position of the first user changes in the normal direction while the camera is taking the image of the first user, the server carries out adjustment processing of adjusting a display size of the image of the first user in the composite image so as to be the display size before the position of the first user changes based on the distance between the reference surface and the first user identified,
when the movement of the face is detected, the server carries out transition processing of causing the composite image displayed on the display to transition from a state before detecting the movement of the face, and
the server causes the composite image to transition to a state where one of a display position of the image of the first user in the composite image and a range

Assignees

• Daiwa House Ind
• Sony Corp

Inventors

• Takahashi Yasuo
• Nakano Tsukasa
• Orime Takashi
• Takeuchi Yuichiro
• Rekimoto Junichi
• Miyajima Yasushi

Classifications

• H04N7/144 Primary

  camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact · CPC title

• H04N23/63

  by using electronic viewfinders · CPC title

• H04N23/611

  where the recognised objects include parts of the human body · CPC title

• H04N23/6811 Primary

  based on the image signal · CPC title

• H04N23/61

  based on recognised objects · CPC title