Tiering and manipulation of peer's heads in a telepresence system

Patent metadata

Field	Value
Publication number	US-9756288-B2
Application number	US-201314767336-A
Country	US
Kind code	B2
Filing date	Apr 10, 2013
Priority date	Apr 10, 2013
Publication date	Sep 5, 2017
Grant date	Sep 5, 2017

Abstract

In a telepresence system, a local station receives images of participants at remote stations processed to isolate the remote participant in the image from image background. The local station merges the images of the participants at the remote stations to generate an output image in which a first remote participant appears to partially overlay another participant. The output image undergoes display at the local station.

First claim

The invention claimed is:

1. A method of processing data at a local station to produce data representing images of participants at remote stations, comprising the steps of:
receiving at the local station data representing the images of the participants at the remote stations and data indicating facings of the participants at the remote stations with respect to cameras at the remote stations;
processing at the local station the data representing the images of the participants at the remote stations to isolate the participant in the image from image background;
merging, at the local station, data representing the processed images of the participants at the remote stations to generate data representing an output image in which a first processed participant image appears to partially overlay a second processed participant image and in which the first processed participant image appears in the output image horizontally translated relative to the second processed participant image by a distance determined in response to the data indicating the facings of the first and second participants; and
providing the data representing the output image at an output of the local station suitable for coupling to a display device.

2. The method according to claim 1 wherein the receiving step comprises:
accepting from the remote stations images of participants already processed to isolate the participant in the image from image background.

3. The method according to claim 1 further including the step of normalizing the first and second processed participant images in height.

4. The method according to claim 3 wherein the normalizing occurring in accordance with a distance from the top of a participant's head to a centerline of the participant's eyes in the image of the participant at the remote station.

5. The method according to claim 1 further including the step of scaling at least one of the processed participant images to cause the second processed participant image to appear smaller than the first processed participant image, whereby the second processed participant images appears further away due to perspective.

6. The method according to claim 1 wherein the distance the first processed participant image appears in the output image horizontally translated from the second processed participant image has a first value when at least one of the participants in the first and second processed participant images has a first facing and has a second value less than the first value otherwise.

7. The method according to claim 1 wherein at least one of the processed participant images is horizontally flipped with respect to the corresponding image of the participant at the remote station.

8. The method according to claim 1 further comprising the step of:
accepting an ordering of remote participants from a local participant,
whereby a local participant can select which of the remote participants corresponds to the first and second processed participant images.

9. Apparatus comprising:
an input buffer receiving and storing data representing images of a plurality of participants, each at one of a plurality of remote stations, and to receive and store data representing facings of the participants at the remote stations with respect to cameras at the remote stations;
a video controller coupled to the input buffer and processing the data representing the images to isolate the participant in the image from the image background and to generate data representing an output image in which a first processed participant image appears to partially overlay a second processed participant image and in which the first processed participant image appears in the output image horizontally translated relative to the second processed participant image by a distance determined in response to the data indicating the facings of the first and second participants; and
an output buffer coupled to the video controller and storing the data representing the output image and supplying the data representing the output image at an output suitable for coupling to a display device.

10. The apparatus according to claim 9 wherein the images stored in the input buffer have already undergone processing to isolate the participant in the image from image background.

11. The apparatus according to claim 9 wherein the video controller processes the data representing the first and second processed participant images to normalize the first and second processed participant images in height.

12. The apparatus according to claim 11 wherein the video controller processes the data representing the first and second processed participant images to normalize the first and second processed participant images in height in accordance with a distance from the top of a participant's head to a centerline of the participant's eyes in the image of the participant at the remote station.

13. The apparatus according to claim 9 wherein the video controller processes the data representing the first and second processed participant images to scale the second processed participant image to appear smaller than the first processed participant image, whereby the second processed participant images appears further away due to perspective.

14. The apparatus according to claim 9 wherein the distance the first processed participant image appears in the output image horizontally translated from the second processed participant image has a first value when at least one of the participants in the first and second processed participant images has a first facing, and has a second value less than the first value when the at least one participant has a second facing.

15. The apparatus according to claim 9 wherein the video controller processes the data representing at least one of the first and second processed participant images to horizontally flip the at least one of the processed participant images with respect to the corresponding image of the participant at remote station.

16. The apparatus according to the claim 9 wherein the video controller processes the data representing the first and second processed participant images to alter an ordering for the first and second processed participant images responsive to a command from a local participant to select which of the remote participants corresponds to the first and second processed participant images.

17. The method of claim 1 wherein any of one of the first and second processed participant images serves as a placeholder image for at least a portion of time corresponding to the participant being out of view at the corresponding remote station.

18. The method of claim 17 wherein the placeholder image constitutes a freeze frame of the corresponding participant.

19. The method of claim 18 wherein the freeze frame undergoes processing to further indicate status of the corresponding participant.

20. The apparatus according to claim 9 wherein one of the images of the plurality of participants constitutes a placeholder image for at least a portion of a time that the corresponding participant is out of view at the corresponding remote station.

21. The apparatus according to claim 20 wherein the placeholder image constitutes a freeze frame of the corresponding participant.

22. The apparatus according to claim 21 wherein the freeze frame undergoes processing to further indicate the status of the corresponding participant.

Assignees

• Thomson Licensing

Inventors

• Huber Mark J
• Walker Mark Leroy
• Redmann William Gibbens

Classifications

• G06T7/10

  Segmentation; Edge detection (motion-based segmentation G06T7/215) · CPC title

• G06K9/52

  Physics · mapped topic

• H04N21/4223

  Cameras (H04N23/00 takes precedence) · CPC title

• H04N7/142

  Constructional details of the terminal equipment, e.g. arrangements of the camera and the display · CPC title

• G06T11/60

  Creating or editing images; Combining images with text · CPC title