Providing user video having a virtual curtain to an online conference

Patent metadata

Field	Value
Publication number	US-9282285-B2
Application number	US-201313913748-A
Country	US
Kind code	B2
Filing date	Jun 10, 2013
Priority date	Jun 10, 2013
Publication date	Mar 8, 2016
Grant date	Mar 8, 2016

Abstract

A technique provides user video to an online conference. The technique involves receiving a live user video signal from a camera device. The live user video signal defines a field of view. The technique further involves automatically identifying live initial content of a presentation region within the field of view and live initial content of a privacy region (e.g., a background region) within the field of view. The technique further involves generating, as the user video signal to the online conference, a modified user video signal based on the live user video signal. The modified video signal includes (i) the live initial content of the presentation region within the field of view and (ii) modified video content in place of the live initial content of the privacy region within the field of view. Such operation effectively forms a virtual curtain in which anything in the background is hidden.

First claim

What is claimed is:

1. A method of providing a user video signal to an online conference, the method comprising:
receiving a live user video signal from a camera device, the live user video signal defining a field of view;
automatically identifying live initial content of a presentation region within the field of view and live initial content of a privacy region within the field of view; and
generating, as the user video signal to the online conference, a modified user video signal based on the live user video signal, the modified video signal including (i) the live initial content of the presentation region within the field of view and (ii) modified video content in place of the live initial content of the privacy region within the field of view;
wherein the camera device includes a camera and a depth sensor;
wherein receiving the live user video signal from the camera device includes acquiring, as the live user video signal, a series of video frames including captured images of a user from the camera and depth measurement data from the depth sensor; and
wherein automatically identifying the live initial content of the presentation region and the live initial content of the privacy region includes:
isolating the live initial content of the privacy region from the live initial content of the presentation region based on the depth measurement data.

2. A method as in claim 1 wherein each video frame includes an array of pixels, each pixel having an original pixel value and a depth value, the depth values of the pixels forming at least some of the depth measurement data; and
wherein isolating the live initial content of the privacy region from the live initial content of the presentation region based on the depth measurement data includes, for each pixel, (i) maintaining the original pixel value when the depth value of that pixel is below a predefined depth threshold and (ii) replacing the original pixel value with a different pixel value when the depth value of that pixel exceeds the predefined depth threshold.

3. A method as in claim 2 wherein replacing the original pixel value with the different pixel value when the depth value of that pixel exceeds the predefined depth threshold includes:
modifying the original pixel value based on a blurring algorithm to blur the live initial content of the privacy region.

4. A method as in claim 3 wherein modifying the original pixel value based on the blurring algorithm includes:
varying a blur effect of the blurring algorithm to (i) mildly blur the live initial content of the privacy region which is closest to the presentation region, and (ii) substantially blur the live initial content of the privacy region which is farthest from the presentation region.

5. A method as in claim 4 , further comprising:
after generating the modified user video signal based on the live user video signal, receiving blur control commands which selectively disable and enable use of the blurring algorithm during subsequent operation.

6. A method as in claim 4 , further comprising:
after generating the modified user video signal based on the live user video signal, receiving blur variation commands which incrementally vary an amount of blurring performed via the blurring algorithm during subsequent operation.

7. A method as in claim 1 wherein the depth sensor is an infrared laser sensor; and wherein the depth measurement data includes infrared laser measurements of distances between the infrared laser sensor and an environment of the user.

8. A method as in claim 1 wherein the camera is a stereo camera; and wherein the depth measurement data includes depth estimate values of distances between the stereo camera and an environment of the user.

9. A method as in claim 1 wherein the camera is a webcam coupled to a user computer; and wherein the method further comprises:
rendering, by a display of the user computer, an online conference video of the online conference to enable a user of the user computer to view the online conference.

10. A method as in claim 9 wherein generating the modified video signal includes:
outputting, by the user computer, the modified video signal to an external online conference server through a computer network, the external online conference server being constructed and arranged to manage the online conference.

11. A method as in claim 1 wherein each video frame includes an array of pixels, each pixel having an associated depth value;
wherein isolating the live initial content of the privacy region includes:
(i) dynamically establishing a depth threshold based on distance of a foreground object in the field of view from the camera device, the depth threshold being greater than the distance of the foreground object in the field of view from the camera device, and
(ii) isolating the live initial content of the privacy region from the live initial content of the presentation region based on comparison of the associated depth value of each pixel to the dynamically established depth threshold.

12. A method as in claim 1 wherein each video frame includes an array of pixels, each pixel being constructed and arranged to have an associated depth value; and
wherein isolating the live initial content of the privacy region includes:
processing pixels lacking associated depth values as live initial content of the presentation region for inclusion in the modified video signal to prevent loss of content.

13. A method as in claim 1 wherein each video frame includes an array of pixels, each pixel being constructed and arranged to have an associated depth value; and
wherein isolating the live initial content of the privacy region includes:
processing pixels lacking associated depth values as live initial content of the privacy region for replacement with a virtual curtain in the modified video signal to protect privacy.

14. A method as in claim 1 wherein each video frame includes an array of pixels, each pixel being constructed and arranged to have an associated depth value; and wherein the method further comprises:
prior to generating the modified video signal, performing a preprocessing operation which associates new depth values to particular pixels initially lacking associated depth values to repair the particular pixels.

15. A method as in claim 1 wherein acquiring the series of video frames includes:
obtaining, for a particular video frame, an array of pixels that forms a particular captured image, each pixel of the array having (i) a pixel value that identifies a color for an object in the particular captured image and (ii) a depth value that defines a measured physical distance between the depth sensor and the object in the particular captured image.

16. An electronic apparatus, comprising:
a camera device;
memory; and
control circuitry coupled to the camera device and the memory, the memory storing instructions which, when carried out by the control circuitry, cause the control circuitry to:
receive a live user video signal from the camera device, the live user video signal defining a field of view,
automatically identify live initial content of a presentation region within the field of view and live initial content of a privacy region within the field of view, and
generate, as a user video signal to an online conference, a modified user video signal based on the live user video signal, the modified video signal including (i) the live initial content of the presentation region within the field of view and (ii) modified video content in place of the live initial content of the privacy region within the field of view

Assignees

• Citrix Systems Inc

Inventors

• Winterstein Florian
• Alexandrov Albert

Classifications

• H04N7/15 Primary

  Conference systems · CPC title

• H04N7/141

  between two video terminals, e.g. videophone (telephonic communication systems combined with television receiver for reception of entertainment or information matter H04M11/085) · CPC title

• G06T5/002

  Physics · mapped topic

• H04N7/147

  Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q) · CPC title

• G06T5/70 Primary

  Denoising; Smoothing · CPC title