System and Method for Virtual Clothes Fitting Based on Video Augmented Reality in Mobile Phone

Patent metadata

Field	Value
Publication number	US-2017018024-A1
Application number	US-201514799907-A
Country	US
Kind code	A1
Filing date	Jul 15, 2015
Priority date	Jul 15, 2015
Publication date	Jan 19, 2017
Grant date	—

Abstract

A system and method for virtual clothes fitting based on video augmented reality in a mobile wireless device is disclosed. In an embodiment, a method for virtual cloth-fitting with video augmentation in a mobile wireless device includes receiving a video stream of an upper body of a user; detecting a face in the video stream; detecting a shoulder contour; determining keypoints of the shoulder and neck portions of the video stream of the upper body of the user according to the detected face and the detected shoulder contour; mapping an image of clothes to the video stream of the upper body of the user according to the keypoints; and displaying an augmented video stream of the upper body of the user with the image of the clothes overlaid over a portion of the video stream of the upper body of the user.

First claim

What is claimed is:

1 . A method for virtual cloth-fitting with video augmentation in a mobile wireless device, comprising:
receiving a video stream of an upper body of a user; detecting a face in the video stream; detecting a shoulder contour; determining keypoints of shoulder and neck portions of the video stream of the upper body of the user according to the detected face and the detected shoulder contour; mapping an image of clothes to the video stream of the upper body of the user according to the keypoints; and displaying an augmented video stream of the upper body of the user with the image of the clothes overlaid over a portion of the video stream of the upper body of the user.

2 . The method of claim 1 , further comprising:
tracking movement of the user; remapping the image of the clothes to the movement; and displaying a remapped augmented video stream of the upper body with the remapped image of the clothed overlaid over a portion of the video stream of the upper body of the user, wherein the video display of the clothes moves with the movement of the image of the user in the augmented video stream.

3 . The method of claim 2 , wherein remapping the image of the clothes comprises transforming relevant positions on a plane of the clothes with corresponding positions on a plane of the image of the user's upper body.

4 . The method of claim 2 , wherein the remapping the image of the clothes comprises a Delaunay triangulation.

5 . The method of claim 1 , wherein the shoulder contour is automatically determined.

6 . The method of claim 1 , wherein the face and the shoulder contour are determined according to user input.

7 . The method of claim 6 , further comprising attempting to detect the face, presenting a candidate face region, and receiving a user input confirming that the candidate face region corresponds to the user's face.

8 . The method of claim 6 , further comprising receiving user input moving a shoulder contour outline.

9 . A mobile wireless device configured for virtual cloth-fitting with video augmentation, comprising:
a processor; and a non-transitory computer readable storage medium storing programming for execution by the processor, the programming including instructions to:
receive a video stream of an upper body of a user;
detect a face in the video stream;
detect a shoulder contour;
determine keypoints of shoulder and neck portions of the video stream of the upper body of the user according to the detected face and the detected shoulder contour;
map an image of clothes to the video stream of the upper body of the user according to the keypoints; and
display an augmented video stream of the upper body of the user with the image of the clothes overlaid over a portion of the video stream of the upper body of the user.

10 . The mobile wireless device of claim 9 , wherein the programming further includes instructions to:
track movement of the user; remap the image of the clothes to the movement; and display a remapped augmented video stream of the upper body with the remapped image of the clothed overlaid over a portion of the video stream of the upper body of the user, wherein the video display of the clothes moves with the movement of the image of the user in the augmented video stream.

11 . The mobile wireless device of claim 10 , wherein the instructions to remap the image of the clothes comprises instructions to transform relevant positions on a plane of the clothes with corresponding positions on a plane of the image of the user's upper body.

12 . The mobile wireless device of claim 10 , wherein the instructions to remap the image of the clothes comprises instructions to execute a Delaunay triangulation.

13 . The mobile wireless device of claim 9 , wherein the shoulder contour is automatically determined.

14 . The mobile wireless device of claim 9 , wherein the face and the shoulder contour are determined according to user input.

15 . The mobile wireless device of claim 14 , wherein the programming further comprises instructions to attempt to detect the face, present a candidate face region, and receive a user input confirming that the candidate face region corresponds to the user's face.

16 . A mobile wireless device configured for virtual cloth-fitting with video augmentation, comprising:
a camera; a live video stream acquisition unit comprising a processor, the live video stream acquisition unit configured to receive a live video stream from the camera; a system initialization unit comprising a processor, the system initialization unit configured to detect a face in the live video stream, detect a shoulder contour, and determine keypoints of the shoulder and neck portions of the video stream of an upper body of the user according to the detected face and the detected shoulder contour; a virtual-cloth fitting unit comprising a processor, the virtual-cloth fitting unit configured to map an image of clothes to the video stream of the upper body of the user according to the keypoints; and a video display configured to display an augmented video stream of the upper body of the user with the image of the clothes overlaid over a portion of the video stream of the upper body of the user.

17 . The mobile wireless device of claim 16 , wherein the virtual-cloth fitting unit is further configured to track movement of the user and to remap the image of the clothes to the movement.

18 . The mobile wireless device of claim 17 , wherein the virtual-cloth fitting unit is further configured to transform relevant positions on a plane of the clothes with corresponding positions on a plane of the image of the user's upper body.

19 . The mobile wireless device of claim 17 , wherein the virtual-cloth fitting unit is further configured to execute a Delaunay triangulation.

20 . The mobile wireless device of claim 16 , wherein the display is configured to present a candidate face region and to receive a user input confirming that the candidate face region corresponds to the user's face.

Assignees

• Futurewei Technologies Inc

Inventors

• Xu Jingsong
• Cui Ying
• Wu Qiang
• Zhang Jian
• Zhang Chen-Xiong
• Liu Haibo
• Fang Kui

Classifications

• G06K9/00362

  Physics · mapped topic

• G06T11/60

  Creating or editing images; Combining images with text · CPC title

• G06T3/0093

  Physics · mapped topic

• G06T17/20

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title

• G06Q30/0643 Primary

  graphically representing goods, e.g. 3D product representation · CPC title