Systems and methods for computer assisted operation

Patent metadata

Field	Value
Publication number	US-10282914-B1
Application number	US-201715795252-A
Country	US
Kind code	B1
Filing date	Oct 26, 2017
Priority date	Jul 17, 2015
Publication date	May 7, 2019
Grant date	May 7, 2019

Abstract

Systems and methods are disclosed method for rendering reality for an object by performing motion tracking and area learning of an environment; selecting a pattern or color from a plurality of product variations; blending the pattern or color of the object and the environment; and displaying the color on the object as an augmented reality view.

First claim

What is claimed is:

1. A method for rendering reality, comprising:
performing tracking and area learning of an environment by motion-tracking visual features of the environment using concurrent odometry and mapping (COM) including feature points and planes to estimate a pose of the camera relative to the environment and understanding an environment with points or planes using accelerometer sensor and estimating light or color in the environment;
capturing images from a plurality of angles of the environment;
acquiring sensor data from sensors and optimizing features extracted from each image and sensor data, where a feature conveys data unique to the image at a specific pixel location;
selecting a pattern or color from a plurality of product variations;
blending the pattern or color of the object and the environment;
displaying the color on the object as an augmented reality view;
scaling the 3D model of the product based on dimensions of the environment and the product;
projecting the product in the environment;
selecting a best fit from jewelry variations, shoe variations, clothing variations, apparel variations or footwear variations; and
determining content to be presented based on previous activities or selections while generating an augmented or virtual reality display of the new product in the environment.

2. The method of claim 1 , comprising:
performing motion tracking and understanding an environment by detecting points or planes from images and estimating light or color in the environment using the video camera and an accelerometer;
determining a target volume or space;
projecting the product in the environment; and
iteratively changing product variations until satisfied.

3. The method of claim 1 , comprising applying a 3D deformable template to the object and warping the template to one or more predetermined points on the model.

4. The method of claim 1 , wherein the object comprises hair, foot, hand, finger, toe, wrist, neck, or ear, wherein the object comprises jewelry.

5. The method of claim 1 , comprising providing a 3D printer for printing a custom object, or providing a bed of sand having a liquid flowable state and a solid state with an impression for a reformably molded object.

6. The method of claim 1 , wherein the object comprises a body portion, a head, a chest region, a leg, and the object comprises body protection gear.

7. The method of claim 1 , wherein the object comprises a face and the product variation comprises facial makeup products.

8. The method of claim 1 , wherein the object comprises a head and the product variation comprises hairstyles.

9. The method of claim 1 , wherein the anatomical portion is a face, comprising:
selecting a makeup pattern or color from a plurality of makeup product variations; blending the makeup pattern or color onto the model; and
displaying the makeup color on the face.

10. The method of claim 1 , comprising modeling one or more applications of light foundation creme, eye disguise, highlighter, concealer or blush and recommending a makeup to the subject.

11. The method of claim 4 , wherein the jewelry comprises cosmetic jewelry, a facial jewelry, hand jewelry, foot jewelry, ear jewelry, dental jewelry, navel jewelry or a hair jewelry.

12. The method of claim 1 , comprising motion tracking, area learning and depth sensing a body portion.

13. The method of claim 1 , comprising matching used jewelry or consigned jewelry to the subject.

14. The method of claim 1 , comprising rendering the product variations on an online portal.

15. The method of claim 1 , comprising providing augmented reality view or virtual reality view of the one or more products with a user custom view.

16. A method for selecting a product, comprising:
selecting one or more 3D models from a product database;
motion-tracking visual features of the environment using concurrent odometry and mapping (COM) including feature points and planes to estimate a pose of the camera relative to the environment and understanding an environment with points or planes using accelerometer sensor and estimating light or color in the environment;
capturing images from a plurality of angles of the environment;
acquiring sensor data from sensors and optimizing features extracted from each image and sensor data, where a feature conveys data unique to the image at a specific pixel location;
selecting a pattern or color from a plurality of product variations;
blending the pattern or color of the product and the environment; and
displaying the color on the product as an augmented reality view;
scaling the 3D model of the product based on dimensions of the environment and the product;
determining content to be presented based on previous activities or selections while projecting the product in the environment;
selecting a best fit from jewelry variation, shoe variations, clothing variations, apparel variations or footwear variations; and
generating an augmented or virtual reality display of the new product in the environment.

17. A method for fitting, comprising:
selecting one or more 3D models from a product database;
capturing images of a user view relative to a reference frame using a mobile camera, including:
motion-tracking visual features of the environment using concurrent odometry and mapping (COM) including feature points and planes to estimate a pose of the camera relative to the environment and understanding an environment with points or planes using accelerometer sensor and estimating light or color in the environment; capturing images from a plurality of angles of the environment; acquiring sensor data from sensors and optimizing features extracted from each image and sensor data, where a feature conveys data unique to the image at a specific pixel location; selecting a pattern or color from a plurality of product variations; blending the pattern or color of the object and the environment; and displaying the color on the object as an augmented reality view;

generating an updated model of the user view by selecting a closest 3D model to the user view and matching points on the closest 3D product model to one or more predetermined points on the user view;
scaling the 3D model of the product based on dimensions of the environment and the product;
determining content to be presented based on previous activities or selections while projecting the product in the environment;
selecting a best fit from jewelry variations, shoe variations, clothing variations, apparel variations or footwear variations.

18. The method of claim 17 , comprising providing augmented reality view or virtual reality view of the one or more products with a user custom view.

Assignees

• Tran Bao
• Tran Ha

Inventors

• Tran Bao
• Tran Ha

Classifications

• G06T11/10

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• A61B5/0205

  Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition · CPC title

• G06T17/00 Primary

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T2219/2012

  Colour editing, changing, or manipulating; Use of colour codes · CPC title