Image recognition based on augmented reality

Patent metadata

Field	Value
Publication number	US-10762382-B2
Application number	US-202016841467-A
Country	US
Kind code	B2
Filing date	Apr 6, 2020
Priority date	Jan 11, 2017
Publication date	Sep 1, 2020
Grant date	Sep 1, 2020

Abstract

A method includes: receiving, at a server and from an augmented reality device, user information of a user using the augmented reality device; generating, by the server, a picture feature sample sub-library corresponding to the user information, including screening out, from a preset picture feature sample library, picture feature samples associated with the user information; and providing, to the augmented reality device, the picture feature sample sub-library, wherein the picture feature sample sub-library can be configured to be used by the augmented reality device to perform, during image scanning of an offline environment of the user, image recognition of images scanned from the offline environment by comparing the scanned images to picture features in the picture feature sample sub-library.

First claim

What is claimed is:

1. A method comprising:
submitting, by an augmented reality device and to a server, user information of a user using the augmented reality device;
receiving, at the augmented reality device and from the server, a picture feature sample sub-library corresponding to the user information, wherein the picture feature sample sub-library is generated by the server based on screening out picture feature samples associated with the user information from a preset picture feature sample library; and
performing, by the augmented reality device and during image scanning of a real-world environment of the user, image recognition of images scanned from the real-world environment by comparing the scanned images to picture features in the picture feature sample sub-library.

2. The method of claim 1 , wherein the user information includes a location of the user, and wherein the method further comprises:
collecting, from a connected third-party server, first picture samples and first locations associated with the first picture samples;
collecting, from online service data and using a preset data collection algorithm, second picture samples and second locations associated with the second picture samples;
extracting picture feature samples from the first and second picture samples; and
updating the preset picture feature sample library using the extracted picture feature samples.

3. The method of claim 2 , further comprising:
acquiring, from an operator, presentation data and presentation effects configured for the picture feature samples in the preset picture feature sample library;
storing the acquired presentation data and presentation effects in association with the picture feature samples in the preset picture feature sample library; and
dynamically updating the presentation data and the presentation effects in response to an update operation of the operator.

4. The method of claim 3 , further comprising:
providing, when an offline target corresponding to any picture feature sample in the picture feature sample sub-library can be recognized from the image information, presentation data associated with the picture feature sample and a presentation effect associated with the presentation data to the augmented reality device, so that the augmented reality device displays, on the basis of the presentation effect, the presentation data in an augmented manner on a location corresponding to the offline target in an augmented reality scene.

5. The method of claim 1 , further comprising:
receiving, from the augmented reality device, the scanned image information; and
performing, on the server, the image recognition on the image information on the basis of the generated picture feature sample sub-library.

6. The method of claim 2 , wherein screening out picture feature samples associated with the user information from the preset picture feature sample library comprises:
calculating distances between the location of the user and locations associated with the picture feature samples; and
screening out, from the preset picture feature sample library, picture feature samples which have the distances less than a preset threshold.

7. The method of claim 6 , wherein the user information further comprises behavior logs of the user, and wherein the method further comprises:
performing data analysis on the behavior logs of the user to determine a behavioral preference of the user; and
further screening out picture feature samples satisfying the behavioral preference of the user from the picture feature samples which have the distances less than the preset threshold.

8. A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations comprising:
submitting, by an augmented reality device and to a server, user information of a user using the augmented reality device;
receiving, at the augmented reality device and from the server, a picture feature sample sub-library corresponding to the user information, wherein the picture feature sample sub-library is generated by the server based on screening out picture feature samples associated with the user information from a preset picture feature sample library; and
performing, by the augmented reality device and during image scanning of a real-world environment of the user, image recognition of images scanned from the real-world environment by comparing the scanned images to picture features in the picture feature sample sub-library.

9. The non-transitory, computer-readable medium of claim 8 , wherein the user information includes a location of the user, and wherein the method further comprises:
collecting, from a connected third-party server, first picture samples and first locations associated with the first picture samples;
collecting, from online service data and using a preset data collection algorithm, second picture samples and second locations associated with the second picture samples;
extracting picture feature samples from the first and second picture samples; and
updating the preset picture feature sample library using the extracted picture feature samples.

10. The non-transitory, computer-readable medium of claim 9 , the operations further comprising:
acquiring, from an operator, presentation data and presentation effects configured for the picture feature samples in the preset picture feature sample library;
storing the acquired presentation data and presentation effects in association with the picture feature samples in the preset picture feature sample library; and
dynamically updating the presentation data and the presentation effects in response to an update operation of the operator.

11. The non-transitory, computer-readable medium of claim 10 , the operations further comprising:
providing, when an offline target corresponding to any picture feature sample in the picture feature sample sub-library can be recognized from the image information, presentation data associated with the picture feature sample and a presentation effect associated with the presentation data to the augmented reality device, so that the augmented reality device displays, on the basis of the presentation effect, the presentation data in an augmented manner on a location corresponding to the offline target in an augmented reality scene.

12. The non-transitory, computer-readable medium of claim 8 , the operations further comprising:
receiving, from the augmented reality device, the scanned image information; and
performing, on the server, the image recognition on the image information on the basis of the generated picture feature sample sub-library.

13. The non-transitory, computer-readable medium of claim 9 , wherein screening out picture feature samples associated with the user information from the preset picture feature sample library comprises:
calculating distances between the location of the user and locations associated with the picture feature samples; and
screening out, from the preset picture feature sample library, picture feature samples which have the distances less than a preset threshold.

14. The non-transitory, computer-readable medium of claim 13 , wherein the user information further comprises behavior logs of the user, and wherein the method further comprises:
performing data analysis on the behavior logs of the user to determine a behavioral preference of the user; and
further screening out picture feature samples satisfying the behavioral preference of the user from the picture feature samples which have the distances less than the preset threshold.

15. A computer-impl

Assignees

• Alibaba Group Holding Ltd

Inventors

• Wang Linqing
• Zeng Xiaodong
• Zhang Hong
• Lin Feng

Classifications

• G06V10/751

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title

• G06F16/5838 Primary

  using colour · CPC title

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• H04L67/131

  Protocols for games, networked simulations or virtual reality · CPC title

• H04L67/53

  using third party service providers · CPC title