Multi-angle object recognition

Patent metadata

Field	Value
Publication number	US-11961271-B2
Application number	US-202318182737-A
Country	US
Kind code	B2
Filing date	Mar 13, 2023
Priority date	Aug 8, 2018
Publication date	Apr 16, 2024
Grant date	Apr 16, 2024

Abstract

Methods, systems, and apparatus for controlling smart devices are described. In one aspect a method includes capturing, by a camera on a user device, a plurality of successive images for display in an application environment of an application executing on the user device, performing an object recognition process on the images, the object recognition process including determining that a plurality of images, each depicting a particular object, are required to perform object recognition on the particular object, and in response to the determination, generating a user interface element that indicates a camera operation to be performed, the camera option capturing two or more images, determining that a user, in response to the user interface element, has caused the indicated camera operation to be performed to capture the two or more images, and in response, determining whether a particular object is positively identified from the plurality of images.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
capturing, by a camera, a first plurality of images at a first zoom level;
performing a first object recognition process on the first plurality of images to detect a first object depicted in the first plurality of images;
in response to the first object recognition process failing to detect the first object in the first plurality of images, providing, for output at a computing device, instructions requesting an image capture operation to be performed, wherein the image capture operation includes capturing, by the camera, a second plurality of images at a second zoom level, different from the first zoom level, the second plurality of images being greater than the first plurality of images; and
in response to the image capture operation being performed, performing a second object recognition process on the second plurality of images to detect the first object depicted in the second plurality of images.

2. The computer-implemented method of claim 1 , wherein:
the first plurality of images are captured at a first frame rate; and
the second plurality of images are captured at a second frame rate greater than the first frame rate.

3. The computer-implemented method of claim 1 , wherein the first object comprises a machine-readable code.

4. The computer-implemented method of claim 1 , further comprising:
in response to the first object recognition process failing to detect the first object in the first plurality of images, providing, for display at the computing device, a first user interface element descriptive of one or more actions to perform.

5. The computer-implemented method of claim 4 , further comprising:
in response to the second object recognition process successfully detecting the first object in the second plurality of images, providing, for display at the computing device, a second user interface element indicative of a successful recognition of the first object.

6. The computer-implemented method of claim 4 , wherein the first user interface element comprises one or more of:
an icon;
an animation; or
a descriptive text.

7. The computer-implemented method of claim 4 , wherein:
the first user interface element is displayed adjacent to the first object being recognized; or
the first user interface element is displayed at a fixed location.

8. A user computing device, the user computing device comprising:
a camera;
one or more processors; and
one or more non-transitory computer-readable media configured to store instructions that, when executed by the one or more processors, cause the user computing device to perform operations, the operations comprising:
capturing, by the camera, a first plurality of images at a first zoom level,
performing a first object recognition process on the first plurality of images to detect a first object depicted in the first plurality of images,
in response to the first object recognition process failing to detect the first object in the first plurality of images, providing, for output at the user computing device, instructions requesting an image capture operation to be performed, wherein the image capture operation includes capturing, by the camera, a second plurality of images at a second zoom level, different from the first zoom level, the second plurality of images being greater than the first plurality of images, and
in response to the image capture operation being performed, performing a second object recognition process on the second plurality of images to detect the first object depicted in the second plurality of images.

9. The user computing device of claim 8 , wherein:
the first plurality of images are captured at a first frame rate; and
the second plurality of images are captured at a second frame rate greater than the first frame rate.

10. The user computing device of claim 8 , wherein the first object comprises a machine-readable code.

11. The user computing device of claim 8 , further comprising a display device,
wherein the operations further comprise:
in response to the first object recognition process failing to detect the first object in the first plurality of images, providing, for presentation on the display device, a first user interface element descriptive of one or more actions to perform.

12. The user computing device of claim 11 , wherein the operations further comprise:
in response to the second object recognition process successfully detecting the first object in the second plurality of images, providing, for presentation on the display device, a second user interface element indicative of a successful recognition of the first object.

13. The user computing device of claim 11 , wherein the first user interface element comprises one or more of:
an icon;
an animation; or
a descriptive text.

14. The user computing device of claim 11 , wherein:
the first user interface element is displayed adjacent to the first object being recognized; or
the first user interface element is displayed at a fixed location.

15. One or more non-transitory computer-readable media configured to store instructions that, when executed by one or more processors, cause the one or more processors to perform operations, the operations comprising:
capturing, by a camera, a first plurality of images at a first zoom level;
performing a first object recognition process on the first plurality of images to detect a first object depicted in the first plurality of images;
in response to the first object recognition process failing to detect the first object in the first plurality of images, providing, for output at a computing device, instructions requesting an image capture operation to be performed, wherein the image capture operation includes capturing, by the camera, a second plurality of images at a second zoom level, different from the first zoom level, the second plurality of images being greater than the first plurality of images; and
in response to the image capture operation being performed, performing a second object recognition process on the second plurality of images to detect the first object depicted in the second plurality of images.

16. The one or more non-transitory computer-readable media of claim 15 , wherein:
the first plurality of images are captured at a first frame rate; and
the second plurality of images are captured at a second frame rate greater than the first frame rate.

17. The one or more non-transitory computer-readable media of claim 15 , wherein the operations further comprise:
in response to the first object recognition process failing to detect the first object in the first plurality of images, providing, for display at the computing device, a first user interface element descriptive of one or more actions to perform, and
in response to the second object recognition process successfully detecting the first object in the second plurality of images, providing, for display at the computing device, a second user interface element indicative of a successful recognition of the first object.

Assignees

• Google Llc

Inventors

• Badr Ibrahim

Classifications

• G06V10/235 Primary

  based on user input or interaction · CPC title

• G06V10/16

  using multiple overlapping images; Image stitching · CPC title

• G06V10/24

  Aligning, centring, orientation detection or correction of the image · CPC title

• G06V10/776

  Validation; Performance evaluation · CPC title

• G06V10/98

  Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns · CPC title