System and method for toy recognition

Patent metadata

Field	Value
Publication number	US-12070693-B2
Application number	US-202318297140-A
Country	US
Kind code	B2
Filing date	Apr 7, 2023
Priority date	Nov 10, 2014
Publication date	Aug 27, 2024
Grant date	Aug 27, 2024

Abstract

A recognition system for recognizing real-world toy objects from one or more images having an image capturing device and one or more processors. The processor implements a detection module, a recognition module, and a user experience module, and is configured to capture an image of a real-world scene, and detect one or more regions of interest in the image. The recognition system is configured to generate at least one part-image from the captured image, each part image including at least one of the one or more detected regions of interest, and feed the generated part-image to the recognition module. The recognition system is configured to recognize a real-world toy object in the part-image, the real-world toy object comprising at least one toy construction element, and provide a digital representation of the recognized real-world toy object.

First claim

What is claimed is:

1. A recognition system for recognizing real-world toy objects from one or more images, wherein the recognition system comprises an image capturing device and one or more processors, the processor implementing a detection module, a recognition module and a user experience module, and wherein the recognition system is configured to:
capture, by the image capturing device, at least one image of a real-world scene;
detect, by the detection module, one or more regions of interest in the at least one captured image;
generate, by the detection module, at least one part-image from the at least one captured image, each part image including at least one of the one or more detected regions of interest;
feed, by the detection module, the generated at least one part-image to the recognition module;
recognize, by the recognition module, a real-world toy object in the at least one part-image, the real-world toy object comprising at least one toy construction element; and
provide, by the user experience module, a digital representation of the recognized real-world toy object.

2. The recognition system according to claim 1 , further configured to discard one or more of the part-images based on a confidence parameter output by the recognition system or based on an evaluation whether the recognized real-world toy object is consistent with prior knowledge about other toy objects in the scene and about physical constraints of the placement and/or interconnection of toy objects relative to each other.

3. The recognition system according to claim 1 , wherein the recognition system is configured to:
capture, by the image capturing device, a sequence of images of a real-world scene;
detect, by the detection module, a change in the scene based on the captured sequence of images;
generate, by the detection module, a part-image from a captured image of the sequence of captured images, the part image including the detected change;
feed, by the detection module, the generated part-image to the recognition module;
recognize, by the recognition module, a real-world toy object in the generated part-image, the real-world toy object comprising at least one toy construction element; and
provide, by the user experience module, a digital representation of the recognized real-world toy object.

4. The recognition system according to claim 3 , wherein the real-world toy object includes at least one toy construction element added by the user to the scene and/or at least one toy construction element repositioned relative to a toy construction model.

5. The recognition system according to claim 3 , configured to detect toy objects that have been added to the scene, wherein the recognition module is configured to:
provide a list of candidate toy objects each having a likelihood of being a newly added toy object;
determine whether placement of a candidate toy object of said list at the detected position within the scene is compatible with one or more constraints imposed by the toy construction system; and
determine a most likely one of the candidate toy objects.

6. The recognition system according to claim 3 , wherein the recognition module is configured to:
create a list of candidate object identifiers and associated likelihood scores indicative of a likelihood of the respective candidate object identifiers matching the real-world toy object;
for each of the candidate object identifiers and based on the captured image, estimate a placement of the corresponding virtual object in a virtual scene corresponding to the real-world scene;
compute a respective correspondence score for each candidate object identifier based on a correlation of the estimated placement with at least the captured image; and
select a candidate object identifier and a corresponding placement based at least on the computed correspondence scores.

7. The recognition system according to claim 1 , comprising a database of real-world toy objects available for inclusion as virtual objects in a virtual environment, each virtual object being associated with an object identifier of a corresponding real-world toy object and with accompanying information about one or more virtual properties of the virtual object;
wherein the recognition module is configured to predict a matching object identifier, matching the recognized real-world toy object; and wherein the user experience module is configured to add one or more virtual objects associated with the matched object identifier to a virtual environment, and to affect a gameplay experience based on the accompanying information.

8. The recognition system according to claim 1 , wherein the recognition system is further configured to estimate one or more additional attributes of the real-world object depicted in the captured image, in addition to an identification of the real-world object.

9. The recognition system according to claim 8 , wherein the one or more additional attribute comprises a relative orientation of the real-world toy object relative to the image capturing device.

10. A recognition system for recognizing real-world toy objects from one or more images, wherein the recognition system comprises an image capturing device and one or more processors, the processor implementing a recognition module and a user experience module, and wherein the recognition system is configured to:
capture, by the image capturing device, an image of a real-world scene;
recognize, by the recognition module, two or more real-world toy objects in the captured image;
create two or more virtual characters corresponding the respective recognized real-world toy objects; and
control interaction of the created two or more virtual characters with each other, where one or more aspects or results of the interaction are at least in part controlled by virtual attributes of the recognized toy objects.

11. The recognition system according to claim 10 , wherein the interaction involves two virtual characters engaging in a battle where capabilities or other attributes of the virtual characters are at least in part determined by features of the recognized toy objects.

12. The recognition system according to claim 11 , wherein the capabilities or other attributes of the virtual characters are at least in part determined by detected and recognized accessories connected to the recognized toy objects.

13. The recognition system according to claim 10 , comprising a database of real-world toy objects available for inclusion as virtual objects in a virtual environment, each virtual object being associated with an object identifier of a corresponding real-world toy object and with accompanying information about one or more virtual properties of the virtual object;
wherein the recognition module is configured to predict a matching object identifier, matching the recognized real-world toy object; and wherein the user experience module is configured to add one or more virtual objects associated with the matched object identifier to a virtual environment, and to affect a gameplay experience based on the accompanying information.

14. The recognition system according to claim 10 , wherein the recognition system is further configured to estimate one or more additional attributes of the real-world object depicted in the captured image, in addition to an identification of the real-world object.

15. The recognition system according to claim 14 , wherein the one or more additional attribute comprises a relative orientation of the real-world toy object relative to the image capturing device.

16. A toy construction system, comprising a plura

Assignees

• Lego As

Inventors

• Velic Marko
• Noe Karsten Østergaard
• Mosegaard Jesper
• Christensen Brian Bunch
• Rimestad Jens

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N5/04

  Inference or reasoning models · CPC title

• A63H33/08

  provided with complementary holes, grooves, or protuberances, e.g. dovetails · CPC title

• A63F13/213

  comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence) · CPC title