Generation of building instructions for construction element models

Patent metadata

Field	Value
Publication number	US-9821242-B2
Application number	US-201214119534-A
Country	US
Kind code	B2
Filing date	May 22, 2012
Priority date	May 23, 2011
Publication date	Nov 21, 2017
Grant date	Nov 21, 2017

Abstract

A toy construction system comprising a set of toy construction elements with coupling means for releasably interconnecting the toy construction elements; and a data processing system comprising image capturing means, processing means, and display means, wherein the data processing system is adapted to capture an image of a partial toy construction model constructed from a subset of the toy construction elements; process the captured image to detect at least a position and an orientation of the partial toy construction model; identify a user selection indicative of a user selection of at least one of a set of subsequent construction elements, each subsequent construction element being connectable to the partial toy construction model; responsive to the detected position and orientation of the partial toy construction model, display on said display means a composite image comprising the captured image having superimposed an image of at least the selected subsequent construction element.

First claim

The invention claimed is:

1. A computer-implemented method for generating building instructions for constructing a physical toy construction model from toy construction elements of a toy construction system, the toy construction elements comprising couplings for releasably interconnecting the toy construction elements; the method comprising:
capturing, by a digital camera, an image of a partial physical toy construction model constructed from a subset of the toy construction elements;
processing, by a digital processor, the captured image to detect at least a position and an orientation of the partial toy construction model;
determining a set of subsequent construction elements from a digital representation of the partial physical toy construction model and from a stored data structure indicative of a plurality of sequences of optional construction steps, each sequence resulting in one of a set of alternative construction models constructable from the set of toy construction elements;
displaying digital representations of the set of alternative construction elements in a user interface on a display;
identifying a user selection indicative of at least one of the set of alternative subsequent construction elements, each subsequent construction element being connectable to the partial physical toy construction model; and
responsive to the detected position and orientation of the partial physical toy construction model, displaying, by the display, one or more composite images comprising the captured images of the partial physical toy construction model having superimposed thereon the digital representations of alternate of at least the selected subsequent construction elements connected thereto.

2. A computer implemented method according to claim 1 , further comprising obtaining a digital representation of the partial toy construction model; and, responsive to the obtained digital representation, displaying respective images of the set of subsequent construction elements.

3. A computer implemented method according to claim 2 , wherein obtaining a digital representation of the partial toy construction model comprises generating the digital representation of the partial toy construction model from a digital representation of a previous partial toy construction model and from a previous user selection.

4. A computer implemented method according to claim 2 , wherein the digital representation includes respective position coordinates of each of the construction elements with respect to a predetermined coordinate system.

5. A computer implemented method according to claim 1 , further comprising generating a digital representation of an updated partial toy construction model from a digital representation of the partial toy construction model and from the identified user selection.

6. A computer implemented method according to claim 1 , wherein processing the captured image to detect at least a position and an orientation of the partial toy construction model comprises detecting at least one augmented reality marker and determining the position and orientation of the toy construction model from the detected at least one augmented reality marker.

7. A computer implemented method according to claim 1 , wherein processing the captured image comprises identifying at least one added construction element; and determining the user selection from said identified added construction element.

8. A computer implemented method according to claim 7 , wherein identifying the at least one added construction element comprises detecting an augmented reality marker included in the added construction element.

9. A computer implemented method according to claim 7 , further comprising identifying a position and orientation of the added construction element relative to the partial toy construction model.

10. A computer implemented method according to claim 1 , further comprising, responsive to the identified user selection and to the detected position and orientation of the partial toy construction model, displaying a composite image comprising the captured image having superimposed at least one additional computer-generated image.

11. A computer implemented method according to claim 1 , wherein the set of toy construction elements comprises one or more marker construction elements comprising such couplings and each having a visual appearance recognisable by an image processor.

12. A computer implemented method according to claim 11 , comprising at least two marker construction elements, where the two marker construction elements are releasably connectable with each other via their respective couplings, so as to form a composite marker element.

13. A computer implemented method according to claim 11 , wherein the toy construction elements have one or more attributes, said attributes including one or more of color, texture, or decorations.

14. A computer implemented method according to claim 13 , wherein the couplings are arranged in one or more regular planar grids defining the direction of connection; and wherein each marker construction element comprises an insignia visible, when the marker construction element is connected to at least one other toy construction element of the toy construction system, from a direction parallel to at least one of the planar grids or from a direction normal to at least one of the planar grids.

15. A computer implemented method according to claim 14 , wherein each of the marker construction elements has a top surface, a bottom surface, and at least one side surface; wherein said couplings are placed on at least one of the top and the bottom surface; wherein the marker construction element comprises at least a first surface without couplings, where the first surface comprises said insignia.

16. A computer implemented method according to claim 12 , further comprising detecting the composite marker element and identifying said user selection indicative of a user selection of at least one of said set of subsequent construction elements based on the detected composite marker element.

17. A computer implemented method according to claim 16 , wherein each marker construction element comprises an insignia indicative of a code symbol of a code for encoding information; and further comprising determining, from the visual appearance of the composite marker element, an encoded set of code symbols, and decoding the determined set of code symbols to obtain decoded information.

18. A computer implemented method according to claim 1 , wherein the couplings define a regular grid restricting a position and/or orientation of the toy constructing elements in a toy construction model to discrete positions and/or discrete orientations relative to said regular grid.

19. A computer implemented method according to claim 18 , wherein the toy construction system comprises at least two marker construction elements, and further comprising detecting respective positions and/or orientations of the two marker construction elements in a toy construction model relative to said regular grid, and identifying said user selection indicative of a user selection of at least one of said set of subsequent construction elements based on the detected respective positions and/or orientations.

20. A computer implemented method according to claim 19 , further comprising determining grid locations of a regular image grid from the detected locations of the marker construction elements, and generating an image feature of a computer-generated image at a location relative to the dete

Assignees

• Muthyala Siddharth
• Van Beek Harm Jan
• Guyon Nicolas David Alain
• Lasorne Frantz
• Jensen Mikkel Holm
• Lego As

Inventors

• Muthyala Siddharth
• Van Beek Harm Jan
• Guyon Nicolas David Alain
• Lasorne Frantz
• Jensen Mikkel Holm

Classifications

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T2219/2008

  Assembling, disassembling · CPC title

• A63H33/06

  to be assembled without the use of additional elements · CPC title

• A63F2009/246

  Computer generated or synthesized image · CPC title

• A63H33/042 Primary

  Mechanical, electrical, optical, pneumatic or hydraulic arrangements; Motors · CPC title