Shot-processing device

Patent metadata

Field	Value
Publication number	US-12062203-B2
Application number	US-202017614111-A
Country	US
Kind code	B2
Filing date	May 22, 2020
Priority date	May 24, 2019
Publication date	Aug 13, 2024
Grant date	Aug 13, 2024

Abstract

The invention relates to a shot-processing device. The memory receives scene data comprising three-dimensional object pairs associating an object identifier, the ellipsoid data, which define an ellipsoid and its orientation and a position of center, and shot data defining a two-dimensional image of the scene, from a viewpoint corresponding to a desired pose. The detector receives shot data and returns two-dimensional object pairs comprising an object identifier and its associated shot region. The preparer determines a set of positioning elements, each associating the object identifier, the ellipsoid data, and ellipse data which define an ellipse approximating the shot region. The combiner generates candidates associating positioning elements and a shot orientation. The estimator calculates a pose comprising a position and an orientation. The selector projects the ellipsoid data onto the shot data from the pose, determines a measurement of similarity and selects the pose that has the highest likelihood value.

First claim

The invention claimed is:

1. Shot-processing device, comprising:
a memory arranged to receive, on the one hand, scene data that comprise three-dimensional object pairs each associating an object identifier, and ellipsoid data which define an ellipsoid and its orientation and a position of its center in a common frame of reference and, on the other hand, shot data defining a two-dimensional image of the scene associated with the scene data, from a viewpoint corresponding to a desired pose,
a detector arranged to receive shot data and to return one or more two-dimensional object pairs each comprising an object identifier present in the scene data, and a shot region associated with this object identifier,
a preparer arranged to determine, for at least some of the two-dimensional object pairs from the detector, a set of positioning elements, the number whereof is less than or equal to the number of three-dimensional object pairs in the scene data that comprise the object identifier of the two-dimensional object pair in question, each positioning element associating the object identifier and the ellipsoid data of a three-dimensional object pair comprising this object identifier, and ellipse data which define an ellipse approximating the shot region of the two-dimensional object pair in question and its orientation as well as a position of its center in the two-dimensional image,
a combiner arranged to generate a list of candidates each associating one or more positioning elements and a shot orientation, and/or the combination of at least two positioning elements, the positioning elements of a candidate being taken from separate two-dimensional object pairs and not relating to the same three-dimensional object pair,
an estimator arranged to calculate, for at least one of the candidates, a pose comprising a position and an orientation in the common frame of reference from the ellipse data and the ellipsoid data of the positioning elements, or from the ellipse data and the ellipsoid data of the one or more positioning elements and the shot orientation,
a selector arranged, for at least some of the poses, to project all of the ellipsoid data of the scene data onto the shot data from the pose, to determine a measurement of similarity between each projection of the ellipsoid data and each ellipse defined by a two-dimensional object pair coming from the detector, and to calculate a likelihood value by using, for each projection of the ellipsoid data, the largest measurement of similarity determined, and to select the pose that has the highest likelihood value.

2. Shot-processing device according to claim 1 , wherein the selector is arranged to carry out a measurement of likelihood based, for a given pose, on the number of measurements of similarity that exceed a given threshold.

3. Shot-processing device according to claim 1 , wherein the preparer determines the ellipse data of each positioning element using a method of least squares.

4. Shot-processing device according to claim 1 , wherein
the detector determines a rectangular box as a shot region of each two-dimensional pair to be returned, and
the preparer determines the ellipse data of each positioning element based on the ellipse inscribed in the rectangular box forming the shot region.

5. Shot-processing device according to claim 1 , wherein the combiner is arranged to determine the shot orientation as a function of the detection of vanishing points within the two-dimensional image of the scene associated with the scene data.

6. Shot-processing device according to claim 1 , wherein the combiner is arranged to determine the shot orientation by means of a sensor.

7. Shot-processing device according to claim 1 , further comprising a shot device arranged to capture a shot of the scene from a given viewpoint in order to derive the shot data therefrom.

8. Shot-processing method based on:
scene data that comprise three-dimensional object pairs each associating an object identifier, and ellipsoid data which define an ellipsoid and its orientation and a position of its center in a common frame of reference, and
shot data defining a two-dimensional image of the scene associated with the scene data, from a viewpoint corresponding to a desired pose,
the method comprising the following operations:
a) detecting, from shot data, one or more two-dimensional object pairs each comprising an object identifier present in the scene data, and a shot region associated with this object identifier,
b) determining, for at least one of the two-dimensional object pairs detected, a set of positioning elements, the number whereof is less than or equal to the number of three-dimensional object pairs in the scene data that comprise the object identifier of the two-dimensional object pair in question, each positioning element associating the object identifier and the ellipsoid data of a three-dimensional object pair comprising this object identifier, and ellipse data which define an ellipse approximating the shot region of the two-dimensional object pair in question and its orientation as well as a position of its center in the two-dimensional image,
c) generating a list of candidates each associating one or more positioning elements and a shot orientation, and/or the combination of at least two positioning elements, the positioning elements of a candidate being taken from two separate two-dimensional object pairs and not relating to the same three-dimensional object pair,
d) calculating, for at least one of the candidates, a pose comprising a position and an orientation in the common frame of reference from the ellipse data and the ellipsoid data of the positioning elements, or from the ellipse data and the ellipsoid data of the one or more positioning elements and the shot orientation,
e) projecting, for at least some of the poses, all of the ellipsoid data of the scene data onto the shot data from the pose, to determine a measurement of similarity between each projection of the ellipsoid data and each ellipse defined by a two-dimensional object pair detected, and to calculate a likelihood value by using, for each projection of the ellipsoid data, the largest measurement of similarity determined, and
f) selecting the pose that has the highest likelihood value.

Assignees

• Institut National De Rech En Informatique Et En Automatique
• Univ De Lorraine

Inventors

• Berger Marie-Odile
• Gaudilliere Vincent
• Simon Gilles

Classifications

• G06T2207/20068

  Projection on vertical or horizontal image axis · CPC title

• G06T7/60

  Analysis of geometric attributes · CPC title

• G06V20/647

  by matching two-dimensional images to three-dimensional objects · CPC title

• G06V10/225

  based on a marking or identifier characterising the area · CPC title

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title