Systems and methods for object location

Patent metadata

Field	Value
Publication number	US-2018192035-A1
Application number	US-201715621716-A
Country	US
Kind code	A1
Filing date	Jun 13, 2017
Priority date	Jan 4, 2017
Publication date	Jul 5, 2018
Grant date	—

Abstract

A method performed by an apparatus is described. The method includes receiving a set of image frames including at least one object. The method also includes receiving a camera position for each image frame. The method further includes associating the at least one object between image frames based on one or more object points and the received camera position for each image frame to produce two-dimensional (2D) object location data. The method additionally includes estimating three-dimensional (3D) pose data of the at least one object based on the 2D object location data. The method also includes refining the 3D pose data based on a shape constraint.

First claim

What is claimed is:

1 . A method performed by an apparatus, comprising:
receiving a set of image frames including at least one object; receiving a camera position for each image frame; associating the at least one object between image frames based on one or more object points and the received camera position for each image frame to produce two-dimensional (2D) object location data; estimating three-dimensional (3D) pose data of the at least one object based on the 2D object location data; and refining the 3D pose data based on a shape constraint.

2 . The method of claim 1 , further comprising interpolating camera pose variables for one or more points of the at least one object.

3 . The method of claim 2 , wherein interpolating camera pose variables is based on a pixel location of the at least one object in the set of image frames.

4 . The method of claim 2 , wherein interpolating camera pose variables is based on a timing of a pixel capture of the at least one object in the set of image frames.

5 . The method of claim 2 , wherein the interpolation is performed for at least a road sign or a lane marker.

6 . The method of claim 2 , wherein at least one of said associating the at least one object, said estimating the 3D pose data, or said refining the 3D pose data is based on interpolated camera pose variables.

7 . The method of claim 1 , wherein refining the 3D pose data comprises:
reducing a first reprojection error for an individual sign corner; reparametrizing 3D sign pose data; and reducing a second reprojection error for reparametrized 3D sign pose data.

8 . The method of claim 1 , wherein refining the 3D pose data comprises reducing a reprojection error for spline parameters.

9 . The method of claim 1 , wherein estimating the 3D pose data comprises estimating a road normal vector.

10 . The method of claim 9 , wherein estimating the road normal vector is performed online based on an offline estimated road normal vector.

11 . The method of claim 1 , further comprising uploading refined 3D pose data to a mapping database.

12 . An apparatus, comprising:
a memory; a processor coupled to the memory, wherein the processor is configured to: receive a set of image frames including at least one object; receive a camera position for each image frame; associate the at least one object between image frames based on one or more object points and the received camera position for each image frame to produce two-dimensional (2D) object location data; estimate three-dimensional (3D) pose data of the at least one object based on the 2D object location data; and refine the 3D pose data based on a shape constraint.

13 . The apparatus of claim 12 , wherein the processor is configured to interpolate camera pose variables for one or more points of the at least one object.

14 . The apparatus of claim 13 , wherein the processor is configured to interpolate the camera pose variables based on a pixel location of the at least one object in the set of image frames.

15 . The apparatus of claim 13 , wherein the processor is configured to interpolate the camera pose variables based on a timing of a pixel capture of the at least one object in the set of image frames.

16 . The apparatus of claim 13 , wherein the processor is configured to perform the interpolation for at least a road sign or a lane marker.

17 . The apparatus of claim 13 , wherein the processor is configured to perform at least one of said associating the at least one object, said estimating the 3D pose data, or said refining the 3D pose data based on interpolated camera pose variables.

18 . The apparatus of claim 12 , wherein the processor is configured to refine the 3D pose data by:
reducing a first reprojection error for an individual sign corner; reparametrizing 3D sign pose data; and reducing a second reprojection error for reparametrized 3D sign pose data.

19 . The apparatus of claim 12 , wherein the processor is configured to refine the 3D pose data by reducing a reprojection error for spline parameters.

20 . The apparatus of claim 12 , wherein the processor is configured to estimate the 3D pose data by estimating a road normal vector.

21 . The apparatus of claim 20 , wherein the processor is configured to estimate the road normal vector online based on an offline estimated road normal vector.

22 . The apparatus of claim 12 , wherein the processor is configured to upload refined 3D pose data to a mapping database.

23 . A non-transitory tangible computer-readable medium storing computer executable code, comprising:
code for causing an electronic device to receive a set of image frames including at least one object; code for causing the electronic device to receive a camera position for each image frame; code for causing the electronic device to associate the at least one object between image frames based on one or more object points and the received camera position for each image frame to produce two-dimensional (2D) object location data; code for causing the electronic device to estimate three-dimensional (3D) pose data of the at least one object based on the 2D object location data; and code for causing the electronic device to refine the 3D pose data based on a shape constraint.

24 . The computer-readable medium of claim 23 , further comprising code for causing the electronic device to interpolate camera pose variables for one or more points of the at least one object.

25 . The computer-readable medium of claim 23 , wherein the code for causing the electronic device to refine the 3D pose data comprises:
code for causing the electronic device to reduce a first reprojection error for an individual sign corner; code for causing the electronic device to reparametrize 3D sign pose data; and code for causing the electronic device to reduce a second reprojection error for reparametrized 3D sign pose data.

26 . The computer-readable medium of claim 23 , wherein the code for causing the electronic device to refine the 3D pose data comprises code for causing the electronic device to reduce a reprojection error for spline parameters.

27 . An apparatus, comprising:
means for receiving a set of image frames including at least one object; means for receiving a camera position for each image frame; means for associating the at least one object between image frames based on one or more object points and the received camera position for each image frame to produce two-dimensional (2D) object location data; means for estimating three-dimensional (3D) pose data of the at least one object based on the 2D object location data; and means for refining the 3D pose data based on a shape constraint.

28 . The apparatus of claim 27 , further comprising means for interpolating camera pose variables for one or more points of the at least one object.

29 . The apparatus of claim 27 , wherein the means for refining the 3D pose data comprises:
means for reducing a first reprojection error for an individual sign corner; means for reparametrizing 3D sign pose data; and means for reducing a second reprojection error for reparametrized 3D sign pose data.

30 . The apparatus of claim 27 , wherein the means for refining the 3D pose data comprises means for reducing a reprojection error for spline parame

Assignees

• Qualcomm Inc

Inventors

• Dabeer Onkar Jayant
• Reitmayr Gerhard
• Sukhavasi Ravi Teja
• Somasundaram Kiran
• Gowaikar Radhika Dilip

Classifications

• H04N23/689

  Motion occurring during a rolling shutter mode · CPC title

• G06T2207/20201

  Motion blur correction · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• H04N7/181

  for receiving images from a plurality of remote sources · CPC title

• G06T7/73

  using feature-based methods · CPC title