Method and system for detecting and tracking objects using characteristic points

Patent metadata

Field	Value
Publication number	US-2021192764-A1
Application number	US-202117174311-A
Country	US
Kind code	A1
Filing date	Feb 11, 2021
Priority date	Dec 1, 2016
Publication date	Jun 24, 2021
Grant date	—

Abstract

A method includes identifying an object in a disparity map to be tracked by a movable platform, determining positions of a plurality of elements representing the object in a first image frame captured by an imaging device carried by the movable platform, selecting one or more characteristic points of the plurality of elements representing the object as tracking points of the object in the first image frame, and updating, according to an updated disparity map and a current location of the movable platform, locations of the tracking points of the object in a second image frame captured by the imaging device.

First claim

What is claimed is:

1 . A method, comprising:
identifying an object in a disparity map to be tracked by a movable platform; determining positions of a plurality of elements representing the object in a first image frame captured by an imaging device carried by the movable platform; selecting one or more characteristic points of the plurality of elements representing the object as tracking points of the object in the first image frame; and updating, according to an updated disparity map and a current location of the movable platform, locations of the tracking points of the object in a second image frame captured by the imaging device.

2 . The method according to claim 1 , further comprising:
obtaining the disparity map based on a stereographic image captured by a stereoscopic camera carried by the movable platform.

3 . The method according to claim 1 , wherein identifying the object in the disparity map includes:
determining a continuous region having one or more elements with disparity values within a predefined range to be the object, wherein the disparity values of the one or more elements are higher than disparity values of other elements in the continuous region.

4 . The method according to claim 1 , wherein the plurality of characteristic points are selected to be closer to a navigation path of the movable platform than a rest part of the object.

5 . The method according to claim 1 , wherein the positions of the plurality of elements representing the object in the first image frame are determined based on relative spatial information of imaging device and the movable platform.

6 . The method according to claim 1 , wherein a current location of the movable platform is determined based on data collected by a plurality of sensors associated with the movable platform.

7 . The method according to claim 1 , wherein updating the locations of the tracking points of the object in the second image frame further includes:
tracking a movement trajectory of the tracking points using a light stream formed by the tracking points from the first image frame to the second image frame.

8 . The method according to claim 1 , further comprising:
determining a distance between the object and the movable platform based on information obtained from the updated disparity map.

9 . The method according to claim 8 , further comprising:
generating, according to a determination of the distance between the object and the movable platform to be in a predetermined threshold value, a notification to be sent to a control device of the movable platform.

10 . The method according to claim 1 , further comprising:
selecting a subset of the elements from a raw disparity map directly obtained from a stereographic image frame by comparing disparity values of the plurality of elements with threshold disparity values of a plurality of projections on the 2-dimensional mask that corresponds to the plurality of elements, wherein: the 2-dimension mask includes one or more projection points for defining a predefined 3-dimensional volume adjacent to the movable platform, each projection point having a threshold disparity value for the object within the predefined 3-dimensional volume; and a subset of the disparity values represents actual objects in the predefined 3-dimensional volume.

11 . A system for processing imaging data, comprising:
one or more processors; a memory; and one or more programs, wherein the one or more of programs are stored in the memory and when executed by the one or more processors, cause the one or more processors to:
identify an object in a disparity map to be tracked by a movable platform;
determine positions of a plurality of elements representing the object in a first image frame captured by an imaging device carried by the movable platform;
select one or more characteristic points of the plurality of elements representing the object as tracking points of the object in the first image frame; and
update, according to an updated disparity map and a current location of the movable platform, locations of the tracking points of the object in a second image frame captured by the imaging device.

12 . The system according to claim 11 , wherein one or more of programs further includes instructions for:
obtaining the disparity map based on a stereographic image captured by a stereoscopic camera carried by the movable platform.

13 . The system according to claim 11 , wherein one or more of programs further includes instructions for:
determining a continuous region having one or more elements with disparity values within a predefined range to be the object, wherein the disparity values of the one or more elements are higher than disparity values of other elements in the continuous region.

14 . The system according to claim 11 , wherein the plurality of characteristic points are selected to be closer to a navigation path of the movable platform than a rest part of the object.

15 . The system according to claim 11 , wherein the positions of the plurality of elements representing the object in the first image frame are determined based on relative spatial information of imaging device and the movable platform.

16 . The system according to claim 11 , wherein a current location of the movable platform is determined based on data collected by a plurality of sensors associated with the movable platform.

17 . The system according to claim 11 , wherein the instructions for updating the locations of the tracking points of the object in the second image frame further includes instructions for:
tracking a movement trajectory of the tracking points using a light stream formed by the tracking points from the first image frame to the second image frame.

18 . The system according to claim 11 , wherein the one or more of programs further includes instructions for:
determining a distance between the object and the movable platform based on information obtained from the updated disparity map.

19 . An unmanned aerial vehicle (UAV), comprising:
a propulsion system; one or more sensor devices including a stereoscopic camera; an imaging device; and one or more processors and a memory, the memory storing one or more programs, when executed, causing the one or more processors to:
identify an object in a disparity map to be tracked by a movable platform;
determine positions of a plurality of elements representing the object in a first image frame captured by an imaging device carried by the movable platform;
select one or more characteristic points of the plurality of elements representing the object as tracking points of the object in the first image frame; and
update, according to an updated disparity map and a current location of the movable platform, locations of the tracking points of the object in a second image frame captured by the imaging device.

Assignees

• Sz Dji Technology Co Ltd

Inventors

• Liu Ang
• Xu Pu

Classifications

• G06V20/17

  taken from planes or by drones · CPC title

• B64U2101/30

  for imaging, photography or videography · CPC title

• G06V20/13

  Satellite images · CPC title

• G06V10/50

  by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title