Method and apparatus for determining disparity

Patent metadata

Field	Value
Publication number	US-9842400-B2
Application number	US-201514835078-A
Country	US
Kind code	B2
Filing date	Aug 25, 2015
Priority date	Jan 27, 2015
Publication date	Dec 12, 2017
Grant date	Dec 12, 2017

Abstract

A disparity determination method and apparatus are provided. The disparity determination method includes receiving first signals of an event from a first sensor disposed at a first location and second signals of the event from a second sensor disposed at a second location that is different than the first location, and extracting a movement direction of the event, based on at least one among the first signals and the second signals. The disparity determination method further includes determining a disparity between the first sensor and the second sensor, based on the movement direction, a difference between times at which the event is sensed by corresponding pixels in the first sensor, and a difference between times at which the event is sensed by corresponding pixels in the first sensor and the second sensor.

First claim

What is claimed is:

1. A disparity determination method of a processing device, the method comprising:
receiving first signals of an event from a first sensor disposed at a first location and second signals of the event from a second sensor disposed at a second location that is different than the first location;
extracting a movement direction of the event, based on at least one among the first signals and the second signals;
determining a disparity between the first sensor and the second sensor, based on the movement direction, a first difference between first times at which the event is sensed by corresponding first pixels in the first sensor, and a second difference between second times at which the event is sensed by corresponding second pixels in the first sensor and the second sensor;
generating depth information, based on the disparity; and
outputting the depth information.

2. The disparity determination method of claim 1 , wherein a resolution of the disparity is less than a size of pixels in each of the first sensor and the second sensor.

3. The disparity determination method of claim 1 , wherein the corresponding first pixels sense the event corresponding to a point of an object at the first times that are different.

4. The disparity determination method of claim 1 , further comprising setting, as the corresponding first pixels in the first sensor, a first pixel and a second pixel in the first sensor that sense the event, based on the movement direction.

5. The disparity determination method of claim 1 , wherein the extracting comprises:
comparing a time of an element corresponding to one of the first signals to a time of another element neighboring the element in an event map of the first sensor; and
extracting the movement direction based on a result of the comparing.

6. The disparity determination method of claim 1 , wherein the corresponding second pixels in the first sensor and the second sensor sense the event corresponding to a point of an object at the second times that are different.

7. The disparity determination method of claim 1 , further comprising setting, as the corresponding second pixels in the first sensor and the second sensor, a first pixel in the first sensor and a second pixel in the second sensor that sense the event, based on the movement direction, an arrangement of pixels in each of the first sensor and the second sensor, and the disparity having a resolution equal to a size of each of the corresponding second pixels in the first sensor and the second sensor.

8. The disparity determination method of claim 1 , further comprising:
determining whether the movement direction is matched to a virtual direction from the second location of the second sensor to the first location of the first sensor; and
setting third pixels corresponding to a same coordinate in the first sensor and the second sensor as the corresponding second pixels in the first sensor and the second sensor, in response to the determining that the movement direction is matched to the virtual direction.

9. The disparity determination method of claim 1 , wherein a resolution of the disparity is equal to a size of pixels in each of the first sensor and the second sensor.

10. The disparity determination method of claim 1 , wherein the determining comprises:
calculating an acceleration of the event, based on a third difference between third times at which the event is sensed by neighboring third pixels in the first sensor, and a fourth difference between fourth times at which the event is sensed by neighboring fourth pixels in the second sensor; and
determining the disparity, based on the acceleration, the first difference between the first times at which the event is sensed by the corresponding first pixels in the first sensor, and the second difference between the second times at which the event is sensed by the corresponding second pixels in the first sensor and the second sensor.

11. The disparity determination method of claim 1 , wherein the event comprises a same change of light incident on the first sensor being previously detected by the second sensor.

12. The disparity determination method of claim 1 , further comprising performing stereo matching between a first image generated based on the first signals and a second image generated based on the second signals, to generate the disparity having a resolution equal to a size of pixels in each of the first sensor and the second sensor.

13. The disparity determination method of claim 1 , wherein the determining comprises:
calculating the first difference between the first times at which the event is sensed by the corresponding first pixels in the first sensor, based on a first event map of the first sensor; and
calculating the second difference between the second times at which the event is sensed by the corresponding second pixels in the first sensor and the second sensor, based on the first event map and a second event map of the second sensor.

14. The disparity determination method of claim 1 , further comprising:
updating first timestamps of first elements corresponding to the first signals in a first event map of the first sensor, in response to the receiving the first signals; and
updating second timestamps of second elements corresponding to the second signals in a second event map of the second sensor, in response to the receiving the second signals.

15. A non-transitory computer-readable storage medium storing a program comprising instructions to cause a computer to implement the method of claim 1 .

16. A disparity determination apparatus comprising:
an extractor configured to:
receive first signals of an event from a first sensor disposed at a first location and second signals of the event from a second sensor disposed at a second location that is different than the first location, and
extract a movement direction of the event, based on at least one among the first signals and the second signals; and

a processor configured to:
determine a disparity between the first sensor and the second sensor, based on the movement direction, a first difference between first times at which the event is sensed by corresponding first pixels in the first sensor, and a second difference between second times at which the event is sensed by corresponding second pixels in the first sensor and the second sensor;
generate depth information, based on the disparity; and
outputting the depth information.

17. The disparity determination apparatus of claim 16 , wherein at least one among the first sensor and the second sensor comprises an event-based vision sensor configured to:
sense the event based on a change in incident light; and
output an event signal of the event that is sensed.

18. The disparity determination apparatus of claim 16 , wherein at least one among the first sensor and the second sensor comprises an event-based vision sensor comprising pixels, each of the pixels configured to:
sense a movement; and
output an event signal of the movement that is sensed.

19. The disparity determination apparatus of claim 16 , wherein at least one among the first sensor and the second sensor comprises:
a frame-based vision sensor configured to capture consecutive images for each frame; and
an output module configured to
sense the event based on a third difference between the consecutive images, and
output an event signal of the event.

20. The disparity determination apparatus of claim 16 , wherein the extractor is further configured to
receive th

Assignees

• Samsung Electronics Co Ltd

Inventors

• Shin Changwoo
• Lee Kyoobin
• Lee Jun Haeng
• Ryu Eric Hyunsurk
• Woo Jooyeon

Classifications

• G06T7/292

  Multi-camera tracking · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• H04N13/0037

  Electricity · mapped topic

• G06T2200/28

  involving image processing hardware · CPC title

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title