Signal processing apparatus, signal processing method, and program

Patent metadata

Field	Value
Publication number	US-10908257-B2
Application number	US-201615762136-A
Country	US
Kind code	B2
Filing date	Sep 16, 2016
Priority date	Sep 30, 2015
Publication date	Feb 2, 2021
Grant date	Feb 2, 2021

Abstract

A signal processing apparatus including a first position calculation unit that calculates a three-dimensional position of a target on a first coordinate system from a stereo image captured by a stereo camera, a second position calculation unit that calculates a three-dimensional position of the target on a second coordinate system from a sensor signal of a sensor capable of obtaining position information of at least one of a lateral direction and a longitudinal direction and position information of a depth direction, a correspondence detection unit that detects a correspondence relationship between the target on the first coordinate system and the target on the second coordinate system, and a positional relationship information estimating unit that estimates positional relationship information of the first coordinate system and the second coordinate system on the basis of the detected correspondence relationship.

First claim

The invention claimed is:

1. A signal processing apparatus comprising:
circuitry configured to function as:
a first position calculation unit that calculates a three-dimensional position of a target on a first coordinate system from a stereo image;
a second position calculation unit that calculates a three-dimensional position of the target on a second coordinate system from a sensor signal;
a correspondence detection unit that detects a correspondence relationship between the target on the first coordinate system and the target on the second coordinate system; and
a positional relationship information estimating unit that estimates positional relationship information of the first coordinate system and the second coordinate system on the basis of the detected correspondence relationship.

2. The signal processing apparatus according to claim 1 ,
wherein the correspondence detection unit detects the correspondence relationship after collating individual targets on the first coordinate system and the second coordinate system with prior arrangement information of the target and identifying the target.

3. The signal processing apparatus according to claim 1 ,
wherein the correspondence detection unit detects the correspondence relationship by superimposing the three-dimensional position of the target on the first coordinate system and the three-dimensional position of the target on the second coordinate system over each other and making the targets arranged closest to each other correspond to each other.

4. The signal processing apparatus according to claim 1 ,
wherein the first position calculation unit calculates the three-dimensional position of the target whose motion is detected, and
the second position calculation unit calculates the three-dimensional position of the target whose motion is detected.

5. The signal processing apparatus according to claim 1 ,
wherein the first position calculation unit calculates three-dimensional positions of a plurality of the targets from at least one stereo image in one or more frames,
the second position calculation unit calculates three-dimensional positions of the plurality of the targets from sensor signals in one or more frames, and
the correspondence detection unit detects correspondence relationships between the plurality of the targets.

6. The signal processing apparatus according to claim 5 ,
wherein the first position calculation unit calculates three-dimensional positions of a plurality of the targets from a stereo image in one frame, and
the second position calculation unit calculates three-dimensional positions of the plurality of the targets from sensor signals in one frame.

7. The signal processing apparatus according to claim 1 , further comprising:
a storage unit that stores the three-dimensional positions of the target calculated by the first position calculation unit and the second position calculation unit,
wherein, where a predetermined number or more of three-dimensional positions of the target are accumulated in the storage unit, the correspondence detection unit starts detection of the correspondence relationship.

8. The signal processing apparatus according to claim 1 ,
wherein a plurality of the targets is arranged at different positions in a depth direction.

9. The signal processing apparatus according to claim 1 ,
wherein a plurality of the targets is arranged at different positions in a lateral direction.

10. The signal processing apparatus according to claim 1 ,
wherein a plurality of the targets is arranged at an identical height position.

11. The signal processing apparatus according to claim 1 ,
wherein a plurality of the targets is arranged at positions that are not overlapped with each other when viewed from a stereo camera.

12. The signal processing apparatus according to claim 1 ,
wherein the target is a human.

13. The signal processing apparatus according to claim 1 ,
wherein the target is an object with a predetermined texture.

14. The signal processing apparatus according to claim 1 ,
wherein the target is a pole-like object.

15. The signal processing apparatus according to claim 1 ,
wherein the positional relationship information of the first coordinate system and the second coordinate system is a rotation matrix and a translation vector.

16. The signal processing apparatus according to claim 1 ,
wherein the sensor is a millimeter-wave radar.

17. A signal processing method comprising the steps of:
using at least one processor:
calculating a three-dimensional position of a target on a first coordinate system from a stereo image;
calculating a three-dimensional position of the target on a second coordinate system from a sensor signal;
detecting a correspondence relationship between the target on the first coordinate system and the target on the second coordinate system; and
estimating positional relationship information of the first coordinate system and the second coordinate system on the basis of the detected correspondence relationship.

18. At least one non-transitory computer-readable storage medium encoded with executable instructions that, when executed by at least one processor, cause the at least one processor to perform a method comprising:
calculating a three-dimensional position of a target on a first coordinate system from a stereo image;
calculating a three-dimensional position of the target on a second coordinate system from a sensor signal;
detecting a correspondence relationship between the target on the first coordinate system and the target on the second coordinate system; and
estimating positional relationship information of the first coordinate system and the second coordinate system on the basis of the detected correspondence relationship.

Assignees

• Sony Corp

Inventors

• Motoyama Takuto
• Sutou Yasuhiro
• Yamazaki Toshio
• Oyaizu Hideki
• Kawasaki Kenichi

Classifications

• G06V20/647

  by matching two-dimensional images to three-dimensional objects · CPC title

• G08G1/166

  for active traffic, e.g. moving vehicles, pedestrians, bikes · CPC title

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G01S2013/9324

  Alternative operation using ultrasonic waves · CPC title

• G01S2013/9323

  Alternative operation using light waves · CPC title