Information processing device and information processing method, imaging device, computer program, information processing system, and moving body device

Patent metadata

Field	Value
Publication number	US-11994581-B2
Application number	US-201917254212-A
Country	US
Kind code	B2
Filing date	May 14, 2019
Priority date	Jun 29, 2018
Publication date	May 28, 2024
Grant date	May 28, 2024

Abstract

Provided is an information processing device for performing fusion processing of a plurality of sensors for recognizing the outside world.
An information processing device includes a control unit that controls an amount of information to be used for object detection in a first information sensed by a first sensor on the basis of second information sensed by a second sensor. The first sensor is a camera, and the second sensor is a radar. Then, the control unit determines the target area from the captured image on the basis of information on at least one of a position, a speed, or a size of an object detected by the radar.

First claim

The invention claimed is:

1. An information processing device comprising
a control unit that controls an amount of information to be used for object detection in a first information sensed by a first sensor on a basis of second information sensed by a second sensor; and
a condition determination unit coupled to the control unit that determines an imaging condition of the camera;

wherein
the first sensor is a camera,
the control unit determines a target area to be used for object detection in a captured image by the camera on the basis of the second information,
the second sensor is a radar, and
the control unit determines the target area from the captured image on a basis of information on at least one of a position, a speed, or a size of an object detected by the radar by using a threshold value set on a basis of a determination result of the condition determination unit.

2. The information processing device according to claim 1 , wherein
the control unit determines the target area from the captured image on the basis of the second information regarding an object sensed by the second sensor.

3. The information processing device according to claim 1 , wherein
the control unit determines the target area from the captured image on a basis of the information of the object detected by the radar by using either a threshold value set high in a case where the imaging condition of the camera is good or a threshold value set low in a case where the imaging condition of the camera is not good.

4. The information processing device according to claim 1 , further comprising
a size determination unit that determines a size of the target area on a basis of an intensity of a reflected wave to be detected by the radar, wherein
the control unit determines the target area from the captured image on a basis of the size determined by the size determination unit.

5. The information processing device according to claim 1 , wherein
the control unit sets an upper limit number of the target area.

6. The information processing device according to claim 1 , further comprising
a priority determination unit that determines a priority of each object detected by the radar, wherein
the control unit determines the target area from the captured image on a basis of the priority.

7. The information processing device according to claim 6 , wherein
the priority determination unit determines the priority on a basis of the position and a relative speed of the object detected by the radar.

8. The information processing device according to claim 7 , wherein
the priority determination unit determines a higher priority for an object detected in front of the information processing device or an object having a negative relative speed with respect to the information processing device.

9. An information processing method comprising
a control step of controlling an amount of information to be used for object detection in a first information sensed by a first sensor on a basis of second information sensed by a second sensor;
determining an imaging condition of the camera; and
determining a target area to be used for object detection in a captured image by the camera on the basis of the second information, comprising:
determining the target area from the captured image on a basis of information on at least one of a position, a speed, or a size of an object detected by the radar by using a threshold value.

10. A computer program written on a non-transitory computer readable medium to execute, on a computer, processing for:
controlling an amount of information to be used for object detection in a first information sensed by a first sensor on a basis of second information sensed by a second sensor;
determining an imaging condition of the camera; and
determining a target area to be used for object detection in a captured image by the camera on the basis of the second information, comprising:
determining the target area from the captured image on a basis of information on at least one of a position, a speed, or a size of an object detected by the radar by using a threshold value.

11. An imaging device comprising:
an imaging unit comprising a first sensor; and
a control unit coupled to the imaging unit that determines a target area to be used for object detection in a captured image by the imaging unit on a basis of information sensed by another sensor;

wherein
the first sensor is a camera,
the control unit determines a target area to be used for object detection in a captured image by the camera on the basis of the information by the another sensor,
the another sensor is a radar, and
the control unit determines the target area from the captured image on a basis of information on at least one of a position, a speed, or a size of an object detected by the radar by using a threshold value set on a basis of a determination result of the condition determination unit.

12. An information processing system comprising:
a first object detection unit that detects an object on a basis of an amount of information determined on a basis of second information sensed by a second sensor, in first information sensed by a first sensor;
a determination unit coupled to the first object determination unit that determines the object by fusing an object detection result by the first object detection unit with an object detection result using the second sensor; and
a control unit that determines a target area to be used for object detection in a captured image by the camera on the basis of the second information,

wherein
the first sensor is a camera,
the second sensor is a radar, and
the control unit determines the target area from the captured image on a basis of information on at least one of a position, a speed, or a size of an object detected by the radar by using a threshold value set on a basis of a determination result of the determination unit.

13. A moving body device comprising:
a moving means;
a first sensor;
a second sensor,
a first object detection unit that detects an object on a basis of an amount of information determined on a basis of second information sensed by a second sensor, in first information sensed by a first sensor,
a determination unit coupled to the first object detection unit that determines the object by fusing an object detection result by the first object detection unit with an object detection result using the second sensor;
a drive control unit that controls drive of the moving means on a basis of a determination result by the determination unit and
a control unit that determines a target area to be used for object detection in a captured image by the camera on the basis of the second information,

wherein
the first sensor is a camera,
the second sensor is a radar, and
the control unit determines the target area from the captured image on a basis of information on at least one of a position, a speed, or a size of an object detected by the radar by using a threshold value set on a basis of a determination result of the determination unit.

Assignees

• Sony Corp
• Sony Semiconductor Solutions Corp

Inventors

• Tamaoki Akihiro

Classifications

• G01S13/931 Primary

  of land vehicles · CPC title

• G01S13/867

  Combination of radar systems with cameras · CPC title

• G01S13/93

  for anti-collision purposes · CPC title

• G06F18/251

  of input or preprocessed data · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title