Information processing device, system, and method

Patent metadata

Field	Value
Publication number	US-12499765-B2
Application number	US-202318344941-A
Country	US
Kind code	B2
Filing date	Jun 30, 2023
Priority date	Aug 30, 2022
Publication date	Dec 16, 2025
Grant date	Dec 16, 2025

Abstract

An information processing device mounted on a vehicle including a sensor includes an acquisition unit that acquires a surrounding image of the vehicle and information related to a behavior of the vehicle from the sensor, and an identification unit that identifies a moving object in the surrounding image of the vehicle based on a chronological change of the surrounding image of the vehicle and the information related to the behavior of the vehicle.

First claim

What is claimed is:

1. An information processing device mounted on a vehicle including a sensor comprising a vehicle speed sensor or a vehicle steering sensor, the information processing device comprising:
a memory storing non-transitory computer-executable instructions, and a processor configured to execute the instructions to:
acquire
a reference surrounding image of the vehicle at a reference time,
a first surrounding image at a first time after a predetermined time has elapsed from the reference time, and
a second surrounding image at a second time after the predetermined time has further elapsed from the first time;

acquire information related to a behavior of the vehicle from the sensor;
estimate, based on the vehicle behavior information, a first estimated surrounding image after the predetermined time from the reference surrounding image, and a second estimated surrounding image after the further predetermined time from the first surrounding image,
identify a moving object in the surrounding images of the vehicle based on a predetermined differential process between the first surrounding image and the first estimated surrounding image, and between the second surrounding image and the second estimated surrounding image, and
transmit via an input/output interface, data related to the identified moving object among data of the surrounding images of the vehicle to a remote server.

2. The information processing device according to claim 1 , wherein the information related to the behavior of the vehicle is at least one of a speed of the vehicle and a steering angle of the vehicle.

3. A system comprising:
an information processing device mounted on a vehicle including a sensor comprising a vehicle speed sensor or a vehicle steering sensor; and
a server that is able to communicate with the information processing device, wherein
the information processing device includes
a memory storing non-transitory computer-executable instructions, and a processor configured to execute the instructions to:
acquire
a reference surrounding image of the vehicle at a reference time,
a first surrounding image at a first time after a predetermined time has elapsed from the reference time, and
a second surrounding image at a second time after the predetermined time has further elapsed from the first time;

acquire information related to a behavior of the vehicle from the sensor,
estimate, based on the vehicle behavior information, a first estimated surrounding image after the predetermined time from the reference surrounding image, and a second estimated surrounding image after the further predetermined time from the first surrounding image,
identify a moving object in the surrounding images of the vehicle based on a predetermined differential process between the first surrounding image and the first estimated surrounding image, and between the second surrounding image and the second estimated surrounding image, and
transmit data related to the moving object among data of the surrounding images of the vehicle to the server, and
the server includes a second memory storing non-transitory computer-executable instructions, and a second processor configured to execute the instructions to:
receive the data related to the moving object from the information processing device,
store data related to a structure included in the surrounding images of the vehicle, and
integrate the data related to the moving object and the data related to the structure.

4. The system according to claim 3 , wherein the information related to the behavior of the vehicle is at least one of a speed of the vehicle and a steering angle of the vehicle.

5. The system according to claim 3 , wherein a digital twin is provided in a virtual space that is time-synchronized with a reality space, using data obtained by integrating the data related to the moving object and the data related to the structure.

6. A method executed by a computer of an information processing device mounted on a vehicle including a sensor comprising a vehicle speed sensor or a vehicle steering sensor, the method comprising:
a step of acquiring
a reference surrounding image of the vehicle at a reference time,
a first surrounding image at a first time after a predetermined time has elapsed from the reference time, and
a second surrounding image at a second time after the predetermined time has further elapsed from the first time;

a step of acquiring information related to a behavior of the vehicle from the sensor; and
a step of estimating, based on the vehicle behavior information, a first estimated surrounding image after the predetermined time from the reference surrounding image, and a second estimated surrounding image after the further predetermined time from the first surrounding image,
a step of identifying a moving object in the surrounding images of the vehicle based on a predetermined differential process between the first surrounding image and the first estimated surrounding image, and between the second surrounding image and the second estimated surrounding image, and
a step of transmitting via an input/output interface, data related to the identified moving object among data of the surrounding images of the vehicle to a remote server.

Assignees

• Toyota Motor Co Ltd

Inventors

• Yoshida Kaoru
• Kakutani Masatoshi
• Okuda Masataka
• Maki Hiroki
• Watanabe Kosuke
• Kuriyama Kanade

Classifications

• G08G1/0112

  from the vehicle, e.g. floating car data [FCD] · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G01C1/00

  Measuring angles · CPC title

• G01P15/00

  Measuring acceleration; Measuring deceleration; Measuring shock, i.e. sudden change of acceleration · CPC title

• G08G1/163 Primary

  involving continuous checking · CPC title