Method and mobility device using the method for path prediction through interaction analysis between

Patent metadata

Field	Value
Publication number	US-2025296554-A1
Application number	US-202418800213-A
Country	US
Kind code	A1
Filing date	Aug 12, 2024
Priority date	Mar 20, 2024
Publication date	Sep 25, 2025
Grant date	—

Abstract

A method for controlling an autonomous driving of a vehicle is introduced. The method may comprise generating a weighted adjacency matrix from an object graph, based on a location relative to an anticipated collision point among objects. The method may further comprise, based on this matrix and a graph convolution layer of a high-order graph model, generating an Mth-order adjacency matrix, defining interrelations among objects across M number of hops, generating object attribute information from this matrix and feature information defining an attribute of an object, generating, based on a combination layer of the high-order graph model, cumulative feature information by integrating the object attribute information, embedding the cumulative feature information into the object graph, generating, based on the embedding and a predicted path model, an object's predicted path, and outputting, based on this predicted path, a signal for controlling the autonomous driving of the vehicle.

First claim

What is claimed is:

1 . A method for controlling an autonomous driving of a vehicle, the method comprising:
generating, based on a location relative to an expected collision point between objects, a weighted adjacency matrix from an object graph, wherein the object graph represents a relation between the objects; generating, based on the weighted adjacency matrix and a graph convolution layer of a high-order graph model, an Mth-order adjacency matrix, wherein the Mth-order adjacency matrix defines interrelations between the objects at each of M number of hops, and wherein the high-order graph model captures indirect interrelations between the objects; determining, based on the Mth-order adjacency matrix and feature information defining an attribute of an object, object attribute information; generating, based on a combination layer of the high-order graph model, cumulative feature information by integrating the object attribute information, wherein the object attribute information is weighted for the integrating; embedding the cumulative feature information into the object graph, wherein the cumulative feature information is embedded into a target node of the object graph; generating, based on the embedding and a predicted path model, a predicted path of the object; and outputting, based on the predicted path of the object, a signal for controlling the autonomous driving of the vehicle.

2 . The method of claim 1 , wherein the generating the weighted adjacency matrix comprises applying a weight to an adjacency matrix, wherein the adjacency matrix expresses, based on a distance between the object and the expected collision point and based on a distance between the objects, concatenation relations between the objects.

3 . The method of claim 1 , wherein the expected collision point is determined, based on a direction vector of the object, by an intersection between half-lines, wherein the direction vector is formed based on a current location of the object.

4 . The method of claim 1 , wherein the expected collision point is determined by reflecting a semantic map that comprises semantic location information on environment of the object.

5 . The method of claim 1 , wherein the determining the object attribute information comprises performing matrix multiplication for the feature information, the Mth-order adjacency matrix, and a weight for learning the interrelations between the objects at each of the M number of hops.

6 . The method of claim 1 , wherein the Mth-order adjacency matrix is generated based on M number of superpositions of the weighted adjacency matrix.

7 . The method of claim 1 , wherein the generating the cumulative feature information comprises generating the cumulative feature information by extracting multiple pieces of the object attribute information as a single piece of information through the combination layer, wherein the determining the object attribute information comprises determining the object attribute information based on the interrelations between the objects at each of the M number of hops.

8 . The method of claim 7 , wherein the extracting comprises:
concatenating, based on a learnable weight allocated according to each of the M number of hops and weighted integration, the object attribute information at each of the M number of hops; and extracting the single piece of information from the concatenated object attribute information.

9 . The method of claim 1 , wherein the high-order graph model is modularized as a plug-and-play structure and is combined with the predicted path model.

10 . The method of claim 1 , wherein the feature information comprises behavioral information and state information according to a class of the object.

11 . An apparatus for controlling an autonomous driving of a vehicle, the apparatus comprising:
a processor; and a memory configured to store at least one instruction, when executed by the processor, cause the apparatus to: produce, based on a location relative to an expected collision point between objects, a weighted adjacency matrix from an object graph, wherein the object graph represents a relation between the objects; produce, based on the weighted adjacency matrix and a graph convolution layer of a high-order graph model, an Mth-order adjacency matrix, wherein the Mth-order adjacency matrix defines interrelations between the objects at each of M number of hops, and wherein the high-order graph model captures indirect interrelations between the objects; determine, based on the Mth-order adjacency matrix and feature information defining an attribute of an object, object attribute information; generate, based on a combination layer of the high-order graph model, cumulative feature information by integrating the object attribute information, wherein the object attribute information is weighted for the integrating; and embed the cumulative feature information into the object graph, wherein the cumulative feature information is embedded into a target node of the object graph; generate, based on the embedment and a predicted path model, a predicted path of the object; and output, based on the predicted path of the object, a signal for controlling the autonomous driving of the vehicle.

12 . The apparatus of claim 11 , wherein the at least one instruction, when executed by the processor, further cause the apparatus to apply a weight to an adjacency matrix, wherein the adjacency matrix expresses, based on a distance between the object and the expected collision point and based on a distance between the objects, concatenation relations between the objects.

13 . The apparatus of claim 11 , wherein the expected collision point is determined, based on a direction vector of the object, by an intersection between half-lines, wherein the direction vector is formed based on a current location of the object.

14 . The apparatus of claim 11 , wherein the expected collision point is determined by reflecting a semantic map that comprises semantic location information on environment of the object.

15 . The apparatus of claim 11 , wherein the at least one instruction, when executed by the processor, further cause the apparatus to perform matrix multiplication for the feature information, the Mth-order adjacency matrix, and a weight for learning the interrelations between the objects at each of the M number of hops.

16 . The apparatus of claim 11 , wherein the Mth-order adjacency matrix is produced based on M number of superpositions of the weighted adjacency matrix.

17 . The apparatus of claim 11 , wherein the at least one instruction, when executed by the processor, further cause the apparatus to:
determine the object attribute information based on the interrelations between the objects at each of the M number of hops; and extract multiple pieces of the object attribute information as a single piece of information through the combination layer.

18 . The apparatus of claim 17 , wherein the at least one instruction, when executed by the processor, further cause the apparatus to concatenate, based on a learnable weight allocated according to each of the M number of hops and weighted integration, the object attribute information at each of the M number of hops; and
extract the single piece of information from the object attribute information concatenated.

19 . The apparatus of claim 11 , wherein the high-order graph model is modularized as a plug-and-play structure and is combined with the predicted path model.

20 . The appar

Assignees

• Hyundai Motor Co Ltd
• Kia Corp
• Univ Korea Res & Bus Found

Inventors

• Lim Hye Rin
• Kim Jin Kyu
• Kim Sung June
• Kim Sang Pil

Classifications

• B60W2050/0044

  In digital systems · CPC title

• B60W2554/4041

  Position · CPC title

• B60W2554/4049

  Relationship among other objects, e.g. converging dynamic objects · CPC title

• G06N3/08

  Learning methods · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title