Side outer extraction system and method

Patent metadata

Field	Value
Publication number	US-12499556-B2
Application number	US-202318230915-A
Country	US
Kind code	B2
Filing date	Aug 7, 2023
Priority date	Nov 9, 2022
Publication date	Dec 16, 2025
Grant date	Dec 16, 2025

Abstract

A side outer extraction method may include receiving, by at least one processor, an image, of a vehicle, preprocessed from three-dimensional (3D) data from a computer-aided design (CAD) module, detecting, using an artificial intelligence model, a classification value and a bounding box for each region, of a plurality of regions, corresponding to one of a plurality of target references of the preprocessed image, transmitting, to the CAD module, a signal indicating the classification value and the bounding box for each region of the plurality of regions, and causing extraction, by the CAD module, of the plurality of target references from the classification value and the bounding box for each region of the plurality of regions, based on the received signal.

First claim

What is claimed is:

1 . A method comprising:
receiving, by at least one processor, an image, of a vehicle, preprocessed from three-dimensional (3D) data from a computer-aided design (CAD) module; detecting, using an artificial intelligence model, a classification value and a bounding box for each region, of a plurality of regions, corresponding to one of a plurality of target references of the preprocessed image, wherein the plurality of target references comprise lines of the vehicle; transmitting, to the CAD module, a signal indicating the classification value and the bounding box for each region of the plurality of regions; and causing extraction, by the CAD module, of the plurality of target references from the classification value and the bounding box for each region of the plurality of regions, based on the received signal, wherein each of the plurality of regions is a box-shaped region or a segmentation region, and wherein the extraction of the plurality of target references comprises extracting, by the CAD module, as each target reference of the plurality of target references, a line including a black-color pixel by moving pixel-by-pixel in a predetermined direction from a predetermined reference point within each region.

2 . The method of claim 1 , further comprising:
receiving a training dataset from the CAD module; and performing, using the artificial intelligence model, a learning process associated with a method for detecting the plurality of regions from the preprocessed image based on the training dataset.

3 . The method of claim 2 , further comprising:
providing, by the CAD module, the preprocessed image to an application through a terminal; receiving, by the CAD module and from a manager device, data indicating position values and classification values for a plurality of regions of the preprocessed image; and generating, by the CAD module, the training dataset based on the data.

4 . The method of claim 3 , wherein the plurality of regions have a box shape, and
wherein the detecting of the classification value and the bounding box for each region of the plurality of regions comprises:
generating a feature map from the preprocessed image;
deriving at least one region proposal from the feature map;
pooling the feature map to a warped feature with a fixed size from the at least one region proposal;
generating data indicating a probability distribution that a class for the at least one region proposal is likely to be present, from the warped feature with the fixed size; and
defining positions of a plurality of bounding boxes that enclose the plurality of target references with respect to the at least one region proposal from the warped feature with the fixed size.

5 . The method of claim 4 , wherein the detecting of the classification value and the bounding box for each region of the plurality of regions is based on the data indicating the probability distribution and further based on the positions of the plurality of bounding boxes.

6 . The method of claim 5 , wherein the extraction of the plurality of target references comprises:
extracting a line from a quadrilateral representing the bounding box based on the classification value for each region.

7 . The method of claim 3 , wherein the position values for the plurality of regions are for a plurality of segmented regions, and
wherein the detecting of the classification value and the bounding box for each region of the plurality of regions comprises:
detecting an object mask in which there is a target reference of the bounding box for each of the plurality of the segmented regions.

8 . The method of claim 7 , wherein the detecting of the classification value and the bounding box for each region of the plurality of regions further comprises:
generating a feature map from the preprocessed image; deriving at least one region proposal from the feature map; extracting a warped feature with a fixed size by aligning the feature map and the at least one region proposal; generating data indicating a probability distribution that a class for the at least one region proposal is likely to be present from the warped feature with the fixed size; defining positions of a plurality of bounding boxes that enclose the plurality of target references with respect to the at least one region proposal from the warped feature with a fixed size; and generating a mask by predicting whether classes are present in all pixels from the warped feature with the fixed size.

9 . The method of claim 8 , further comprising:
based on the data indicating the probability distribution that the class for the at least one region proposal is likely to be present, based on the positions of the plurality of bounding boxes, and based on the mask, detecting:
the classification value for each of the plurality of segmented regions,
a target bounding box, and
an object mask in the target bounding box.

10 . The method of claim 1 , wherein the plurality of target references comprise at least one of: a vehicle fuel filler line, a vehicle roof unit line, a side door boundary line of a vehicle back seat, a tail lamp boundary line of the vehicle back seat, a side door boundary line of a vehicle back lamp, a line of a thickest center surface at a side door side of a vehicle front seat, or a line of a thickest center surface at the side door side of the vehicle back seat.

11 . A system comprising:
a region detection server configured to:
perform a learning process associated with a method for detecting, from a preprocessed image of a vehicle and using an artificial intelligence model, a plurality of regions corresponding to a plurality of target references; and
detect, via the learned method, the plurality of regions; and
a computer-aided design (CAD) module configured to:
extract a rendering image from three-dimensional (3D) data for the vehicle;
generate, based on the rendering image, the preprocessed image;
generate a training dataset based on a signal received from a management terminal; and
extract the plurality of target references from the plurality of regions,
wherein each of the plurality of regions is a box-shaped region or a segmentation region, and
wherein the CAD module is further configured to extract, as each target reference of the plurality of target references, a line including a black-color pixel by moving pixel-by-pixel in a predetermined direction from a predetermined reference point within each region.

12 . The system of claim 11 , wherein the CAD module comprises:
a preprocessor configured to:
extract, from the 3D data, the rendering image for locating a side outer of the vehicle in a front portion of the vehicle; and
generate, based on the rendering image, the preprocessed image by adjusting at least one of:
a background of a product in the rendering image,
a color of the product in the rendering image,
a color of a line,
thickness of the line, and
a shade; and

a reference mapper configured to:
extract the plurality of target references from the plurality of regions.

13 . The system of claim 11 , wherein the artificial intelligence model comprises:
a box detector configured to detect, from the preprocessed image, a plurality of first regions having a box shape; and a segmentation detector configured to detect a plurality of second regions segmented from the preprocessed image.

14 . The system of claim 13 , wherein the box detector comprises:
a backbone network configured to generate, based on the preprocessed image, a feature map; a region proposal network (RPN) layer configured to derive

Assignees

• Hyundai Motor Co Ltd
• Kia Corp
• Research & Business Found Sungkyunkwan Univ

Inventors

• Park Sunghyun
• Jun Sang Hwan
• Lee Jee-Hyong
• Lee Eun-Ho
• Kim Tae-Hyun
• Lee Jin Sub

Classifications

• G06T3/18

  Image warping, e.g. rearranging pixels individually · CPC title

• G06F18/2415

  based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2200/04

  involving 3D image data · CPC title