Map element extraction method and apparatus, and server

Patent metadata

Field	Value
Publication number	US-11380002-B2
Application number	US-202017075134-A
Country	US
Kind code	B2
Filing date	Oct 20, 2020
Priority date	Oct 12, 2018
Publication date	Jul 5, 2022
Grant date	Jul 5, 2022

Abstract

This application discloses a map element extraction method and apparatus, and a server. The map element extraction method includes obtaining a laser point cloud and an image of a target scene, the target scene including a map element; performing registration between the laser point cloud and the image to obtain a depth map of the image; performing image segmentation on the depth map of the image to obtain a segmented image of the map element in the depth map; and converting a two-dimensional location of the segmented image in the depth map to a three-dimensional location of the map element in the target scene according to a registration relationship between the laser point cloud and the image.

First claim

What is claimed is:

1. A map element extraction method, performed by an electronic device, the method comprising:
obtaining a laser point cloud and an image of a target scene, the target scene comprising a map element;
performing registration between the laser point cloud and the image to obtain a depth map of the image;
performing feature extraction on the depth map of the image to obtain a feature map corresponding to the image;
performing class prediction on a pixel of the feature map to obtain a class of the pixel of the feature map;
fitting pixels belonging to the same class in the feature map to form a segmented image of the map element in the depth map, the class corresponding to one type of map element; and
converting a two-dimensional location of the segmented image in the depth map to a three-dimensional location of the map element in the target scene according to a registration relationship between the laser point cloud and the image.

2. The method according to claim 1 , wherein the performing registration between the laser point cloud and the image to obtain a depth map of the image comprises:
performing registration between the laser point cloud and the image to obtain depth information corresponding to a pixel in the image; and
constructing the depth map for the image according to the depth information corresponding to the pixel in the image.

3. The method according to claim 2 , wherein the performing registration between the laser point cloud and the image to obtain depth information corresponding to a pixel in the image comprises:
constructing a projective transformation function between the laser point cloud and the image;
extracting corresponding feature points from the laser point cloud and the image, and estimating a parameter of the projective transformation function according to the extracted feature points; and
calculating the depth information corresponding to the pixel in the image according to the projective transformation function with the estimated parameter.

4. The method according to claim 3 , wherein the converting a two-dimensional location of the segmented image in the depth map to a three-dimensional location of the map element in the target scene according to a registration relationship between the laser point cloud and the image comprises:
inputting the two-dimensional location of the segmented image in the depth map and the depth information corresponding to the pixel in the image into the projective transformation function with the estimated parameter to calculate the three-dimensional location of the map element in the target scene.

5. The method according to claim 1 , wherein the performing feature extraction on the depth map of the image to obtain a feature map corresponding to the image comprises:
extracting global features of the depth map by using high-level networks in a residual neural network and extracting local features of the depth map by using low-level networks in the residual neural network;
fusing the global features and the local features that are extracted to obtain an intermediate feature map; and
performing linear interpolation on the intermediate feature map to obtain the feature map corresponding to the image.

6. The method according to claim 5 , wherein the fusing the global features and the local features that are extracted to obtain an intermediate feature map comprises:
performing deconvolution and upsampling processing on a global feature corresponding to a highest-level network in the residual neural network to obtain a fused feature map;
performing deconvolution processing on a global feature corresponding to a second highest-level network in the residual neural network and updating the fused feature map by fusing the global feature corresponding to the second highest-level network and the fused feature map; and
performing upsampling processing on the updated fused feature map to update the fused feature map a second time;
traversing global features corresponding to the remaining high-level networks and the local features corresponding to the low-level networks in descending order of networks in the residual neural network, and updating, according to the traversed global features or local features, the fused feature map updated twice; and
using the updated fused feature map as the intermediate feature map after the traversing is completed.

7. The method according to claim 5 , further comprising:
obtaining an image sample, the image sample being labelled with a pixel class;
guiding, according to the obtained image sample, a specified mathematical model to perform model training; and
obtaining the residual neural network constructed by the specified mathematical model that completes the model training.

8. The method according to claim 1 , further comprising:
displaying the map element in a map of the target scene according to the three-dimensional location of the map element in the target scene; and
obtaining a control instruction for the map element in the map of the target scene and generating a high-precision map of the target scene in response to the control instruction.

9. An electronic device, comprising:
a processor; and
a memory, the memory storing computer-readable instructions, and the computer-readable instructions, when executed by the processor, implementing a map element extraction method, the method comprising:
obtaining a laser point cloud and an image of a target scene, the target scene comprising a map element;
performing registration between the laser point cloud and the image to obtain a depth map of the image;
performing feature extraction on the depth map of the image to obtain a feature map corresponding to the image;
performing class prediction on a pixel of the feature map to obtain a class of the pixel of the feature map;
fitting pixels belonging to the same class in the feature map to form a segmented image of the map element in the depth map, the class corresponding to one type of map element; and
converting a two-dimensional location of the segmented image in the depth map to a three-dimensional location of the map element in the target scene according to a registration relationship between the laser point cloud and the image.

10. The device according to claim 9 , wherein the performing registration between the laser point cloud and the image to obtain a depth map of the image comprises:
performing registration between the laser point cloud and the image to obtain depth information corresponding to a pixel in the image; and
constructing the depth map for the image according to the depth information corresponding to the pixel in the image.

11. A non-transitory computer-readable storage medium, storing a computer program, the computer program, when executed by a processor, causing the processor to implement a map element extraction method comprising:
obtaining a laser point cloud and an image of a target scene, the target scene comprising a map element;
performing registration between the laser point cloud and the image to obtain a depth map of the image;
performing feature extraction on the depth map of the image to obtain a feature map corresponding to the image;
performing class prediction on a pixel of the feature map to obtain a class of the pixel of the feature map;
fitting pixels belonging to the same class in the feature map to form a segmented image of the map element in the depth map, the class corresponding to one type of map element; and
converting a two-dimensional location of the segmented image in the depth map to a three-dimensional location of the map element in the target scene according to a registration relationship between the las

Assignees

• Tencent Tech Shenzhen Co Ltd

Inventors

• Shu Mao
• Chen Si

Classifications

• G01C21/3602 Primary

  Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera · CPC title

• G06V20/647

  by matching two-dimensional images to three-dimensional objects · CPC title

• G06V10/806

  of extracted features · CPC title

• G06V10/26

  Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion · CPC title

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title