Method and apparatus with pose estimation

Patent metadata

Field	Value
Publication number	US-12475585-B2
Application number	US-202217735541-A
Country	US
Kind code	B2
Filing date	May 3, 2022
Priority date	Nov 15, 2021
Publication date	Nov 18, 2025
Grant date	Nov 18, 2025

Abstract

A processor-implemented method with pose estimation includes: determining depth data by sensing a depth of a target scene comprising planes orthogonal to each other; determining normal vectors corresponding to depth points of the depth data; determining orientation data of a device by comparing the normal vectors to orientation candidates; and determining position data of the device based on distances from the device to the planes.

First claim

What is claimed is:

1 . A processor-implemented method with pose estimation, the method comprising:
determining depth data by sensing a depth of a target scene comprising planes orthogonal to each other; determining normal vectors corresponding to depth points of the depth data; determining orientation data of a device by comparing the normal vectors to orientation candidates, the orientation candidates being possible orientations of the devices; determining position data of the device based on distances from the device to the planes; and providing an augmented reality (AR) and/or virtual reality (VR) function using an estimated pose associated with the orientation data and the position data of the device.

2 . The method of claim 1 , wherein the planes orthogonal to each other comprise at least a portion of walls orthogonal to each other and a floor orthogonal to the walls.

3 . The method of claim 1 , wherein the determining of the normal vectors comprises:
generating three-dimensional (3D) space data comprising scene points corresponding to the depth points by unprojecting the depth data onto a 3D space; determining a first local plane based on a first scene point of the 3D space data and neighboring scene points located in a neighborhood of the first scene point; and determining a first normal vector of the first scene point based on a normal of the first local plane.

4 . The method of claim 1 , wherein the determining of the orientation data comprises:
determining a matching number of the orientation candidates in response to performing matching between the normal vectors and the orientation candidates; and determining the orientation data based on the matching number of the orientation candidates.

5 . The method of claim 4 , wherein
the normal vectors comprise a first normal vector, and the determining of the matching number comprises matching the first normal vector and one or more orientation candidates having either one of an orthogonal relationship and a parallel relationship with the first normal vector among the orientation candidates.

6 . The method of claim 1 , wherein the determining of the position data comprises:
determining a scene coordinate system based on the distances from the device to the planes and the orientation data; and determining coordinates corresponding to a current position of the device in the scene coordinate system.

7 . The method of claim 6 , wherein
the planes comprise walls orthogonal to each other and a floor orthogonal to the walls, and the determining of the scene coordinate system comprises:
aligning a coordinate axis of the scene coordinate system based on the orientation data; and
determining an intersection of the walls and the floor to be an origin of the scene coordinate system.

8 . The method of claim 1 , wherein the determining of the depth data comprises sensing the target scene using either one or both of a camera and a depth sensor of the device.

9 . The method of claim 1 , wherein
the depth data is determined from sensing data of a current time point, and the orientation data and the position data correspond to an absolute estimate of the current time point and are determined independently of sensing data of another time point.

10 . The method of claim 9 , further comprising determining a pose of the current time point based on a relative estimate according to a comparison between sensing data of a previous time point and the sensing data of the current time point and the absolute estimate according to the sensing data of the current time point.

11 . The method of claim 10 , wherein the relative estimate is determined using at least a portion of a visual inertial odometry (VIO) and simultaneous localization and mapping (SLAM).

12 . A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, configured the one or more processors to perform the method of claim 1 .

13 . The method of claim 1 ,
wherein the determining of the normal vectors comprises determining the normal vectors using a device, and wherein the comparing of the normal vectors to the orientation candidates comprises determining, for each of the orientation candidates, a number of the normal vectors orthogonal or parallel to reference directions of the orientation candidate; and further comprising determining a pose of the device by determining, as an orientation of the device, an orientation candidate of the orientation candidates corresponding to a greatest number among the determined numbers.

14 . An apparatus with pose estimation, the apparatus comprising:
one or more processors configured to:
determine depth data by sensing a depth of a target scene comprising planes orthogonal to each other;
determine normal vectors corresponding to depth points of the depth data;
determine orientation data of a device by comparing the normal vectors to orientation candidates, the orientation candidates being possible orientations of the devices;
determine position data of the device based on distances from the device to the planes; and
provide an augmented reality (AR) and/or virtual reality (VR) function using an estimated pose associated with the orientation data and the position data of the device.

15 . The apparatus of claim 14 , wherein the planes orthogonal to each other comprise at least a portion of walls orthogonal to each other and a floor orthogonal to the walls.

16 . The apparatus of claim 14 , wherein, for the determining of the normal vectors, the one or more processors are configured to:
generate three-dimensional (3D) space data comprising scene points corresponding to the depth points by unprojecting the depth data onto a 3D space; determine a first local plane based on a first scene point of the 3D space data and neighboring scene points located in a neighborhood of the first scene point; and determine a first normal vector of the first scene point based on a normal of the first local plane.

17 . The apparatus of claim 14 , wherein, for the determining of the orientation data, the one or more processors are configured to:
determine a matching number of the orientation candidates in response to performing matching between the normal vectors and the orientation candidates; and determine the orientation data based on the matching number of the orientation candidates.

18 . The apparatus of claim 14 , wherein, for the determining of the position data, the one or more processors are configured to:
determine a scene coordinate system based on the distances from the device to the planes and the orientation data; and determine coordinates corresponding to a current position of the device in the scene coordinate system.

19 . The apparatus of claim 14 , further comprising a memory storing instructions that, when executed by the one or more processors, configure the one or more processors to perform the determining of the depth data, the determining of the normal vectors, the determining of the orientation data, and the determining of the position data.

20 . An electronic apparatus comprising:
a sensing device configured to generate depth data by sensing a depth of a target scene comprising planes orthogonal to each other; and one or more processors configured to:
determine normal vectors corresponding to depth points of the depth data,
determine orientation data of a device by comparing the normal vector to orientation candidates, the orientation ca

Assignees

• Samsung Electronics Co Ltd

Inventors

• Chang Hyun Sung
• Kim Seungeon
• Son Minjung
• Lee Wonhee
• Jung Kyungboo
• Ha Inwoo

Classifications

• G06T7/73

  using feature-based methods · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06V20/00

  Scenes; Scene-specific elements (control of digital cameras H04N23/60) · CPC title

• G06V20/64

  Three-dimensional [3D] objects · CPC title

• G06T7/33

  using feature-based methods · CPC title