Method and an electronic apparatus for acquiring a floor map of a room layout

Patent metadata

Field	Value
Publication number	US-12249142-B2
Application number	US-202217748378-A
Country	US
Kind code	B2
Filing date	May 19, 2022
Priority date	May 25, 2021
Publication date	Mar 11, 2025
Grant date	Mar 11, 2025

Abstract

A method and an electronic apparatus for acquiring a floor map of a room layout are provided. The method includes acquiring a depth map and a two-dimensional (2D) image of an interior of a room; identifying a boundary line between a ceiling and a wall of the room in the acquired 2D image; determining valid pixels among pixels on the boundary line according to a first preset rule; identifying an electronic device in the room and determining a location of the electronic device; acquiring the floor map of the room layout by projecting three-dimensional (3D) coordinates of actual points corresponding to the valid pixels and 3D coordinates of the determined location of the electronic device onto a horizontal plane, wherein, the floor map of the room layout comprises connection lines between projections of the 3D coordinates of the actual points corresponding to the valid pixels and an icon indicating a type and the location of the electronic device, and wherein, the 3D coordinates of the actual points corresponding to the valid pixels are determined based on the depth map and the 2D image.

First claim

What is claimed is:

1. A method for acquiring a floor map of a room layout, the method comprising:
acquiring a depth map and a two-dimensional (2D) image of an interior of a room;
identifying a boundary line between a ceiling and a wall of the room in the acquired 2D image;
determining valid pixels among pixels on the boundary line according to a first preset rule, wherein the valid pixels indicate a pixel that corresponds to an actual point in the identified line at the junction of the ceiling and the wall;
identifying an electronic device in the room and determining a location of the electronic device based on SLAM positioning technology;
determining three-dimensional (3D) coordinates of the actual points at the junction of the ceiling and the wall of the room corresponding to the valid pixels and 3D coordinates of the determined location of the electronic device based on the depth map and the 2D image;
converting the 3D coordinates of the actual points corresponding to the valid pixels and the 3D coordinates of the determined location of the electronic device coordinate into coordinates in a global coordinate system; and
acquiring the floor map of the room layout by projecting the 3D coordinates of the actual points corresponding to the valid pixels on the boundary line converted into the global coordinate system and the 3D coordinates of the determined location of the electronic device converted into the global coordinate system onto a horizontal plane,
wherein the floor map of the room layout comprises connection lines between projections of the 3D coordinates of the actual points corresponding to the valid pixels and an icon indicating a type and the location of the electronic device.

2. The method of claim 1 , wherein the determining the valid pixels among the pixels on the boundary line according to the first preset rule comprises:
determining 3D coordinate of the actual point corresponding to each pixel on the boundary line and 3D coordinates of actual points corresponding to pixels near the boundary line based on the depth map, wherein the pixels near the boundary line are included in the 2D image and within a preset distance from the boundary line;
determining two adjacent points within a predetermined distance range from the actual point corresponding to the each pixel on the boundary line according to a second preset rule, and determining a direction of a plane formed by the actual point corresponding to the each pixel and the determined two adjacent points, wherein the two adjacent points are included in the actual points corresponding to the pixels near the boundary line;
sequentially determining a difference between directions of planes respectively corresponding to any two adjacent pixels on the boundary line;
based on the difference being less than or equal to a first threshold value, determining that the two adjacent pixels on the boundary line are valid pixels; and
based on the difference being greater than the first threshold value, determining that the two adjacent pixels on the boundary line are invalid pixels, and
wherein, when a previous pixel of a current pixel is determined to be an invalid pixel, a pixel closest to the current pixel among the valid pixels that are previously determined is determined to be a pixel adjacent to the current pixel.

3. The method of claim 1 , wherein the identifying the electronic device in the room and determining the location of the electronic device comprises:
determining whether electronic devices that have been identified include an electronic device of the same type as a currently identified electronic device;
based on the electronic devices that have been identified including the electronic device of the same type as the currently identified electronic device, determining whether a distance between the currently identified electronic device and the electronic device of the same type is greater than or equal to a second threshold;
based on determining that the distance is greater than or equal to the second threshold, determining that the currently identified electronic device is a new electronic device; and
based on determining that the distance is less than the second threshold, determining that the currently identified electronic device and the electronic device of the same type are the same electronic device.

4. The method of claim 3 , wherein the identifying the electronic device in the room and determining the location of the electronic device further comprises:
based on determining that the currently identified electronic device and the electronic device of the same type are the same electronic device, determining newly determined coordinate of the currently identified electronic device to be coordinate of the electronic device of the same type.

5. The method of claim 1 , further comprising:
identifying a furniture in the room and a location of the furniture; and
acquiring the location of the furniture in the floor map of the room layout by projecting 3D coordinates of the location of the furniture onto the horizontal plane,
wherein the floor map of the room layout comprises an icon representing a type and the location of the furniture.

6. The method of claim 1 , further comprising:
displaying the floor map of the room layout on an electronic terminal; and
establishing communication between the electronic terminal and the electronic device,
wherein the icon corresponding to the electronic device is included in the floor map and is used to control the electronic device.

7. An electronic apparatus for acquiring a floor map of a room layout, comprising:
a memory storing instructions; and
a processor configured to execute the instructions to:
acquire a depth map and a two-dimensional (2D) image of an interior of a room;
identify a boundary line between a ceiling and a wall of the room in the acquired 2D image;
determine valid pixels among pixels on the boundary line according to a first preset rule, wherein the valid pixels indicate a pixel that corresponds to the actual point in the identified line at the junction of the ceiling and the wall;
identify an electronic device in the room and determine a location of the electronic device based on SLAM positioning technology;

determine three-dimensional (3D) coordinates of the actual points at the junction of the ceiling and the wall of the room corresponding to the valid pixels and 3D coordinates of the determined location of the electronic device based on the depth map and the 2D image;
convert the 3D coordinates of the actual points corresponding to the valid pixels and the 3D coordinates of the determined location of the electronic device coordinate into coordinates in global coordinate system; and
acquire the floor map of the room layout by projecting the 3D coordinates of the actual points corresponding to the valid pixels on the boundary line converted into the global coordinate system and the 3D coordinate of the determined location of the electronic device converted into the global coordinate system onto a horizontal plane,

wherein the floor map of the room layout comprises connection lines between projections of the 3D coordinates of the actual points corresponding to the valid pixels and an icon indicating a type and the location of the electronic device.

8. The electronic apparatus of claim 7 , wherein the processor is further configured to execute the instructions to:
determine 3D coordinate of actual point corresponding to each pixel on the boundary line and 3D coordinates of actual points corresponding to pixels near the boundary line based on the depth map, wherein the pixels near the boundary line are included in the 2D image and within a preset distance from the boundary line;
determi

Assignees

• Samsung Electronics Co Ltd

Inventors

• Liu Juan
• Wang Huasin
• Yoon Sukun
• Wu Longhai
• Chen Jie

Classifications

• G06V20/64

  Three-dimensional [3D] objects · CPC title

• G06T7/50

  Depth or shape recovery · CPC title

• G06V10/751

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T2200/24

  involving graphical user interfaces [GUIs] · CPC title