Method of recognizing stairs in three dimensional data image

Patent metadata

Field	Value
Publication number	US-9552640-B2
Application number	US-201213669960-A
Country	US
Kind code	B2
Filing date	Nov 6, 2012
Priority date	Nov 23, 2011
Publication date	Jan 24, 2017
Grant date	Jan 24, 2017

Abstract

A method of recognizing stairs in a 3D data image includes an image acquirer that acquires a 3D data image of a space in which stairs are located. An image processor calculates a riser height between two consecutive treads of the stairs in the 3D data image, identifies points located between the two consecutive treads according to the calculated riser height, and detects a riser located between the two consecutive treads through the points located between the two consecutive treads. Then, the image processor calculates a tread depth between two consecutive risers of the stairs in the 3D data image, identifies points located between the two consecutive risers according to the calculated tread depth, and detects a tread located between the two consecutive risers through the points located between the two consecutive risers.

First claim

What is claimed is:

1. A method of recognizing stairs in a 3D data image, the method comprising:
acquiring, by an image sensor, a 3D data image of a space in which stairs are located;
calculating, by an image processor, a riser height between two consecutive treads of the stairs in the 3D data image, identifying points located between the two consecutive treads according to the calculated riser height, and detecting a riser located between the two consecutive treads through the points located between the two consecutive treads by limiting an equation of a first tread of the stairs using parallelism of a ground plane and the first tread and estimating an equation of the first tread through points located at a range of riser heights from the ground plane;
calculating, by the image processor, a tread depth between two consecutive risers of the stairs in the 3D data image, identifying points located between the two consecutive risers according to the calculated tread depth, and detecting a tread located between the two consecutive risers through the points located between the two consecutive risers; and
applying, by the image processor, the detected riser and the detected tread to a path for traveling the stairs.

2. The method according to claim 1 , further comprising detecting the ground plane on which the stairs are located in the 3D data image using a Random Sample Consensus (RANSAC) algorithm.

3. The method according to claim 2 , wherein detecting the ground plane comprises calculating an equation of the ground plane according to the following expression:
ax+by+cz+d= 0,
where a, b, and c are components of a normal vector of the ground plane and d is a constant indicating a shortest distance between an origin and the ground plane.

4. The method according to claim 2 , further comprising labeling the 3D data image to separate a stairs area corresponding to the stairs from the 3D data image, determining a start of the stairs from a contact line between the ground plane and the separated stairs area, and starting stair recognition.

5. The method according to claim 2 , wherein detecting the riser comprises limiting the equation of the first tread according to the following expression:
ax+by+cz+d′= 0,
where a, b, and c are components of a normal vector of the first tread, the components of the normal vector of the first tread are equal to components of a normal vector of the ground plane, and d′ is a variable indicating a shortest distance between an origin and the first tread.

6. The method according to claim 2 , wherein detecting the riser comprises comparing the equation of the ground plane and the estimated equation of the first tread and calculating a riser height between the ground plane and the estimated first tread.

7. The method according to claim 6 , wherein detecting the riser comprises limiting an equation of a first riser of the stairs using perpendicularity of the first riser to the ground plane and the estimated first tread and calculating an equation of the first riser through points located between the ground plane and the estimated first tread.

8. The method according to claim 7 , wherein detecting the riser comprises limiting the equation of the first riser according to the following expression:
nx+my+lz+k= 0,
where n, m, and l denote components of a normal vector of the first riser, the inner product of the normal vector of the first riser and the normal vectors of the ground plane and the first tread is 0, and k is a variable indicating a shortest distance between the first riser and the origin.

9. The method according to claim 1 , wherein detecting the riser comprises limiting an equation of an upper tread among two consecutive upper and lower treads of the stairs using parallelism of the upper tread and the lower tread and estimating an equation of the upper tread through points located at a range of riser heights from the lower tread.

10. The method according to claim 9 , wherein detecting the riser comprises limiting the equation of the upper tread according to the following expression:
ax+by+cz+d′= 0,
where a, b, and c are components of a normal vector of the upper tread, the components of the normal vector of the upper tread are equal to components of a normal vector of the lower tread, and d′ is a variable indicating a shortest distance between an origin and the upper tread.

11. The method according to claim 9 , wherein detecting the riser comprises comparing the equation of the lower tread and the estimated equation of the upper tread and calculating a riser height between the lower tread and the estimated upper tread.

12. The method according to claim 11 , wherein detecting the riser comprises limiting an equation of the riser using parallelism of the riser and an immediately lower riser and calculating an equation of the riser through points located between the lower tread and the estimated upper tread.

13. The method according to claim 12 , wherein detecting the riser comprises limiting the equation of the riser according to the following expression:
nx+my+lz+k= 0,
where n, m, and l denote components of a normal vector of the riser, the components of a normal vector of the riser are equal to components of a normal vector of the immediately lower riser, and k is a variable indicating a shortest distance between the riser and the origin.

14. The method according to claim 9 , wherein detecting the tread comprises limiting the equation of the upper riser according to the following expression:
nx+my+lz+k′= 0,
where n, m, and l are components of a normal vector of the upper riser, the components of the normal vector of the upper riser are equal to components of a normal vector of the lower riser, and k′ is a variable indicating a shortest distance between an origin and the upper riser.

15. The method according to claim 1 , further comprising determining whether or not an end of the stairs has been reached in the 3D data image, terminating stair recognition upon determining that the end of the stairs has been reached, and detecting a next riser and a next tread of the stairs upon determining that the end of the stairs has not been reached.

16. The method according to claim 15 , wherein determining whether or not the end of the stairs has been reached comprises determining that the end of the stairs has been reached when a tread depth between the two consecutive risers is greater than a threshold.

17. A non-transitory computer-readable recording medium storing a program to implement the method of claim 1 .

18. A method of recognizing stairs in a 3D data image, the method comprising:
acquiring, by an image sensor, a 3D data image of a space in which stairs are located;
calculating, by an image processor, a riser height between two consecutive treads of the stairs in the 3D data image, identifying points located between the two consecutive treads according to the calculated riser height, and detecting a riser located between the two consecutive treads through the points located between the two consecutive treads;
calculating, by the image processor, a tread depth between two consecutive risers of the stairs in the 3D data image, identifying points located between the two consecutive risers according to the calculated tread depth, and detecting a tread located between the two consecutive risers through the points located between the two consecutive risers by limiting an equation of an upper riser among two consecutive upper and lower risers of the stairs using parallelism of the upper riser and the lower

Assignees

• Samsung Electronics Co Ltd

Inventors

• Yoon Suk June
• Hwang Hyo Seok
• Ahn Sung Hwan
• Hyung Seong Yong
• Yeo Hyung Sok

Classifications

• G06T2207/20076

  Probabilistic image processing · CPC title

• G06T7/70 Primary

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T7/004 Primary

  Physics · mapped topic