Mobile terminal and method for generating control command using marker attached to finger

Patent metadata

Field	Value
Publication number	US-9304598-B2
Application number	US-201414159496-A
Country	US
Kind code	B2
Filing date	Jan 21, 2014
Priority date	Sep 13, 2013
Publication date	Apr 5, 2016
Grant date	Apr 5, 2016

Abstract

A mobile terminal 200 generating a control command using a marker attached to a finger includes a camera 210 configured to receive a target image including a finger region, a fingertip region detection module 220 configured to detect a finger region in the target image using an image processing technique, and a fingertip region in the finger region, a marker region detection module 230 configured to detect a color marker region attached to the finger in the target image using a color detection technique, a control module 240 configured to operate the fingertip region detection module or the marker region detection module, based on an input from a user, and a coordinate calculation module 250 configured to calculate three-dimensional coordinate values using the fingertip region or the color marker region.

First claim

What is claimed is:

1. A method of generating a control command in a terminal using a marker attached to a finger, comprising:
receiving, by a terminal, a target image including a finger region to which a color marker is attached from a camera;
inputting, by the terminal, a command to use the finger region or a color marker region included in the target image;
detecting, by the terminal, a fingertip region in the target region using an image processing technique when the command to use the finger region is input, or detecting the color marker region when the command to use the color marker region is input; and
calculating, by the terminal, three-dimensional coordinate values using the fingertip region or the color marker region.

2. The method of claim 1 , wherein the command to use the color marker region is a touch input to the color marker region output on a touch display device of the terminal.

3. The method of claim 1 , wherein in the detecting of the color marker region, the color marker region is detected in the target image using a CAMshift algorithm, and the detecting of the color marker region comprises:
converting an RGB image including the color marker region to an HSV image;
generating a base image using maximum frequency characteristics of a histogram for an H channel of the HSV image;
generating a mask image by summing results in which threshold values for both of an S channel and a V channel of the HSV image are derived; and
generating a final image by combining the base image and the mask image.

4. The method of claim 1 , wherein the calculating of the coordinate values includes, when using the finger region, calculating an X coordinate value and a Y coordinate value using a reference point of the fingertip region, and calculating a Z coordinate value using at least one of a width, a height, and an area of the fingertip region.

5. The method of claim 1 , wherein the calculating of the coordinate values includes, when using the color marker region, calculating a Z coordinate value using at least one of a width, a diameter, a height, and an area of the color marker region, and calculating an X coordinate value and a Y coordinate value by calculating an actual distance which a center pixel of the color marker region travels using the Z coordinate value.

6. The method of claim 1 , wherein, when calculating the coordinate values using the color marker region, the terminal calculates a Z coordinate value (FingerPoint.Z) of the three-dimensional coordinate values based on a width or diameter of the color marker region, using the following equation:

FingerPoint
·
Z

=

ColorPoint
⁡

(
cm
)

×

PreviewWidth
⁡

(
pixel
)

ColorPoint
·

Region
⁡

(
pixel
)

×

tan
⁡

(

WidthFOV
/
2

)

×
2

(wherein ColorPoint is an actual width or diameter of the color marker region, PreviewWidth is a pixel value for a width of the whole target image input to a camera, ColorPoint.Region is a pixel value for the width of the color marker region in the target image, and WidthFOV is a horizontal viewing angle of the camera).

7. The method of claim 6 , wherein, when calculating the coordinate values, the terminal calculates an X coordinate value (FingerPoint.X) of the three-dimensional coordinate values based on the width of the color marker region, using the following equation:
FingerPoint. X =ColorPoint. x (pixel)×DistancePerPixel(cm)

(wherein ColorPoint.x is an x-pixel coordinate value for a center pixel of the color marker region in the target image,

DistancePerPixel
=

FovWidthLength
⁡

(
cm
)

PreviewWidth
⁡

(
pixel
)

,

⁢
and

⁢

⁢

FovWidthLength
=

FingerPoint
·

Z
⁡

(
cm
)

×
tan
⁢

WidthFOV
2

×
2

)

.

8. The method of claim 6 , wherein, when calculating the coordinate values, the terminal calculates a Y coordinate value (FingerPoint.Y) of the three-dimensional coordinate values based on the width of the color marker region, using the following equation:
FingerPoint. Y =ColorPoint. y (pixel)×DistancePerPixel(cm)

(wherein ColorPoint.y is a y-pixel coordinate value for the center pixel of the color marker region in the target image,

DistancePerPixel
=

FovHeightLength
⁡

(
cm
)

PreviewHeight
⁡

(
pixel

Assignees

• Univ Sungkyunkwan Res & Bus
• Res And Business Foundation Sungkyunkwan University

Inventors

• Hong Kwang-Seok
• Oh Byung-Hun
• Chang Kyung-Won

Classifications

• G06V10/255

  Detecting or recognising potential candidate objects based on visual cues, e.g. shapes · CPC title

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06V10/56

  relating to colour · CPC title

• G06F18/21375

  involving differential geometry, e.g. embedding of pattern manifold · CPC title

• G06K9/6252

  Physics · mapped topic