Display apparatus for gesture recognition and operating method thereof

Patent metadata

Field	Value
Publication number	US-12561006-B2
Application number	US-202318242890-A
Country	US
Kind code	B2
Filing date	Sep 6, 2023
Priority date	Aug 25, 2022
Publication date	Feb 24, 2026
Grant date	Feb 24, 2026

Abstract

A display apparatus includes: a display; a communicator, a memory storing one or more instructions; and at least one processor configured to execute the one or more instructions. The at least one processor is configured to execute the one or more instructions to: identify a transmission area and a gesture recognition area in an image area of a first image captured by a camera for a video call; control the communicator to transmit a second image corresponding to the transmission area to a counterpart display apparatus participating in the video call, in order to provide a video call service; perform gesture recognition in a third image corresponding to the gesture recognition area; and perform a function corresponding to a recognized gesture from the gesture recognition in the third image.

First claim

The invention claimed is:

1 . A display apparatus comprising:
a display; a communicator; a memory storing one or more instructions; and at least one processor configured to execute the one or more instructions, wherein the at least one processor is configured to execute the one or more instructions to:
divide a first image captured by a camera for a video call into a transmission area and a non-transmission area, the non-transmission area comprising a gesture recognition area, wherein the transmission area extends from an upper end of the first image and has a height corresponding to a specified percentage of a height of the first image, and wherein the non-transmission area extends from a lower end of the first image and has a height corresponding to a specified percentage of the height of the first image;
control the communicator to transmit a second image including the transmission area and excluding the non-transmission area to a counterpart display apparatus participating in the video call, in order to provide a video call service;
perform gesture recognition in a third image corresponding to the gesture recognition area; and
perform a function corresponding to a recognized gesture from the gesture recognition in the third image.

2 . The display apparatus of claim 1 , wherein the at least one processor is configured to execute the one or more instructions to identify the gesture recognition area by detecting the gesture recognition area in a specified position of the image area.

3 . The display apparatus of claim 1 , wherein the at least one processor is configured to execute the one or more instructions to identify the gesture recognition area by:
detecting a hand of a user in the image area; and determining the gesture recognition area based on an area in which the hand is detected.

4 . The display apparatus of claim 1 , wherein the at least one processor is further configured to execute the one or more instructions to:
provide a user interface on the display to adjust the gesture recognition area; and adjust the gesture recognition area based on a user input received through the user interface.

5 . The display apparatus of claim 1 ,
wherein the at least one processor is configured to execute the one or more instructions to perform the gesture recognition in the third image by performing the gesture recognition according to a specified event while the video call service is being provided, and wherein the specified event includes at least one of (i) a user input of selecting the gesture recognition function or (ii) a detection of a motion exceeding a threshold value in the image area.

6 . The display apparatus of claim 1 , wherein the at least one processor is further configured to execute the one or more instructions to, based on a zoom-in function being executed while the gesture recognition function is provided while the video call service is being provided, identify a category of the zoom-in function.

7 . The display apparatus of claim 6 , wherein the at least one processor is further configured to execute the one or more instructions to:
based on the category of the zoom-in function being identified as an optical zoom-in, output a message to the display indicating that the gesture recognition function has been terminated, and based on the category of the zoom-in function being identified as digital zoom-in,
expand the second image corresponding to the transmission area and control the communicator to transmit, to the counterpart display apparatus participating in the video call, expanded second image,
perform the gesture recognition in the third image corresponding to the gesture recognition area, and
perform the function corresponding to the recognized gesture from the gesture recognition in the third image.

8 . The display apparatus of claim 1 ,
wherein the camera includes a non-zoom camera and a zoom camera, and wherein the at least one processor is further configured to execute the one or more instructions to, based on a zoom-in function being executed while the gesture recognition function is provided while the video call service is being provided, switch the non-zoom camera capturing the first image for the video call to the zoom camera for capturing an expanded first image.

9 . The display apparatus of claim 1 , wherein the at least one processor is further configured to execute the one or more instructions to, provide a gesture guide that provides one or more gestures and a function corresponding to each of the one or more gestures, in order to guide the user with available gestures.

10 . An operating method of a display apparatus, the operating method comprising:
dividing an image area of a first image captured by a camera for a video call into a transmission area and a non-transmission area, the non-transmission area comprising a gesture recognition area, wherein the transmission area extends from an upper end of the first image and has a height corresponding to a specified percentage of a height of the first image, and wherein the non-transmission area extends from a lower end of the first image and has a height corresponding to a specified percentage of the height of the first image; transmitting a second image including the transmission area and excluding the non-transmission area to a counterpart display apparatus participating in the video call, in order to provide a video call service; performing gesture recognition in a third image corresponding to the gesture recognition area; and performing a function corresponding to a recognized gesture from the gesture recognition in the third image.

11 . The operating method of claim 10 , wherein the identifying the gesture recognition area comprises identifying the gesture recognition area in a specified position of the image area.

12 . The operating method of claim 10 , wherein the identifying the gesture recognition area comprises:
detecting a hand of a user in the image area; and determining the gesture recognition area based on an area in which the hand is detected.

13 . The operating method of claim 10 , further comprising:
providing a user interface on a display to adjust the gesture recognition area; and adjusting the gesture recognition area based on a user input received through the user.

14 . The operating method of claim 10 ,
wherein the performing the gesture recognition in the third image comprises performing the gesture recognition according to a specified event while the video call service is being provided, and wherein the specified event includes at least one of (i) a user input of selecting the gesture recognition function or (ii) a detection of a motion exceeding a threshold value in the image area.

15 . The operating method of claim 10 , further comprising, based on a zoom-in function being executed while the gesture recognition function is provided while the video call service is being provided, identify a category of the zoom-in function.

16 . The operating method of claim 15 , further comprising,
based on the category of the zoom-in function being identified as an optical zoom-in, outputting a message to the display indicating that the gesture recognition function has been terminated, and based on the category of the zoom-in function being identified as digital zoom-in,
expanding the second image corresponding to the transmission area and transmitting, to the counterpart display apparatus participating in the video call, expanded second image,
performing the gesture recognition in the third image corresponding to the gesture

Assignees

• Samsung Electronics Co Ltd

Inventors

• Choi Kilsoo
• Kim Sehyun
• Park Hyeeun
• Shin Yongwoo
• Lee Jongin

Classifications

• H04N23/69 Primary

  Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming · CPC title

• G06V40/28

  Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113) · CPC title

• G06V40/10

  Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands · CPC title

• H04N23/611

  where the recognised objects include parts of the human body · CPC title

• H04N7/147

  Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q) · CPC title