Display apparatus and processing method for display apparatus with camera

Patent metadata

Field	Value
Publication number	US-12028617-B2
Application number	US-202218060210-A
Country	US
Kind code	B2
Filing date	Nov 30, 2022
Priority date	Jul 1, 2020
Publication date	Jul 2, 2024
Grant date	Jul 2, 2024

Abstract

Disclosed are a display apparatus and a processing method for the display apparatus with a camera. The display apparatus includes a camera, a sound collector and controller. The controller is configured for: starting shooting at least one image through the camera; in response to the at least one image not including a portrait of a user, starting obtaining a first test audio signal input from the user through the sound collector; in response to the first test audio signal, determining a target azimuth corresponding to the user; generating a rotation instruction for the camera according to the target azimuth of the user; sending the rotation instruction to the camera to adjust a shooting direction of the camera to the target azimuth.

First claim

What is claimed is:

1. A display apparatus, comprising:
a display, configured for presenting one or more images and one or more user interfaces, wherein the one or more images comprise images obtained from broadcast system or network;
an interface component, connected with a camera and a sound collector, wherein the camera is configured for capturing images and able to rotate a shooting angle, and the sound collector comprises a microphone array including a plurality of microphones and is configured for collecting one or more audio signals; and
a controller, connected with the display and the interface component, and configured for:
starting shooting at least one image through the camera;
in response to the at least one image not comprising a portrait of a user, starting obtaining a first test audio signal input from the user through the sound collector;
in response to the first test audio signal, determining a target azimuth corresponding to the user, wherein the target azimuth is calculated according to a time difference of the test audio signal by obtaining at least two test audio signals collected by at least two microphones among the microphone array;
generating a rotation instruction for the camera according to the target azimuth of the user;
sending the rotation instruction to the camera to adjust a shooting direction of the camera to the target azimuth; and
controlling the camera to shoot one or more reference images until a reference image comprises a portrait pattern of the user, stopping obtaining a subsequent test audio signal input following the first test audio signal from the user, and generating a tracking instruction according to a position of the portrait pattern in the reference image, wherein the position of the portrait pattern in the reference image is determined according to a skeleton line graph created based on one or more identified key points of the portrait of the user.

2. The display apparatus according to claim 1 , wherein the controller is further configured for:
detecting the portrait pattern in the reference image and determining a preset area in the reference image, wherein the preset area has a maximum allowable coordinate range based on a central position of the portrait of the user;
in response to the portrait pattern being within the preset area, maintaining the shooting direction of the camera; and
in response to the portrait pattern being not within the preset area, adjusting the shooting direction of the camera in response to the tracking instruction.

3. The display apparatus according to claim 2 , wherein the controller is further configured for:
identifying at least one key point of the portrait of the user in the reference image and creating the skeleton line graph based on the at least one identified key point; and
determining a portrait position according to the skeleton line graph, marking the portrait position, sending the tracking instruction to the camera while the user is moving, and tracking a user position by adjusting the shooting direction of the camera according to the portrait position.

4. The display apparatus according to claim 3 , wherein the controller is further configured for:
obtaining the reference image according to a preset frequency through the camera;
detecting the portrait position of the portrait pattern in the reference image;
in response to the portrait pattern being not within the preset area, generating the tracking instruction according to the portrait position of the portrait pattern, wherein the tracking instruction comprises a rotation direction of the camera and a rotation angle of the camera; and
sending the tracking instruction to the camera.

5. The display apparatus according to claim 2 , wherein the controller is further configured for:
in response to the reference image comprising a plurality of portrait patterns of users, seeking a first portrait pattern located in a central area of the reference image from the plurality of portrait patterns;
in response to the central area of the reference image comprising the first portrait pattern, marking the first portrait pattern as a target portrait position; and
in response to the central area of the reference image not comprising the first portrait pattern, marking a second portrait pattern occupying a maximum area among the plurality of portrait patterns as a target portrait position.

6. The display apparatus according to claim 1 , wherein the controller is further configured for:
obtaining an initial image through the camera;
identifying a portrait pattern of the user from the initial image;
in response to the initial image containing the portrait pattern of the user, sending the rotation instruction to the camera; and
in response to the initial image not containing the portrait pattern of the user, obtaining a subsequent test audio signal following the first test audio signal from the user for positioning the user.

7. The display apparatus according to claim 1 , wherein the controller is further configured for:
obtaining a plurality of skeleton line graphs from a plurality of reference images with portraits of the user;
identifying a move state of the user according the plurality of skeleton line graphs;
calculating move change trends according to the move state of the user; and
adjusting the shooting direction of the camera according to the move change trends.

8. The display apparatus according to claim 1 , wherein the controller is further configured for:
obtaining a sound signal through the sound collector;
extracting voiceprint information from the sound signal;
comparing the voiceprint information with a preset test voiceprint;
in response to the voiceprint information being the same as the preset test voiceprint, marking the sound signal as the test audio signal; and
in response to the voiceprint information being different from the preset test voiceprint, controlling the display to present a prompt interface for indicating voice input.

9. The display apparatus according to claim 1 , wherein the controller is further configured for:
obtaining the reference image and detecting a portrait position from the reference image;
comparing the portrait position with a preset area in the reference image;
in response to the portrait position being within the preset area, controlling the display to present an image shot by the camera in real time;
in response to the portrait position being not within the preset area, calculating a coordinate difference between the portrait position and a center of the preset area; and
generating the rotation instruction according to the coordinate difference and sending the rotation instruction to the camera.

10. The display apparatus according to claim 1 , wherein the controller is further configured for:
inquiring about a current work state of the camera;
in response to the current work state of the camera being a rotating state, waiting for the camera to finish rotating; and
in response to the current work state of the camera being a non-rotating state, obtaining a current shooting angle of the camera.

11. A processing method for a display apparatus with a camera, comprising:
starting shooting at least one image through the camera;
in response to the at least one image not comprising a portrait of a user, starting obtaining a first test audio signal input from the user through a sound collector of the display apparatus, wherein the sound collector comprises a microphone array including a plurality of microphones;
in response to the first test audio signal, determining a target azimuth corresponding to the user, wherein the target azimuth is calculated according to a time d

Assignees

• Hisense Visual Tech Co Ltd

Inventors

• Yang Luming
• Wang Dayong
• Wang Xusheng
• Cheng Jin
• Yu Wenqin
• Ma Le
• Ding Jiayi

Classifications

• H04N21/431

  Generation of visual interfaces {for content selection or interaction}; Content or additional data rendering · CPC title

• H04N21/422

  Input-only peripherals {, i.e. input devices connected to specially adapted client devices}, e.g. global positioning system [GPS] {(input devices also receiving signals from specially adapted client devices H04N21/4104)} · CPC title

• G10L17/06

  Decision making techniques; Pattern matching strategies · CPC title

• H04R2499/15

  Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops · CPC title

• H04R3/005 Primary

  for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407) · CPC title