XR system and method for controlling the same

Patent metadata

Field	Value
Publication number	US-11734744-B2
Application number	US-201916554424-A
Country	US
Kind code	B2
Filing date	Aug 28, 2019
Priority date	Aug 23, 2019
Publication date	Aug 22, 2023
Grant date	Aug 22, 2023

Abstract

The present invention relates to an XR system and its control method, and more specifically, may be applied to 5G communication technology field, robot technology field, autonomous driving technology field and AI technology field. The XR system includes an XR device and a storage case that stores the XR device. The control method of the XR system includes recording and learning user experiences and providing a mate service to the user based on an image captured by a camera unit of the storage case and user information including the learned user experience.

First claim

What is claimed is:

1. An extended reality (XR) system comprising:
an XR device; and
a storage case for storing the XR device therein,
wherein the storage case includes:
a first power supply having a battery to supply power;
a camera for capturing at least one image;
a first display for displaying;
a first communication device for communicating with at least the XR device or an external server; and
a first controller for controlling components of the storage case,

wherein the XR device includes:
a second power supply chargeable with the power provided from the first power supply of the storage case;
a second display for displaying;
a second communication device for communicating with at least the storage case or an external server;
a learning processor for recording and learning user experiences; and
a second controller for controlling components of the XR device, and

wherein at least one of the first controller or the second controller controls corresponding components to provide a service to a user based on the at least one image captured by the camera of the storage case and user information including the learned user experiences.

2. The XR system of claim 1 , wherein when the user cooks, cooking materials prepared by the user and a cooking process of the user are captured by the camera of the storage case,
wherein at least one of the first controller or the second controller controls corresponding components to compare and analyze the captured cooking material and process images, a cooking recipe, and learned user experiences to provide a service to guide the cooking process to the user, and
wherein when the user's cooking is completed, the captured cooking materials, the captured cooking process, and guide contents are recorded and are learned as the user experiences by the learning processor.

3. The XR system of claim 1 , wherein when the user plays sports, a user pose for the sports is captured by the camera of the storage case,
wherein at least one of the first controller or the second controller controls corresponding components to compare and analyze the captured pose image, a reference pose for the sports, and learned user experiences to provide a service to coach the user's pose, and
wherein the captured user pose and coaching contents are recorded and are learned as the user experiences by the learning processor.

4. The XR system of claim 1 , wherein when the user is moving, at least one of the first controller or the second controller controls corresponding components to provide a service in which information about surrounding locations around the user along with moving directions and a user product order guidance are provided to the user based on learned user experiences and user schedule information, and a product selected by the user is ordered via communication with an external server, and
wherein when the user order is completed and the user receives the ordered product and then arrives at a final destination, the user schedule information, the surrounding locations information, the order information and the moving directions information are recorded and learned as the user experiences by the learning processor.

5. The XR system of claim 1 , wherein at least one of the first controller or the second controller controls corresponding components to provide a service for guiding the user's fashion based on a user image captured by the camera of the storage case, user schedule information, and the learned user experiences, and
wherein the captured user image, the user schedule information, and, the fashion information selected by the user are recorded and learned as the user experiences by the learning processor.

6. The XR system of claim 1 , wherein at least one of the first controller or the second controller controls corresponding components to provide a service in which characters of a first language captured by the camera of the storage case are converted to characters of a second language different from the first language and selected by the user and then the converted characters are displayed on the first display of the storage case.

7. The XR system of claim 1 , wherein the storage case further includes a recording device having a recording function,
wherein when multiple people attend a meeting, the camera and the recording device of the storage case captures the meeting participants and records meeting contents, respectively, and
wherein when the meeting ends, at least one of the first controller or the second controller controls corresponding components to provide a service in which the recorded contents are classified and the meeting contents are organized for each of attendees based on the captured image and the recorded contents to create an attendee-specific meeting record.

8. The XR system of claim 1 , wherein the storage case further includes a recording device having a recording function,
wherein when a plurality of people attend a seminar and present presentations, the camera and the recording device of the storage case captures and records presenters and presentation contents, respectively, and
wherein when the presentation ends, at least one of the first controller or the second controller controls corresponding components to provide a service in which the presentation contents are organized for each of the presenters based on the captured image and the recorded contents to create a presenter-specific presentation record.

9. The XR system of claim 1 , wherein the storage case further includes a recording device having a recording function,
wherein when the user is practicing presentation, appearance and contents about the user's presentation practice are captured and recorded by the camera and the recording of the storage case, respectively,
wherein at least one of the first controller or the second controller controls corresponding components to compare and analyze the captured image and recorded contents, reference presentation information, and learned user experiences and to provide a service to guide the user's presentation preparation based on the comparing and analyzing results, and
wherein the captured user image and presentation preparation guide contents are recorded and learned as the user experiences by the learning processor.

10. The XR system of claim 1 , wherein the storage case further includes a recording device having a recording function,
wherein when the user is listening to a lecture, a lecturing appearance of a teacher and lecture contents are captured and recorded by the camera and the recording device of the storage case, respectively, and
wherein at an end of the lecture, at least one of the first controller or the second controller controls corresponding components to provide a service in which the lecture contents of the teacher are organized based on the captured image and recorded contents to create a teacher-specific lecture record.

Assignees

• Lg Electronics Inc

Inventors

• Lee Haklim
• Kim Samyoup
• Shin Jiyong
• Im Sanghyuk
• Han Jongbeom

Classifications

• G06Q30/0631 Primary

  Recommending goods or services · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F40/58

  Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation · CPC title

• G06V20/20

  in augmented reality scenes · CPC title