Method for providing augmented reality content in vehicle, and wearable device and electronic device performing the same

Patent metadata

Field	Value
Publication number	US-12174394-B2
Application number	US-202218081120-A
Country	US
Kind code	B2
Filing date	Dec 14, 2022
Priority date	Dec 2, 2021
Publication date	Dec 24, 2024
Grant date	Dec 24, 2024

Abstract

A method of providing an augmented reality (AR) content in a vehicle, and/or a wearable AR device and an electronic device for performing a method. The wearable AR device may include a processor and a memory storing instructions to be executed by the processor, and when the instructions are executed by the processor, the processor is configured to: determine whether the wearable AR device is in a space of the vehicle based on at least one of information received from the vehicle or a value measured using at least one sensor of the wearable AR device; when it is determined that the wearable AR device is not in the space of the vehicle, output an AR content corresponding to a space around the wearable AR device based on the value measured using the at least one sensor of the wearable AR device; when it is determined that the wearable AR device is in the space of the vehicle, determine whether there are anchor devices of the vehicle capable of communicating with the wearable AR device; and when it is determined that there are the anchor devices capable of communication, output an AR content corresponding to a space of the vehicle around the wearable AR device by communicating with the anchor devices.

First claim

What is claimed is:

1. A wearable augmented reality (AR) device providing an AR content, the wearable AR device comprising:
at least one processor; and
memory for storing instructions that, when executed by the at least one processor individually or collectively, cause the wearable AR device to:
determine whether the wearable AR device is in a space of a vehicle based on at least one of information received from the vehicle or a value measured using at least one sensor of the wearable AR device;
determine whether there are anchor devices of the vehicle capable of communicating with the wearable AR device, in response to the wearable AR device is being in the space of the vehicle;
determine a position and a direction of the wearable AR device based on a positional relationship between the wearable AR device and the anchor devices, in response to there being the anchor devices; and
control to output an AR content corresponding to the position and the direction of the wearable AR device,
wherein, when executed by the at least one processor individually or collectively, the instructions cause the wearable AR device further to:
set reference points corresponding to positions of the anchor devices on first spatial map data corresponding to the space of the vehicle; and
determine the position and the direction of the wearable AR device on the first spatial map data based on the positional relationship between the wearable AR device and the anchor devices and the set reference points.

2. The wearable AR device of claim 1 , wherein, based on a determination that the wearable AR device is in the space of the vehicle, the instructions that, when executed by the at least one processor, cause the wearable AR device to:
determine a current position of the wearable AR device and a current direction the wearable AR device faces on the first spatial map data, via communication with the-anchor devices, and control to output an AR content corresponding to the current position and the current direction.

3. The wearable AR device of claim 2 , wherein, based on a determination that the wearable AR device is in the space of the vehicle, the instructions that, when executed by the at least one processor, cause the wearable AR device to:
generate second spatial map data corresponding to the space of the vehicle around the wearable AR device at a position of the wearable AR device in the vehicle;
based on third spatial map data previously generated for the space of the vehicle, retrieve the third spatial map data;
based on spatial information about the space of the vehicle, transform the spatial information into fourth spatial map data; and
generate the first spatial map data based on at least one of the second spatial map data, the third spatial map data, or the fourth spatial map data.

4. The wearable AR device of claim 2 , wherein the instructions that, when executed by the at least one processor, cause the wearable AR device to:
set reference points corresponding to positions of the anchor devices on the first spatial map data; and
determine the current position and the current direction on the first spatial map data based on a current distance and a current angle between the wearable AR device and the anchor devices and on the reference points.

5. The wearable AR device of claim 4 , wherein, based on the vehicle in a stationary state, the instructions that, when executed by the at least one processor, cause the wearable AR device to:
determine an initial distance and an initial angle between the wearable AR device and the anchor devices via communication with the anchor devices;
determine points corresponding to the positions of the anchor devices on the first spatial map data based on the value measured using the at least one sensor of the wearable AR device, the initial distance, and the initial angle; and
set the determined points as the reference points.

6. The wearable AR device of claim 5 , wherein the instructions that, when executed by the at least one processor, cause the wearable AR device to:
determine whether an inertia value measured through an inertial measurement unit (IMU), comprising measurement circuitry, of the wearable AR device has changed;
when it is determined that the inertia value has changed, determine whether the initial distance and the initial angle between the wearable AR device and the anchor devices have changed;
when it is determined that the initial distance and the initial angle have changed, determine the current distance and the current angle between the wearable AR device and the anchor devices; and
determine the current position and the current direction on the first spatial map data based on the current distance, the current angle, and the determined reference points.

7. The wearable AR device of claim 4 , wherein the instructions that, when executed by the at least one processor, cause the wearable AR device to:
receive a user input for determining points on the first spatial map data corresponding to the positions of the anchor devices;
determine the points based on the user input; and
set the determined points as the reference points.

8. The wearable AR device of claim 2 , wherein the instructions that, when executed by the at least one processor, cause the wearable AR device to:
based on no anchor devices of the vehicle capable of communicating with the wearable AR device, determine whether it is possible to receive an inertia value of the vehicle from at least one of the vehicle or an electronic device present in the space of the vehicle;
based on a determination that it is possible to receive the inertia value of the vehicle, correct an inertia value measured through an IMU of the wearable AR device based on the inertia value of the vehicle, the IMU comprising measurement circuitry;
determine an initial position of the wearable AR device and an initial direction the wearable AR device faces on the first spatial map data, based on an image acquired through a camera of the wearable AR device and the corrected inertia value; and
determine the current position and current direction on the first spatial map data, based on the initial position, the initial direction, the inertia value of the vehicle, and the inertia value of the wearable AR device.

9. The wearable AR device of claim 8 , wherein the instructions that, when executed by the at least one processor, cause the wearable AR device to:
determine whether the inertia value of the wearable AR device has changed;
based on a determination that the inertia value of the wearable AR device has changed, determine whether a difference between a variation in the inertia value of the vehicle and a variation in the inertia value of the wearable AR device is greater than or equal to a threshold value;
based on a determination that the difference between the variation in the inertia value of the vehicle and the variation in the inertia value of the wearable AR device is greater than or equal to the threshold value, determine a variation in an inertia value by a movement of the wearable AR device based on the difference between the variation in the inertia value of the vehicle and the variation in the inertia value of the wearable AR device; and
determine the current position and the current direction on the first spatial map data based on the variation in the inertia value by the movement of the wearable AR device.

10. The wearable AR device of claim 9 , wherein the instructions that, when executed by the at least one processor, cause the wearable AR device to:
determine a first variation in position and direction of the wearable AR device based on the variation in the inertia value by the movement of the weara

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kim Dohyoung
• Kim Jimin
• Lee Joayoung
• Lee Changsoo
• Lee Changyoul
• Han Hoon

Classifications

• G06F3/0487

  using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser · CPC title

• G06F3/04815

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F1/1694

  the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer · CPC title