Apparatus and method for controlling interior of vehicle

Patent metadata

Field	Value
Publication number	US-11701984-B2
Application number	US-202016801574-A
Country	US
Kind code	B2
Filing date	Feb 26, 2020
Priority date	Nov 26, 2019
Publication date	Jul 18, 2023
Grant date	Jul 18, 2023

Abstract

An apparatus and a method for controlling an interior of a vehicle are disclosed. The apparatus for controlling the interior of the vehicle may include an interface configured to receive, at predetermined cycles, information on at least one of video of an inside of the vehicle collected by a camera or sound of the inside of the vehicle collected from a microphone, and a processor configured to determine a relationship between passengers in the vehicle based on the information, and control the interior of the vehicle based on the relationship between the passengers. When determining the relationship between passengers, a passenger relationship estimation algorithm used may be a neural network model generated through machine learning, and may be stored in a memory in the apparatus for controlling the interior of the vehicle, or may be provided via a server in an artificial intelligence environment through a 5G network.

First claim

What is claimed is:

1. An apparatus for controlling an interior of a vehicle, comprising:
an interface configured to receive, at predetermined cycles, information on at least one of video of an inside of the vehicle collected by a camera installed in the vehicle or sound of the inside of the vehicle collected from a microphone installed in the vehicle; and
a processor configured to determine a relationship between passengers in the vehicle based on the information received at the predetermined cycles, and control the interior of the vehicle based on the relationship between the passengers,
wherein the processor is configured to determine the relationship between the passengers by applying a passenger relationship estimation algorithm to the information on at least one of the video of the inside of the vehicle or the sound of the inside of the vehicle, and
wherein the passenger relationship estimation algorithm is a neural network model trained to estimate a relationship between a plurality of people from information on at least one of motion or conversation sound between the people, based on at least one of appellations between the people, physical contact, gaze, expression, tone, whether an object is shared, or sitting posture.

2. The apparatus of claim 1 , wherein the processor is configured to:
set a certain range as a personal space of a passenger seated in a seat, based on an initially designated position of the seat;
in response to the relationship between the passengers being determined as a lower relationship having the lowest level of intimacy based on the information, maintain the personal space of each of the passengers having the lower relationship, and in response to the relationship between the passengers being determined as an upper relationship having the highest level of intimacy based on the information, combine the personal spaces of each of the passengers having the upper relationship to change the personal spaces of each of the passengers having the upper relationship into a first shared space; and
control the interior of the vehicle by allowing each seat and components attached to the seat to move only within the personal space and the first shared space.

3. The apparatus of claim 2 , wherein the processor is configured to:
in response to the relationship between the passengers being determined as an intermediate relationship other than the lower relationship and the upper relationship, determine a second shared space by overlapping a portion of the personal space of each of the passengers having the intermediate relationship; and
extend a movement allowance range of seats of the passengers having the intermediate relationship and the components attached to the seats, from the personal space of the passenger to the second shared space.

4. The apparatus of claim 3 , wherein the processor is configured to:
extract situation information from the information on the at least one of the video of the inside of the vehicle or the sound of the inside of the vehicle, which is collected at the predetermined cycles, and based on the situation information, determine a disposition or a state of each of the passengers whose relationship has been determined; and
adjust any one of the personal space, the first shared space, or the second shared space based on the determination result, and
wherein the situation information comprises at least one of conversation sound intensity, conversation amount, conversation words, physical contact, gaze, expression, tone, whether an object is shared, or sitting posture.

5. The apparatus of claim 4 , wherein the processor is configured to, in response to checking that the disposition of the passenger is introverted or the state of the passenger is that the passenger is working or sleeping, adjust the second shared space to be as small as a predetermined space.

6. The apparatus of claim 2 , wherein the processor is configured to provide video content and audio related to the video content in the personal space set for each passenger in the vehicle individually, such that the passengers use different video content.

7. The apparatus of claim 6 , wherein the processor is configured to:
in response to the relationship between the passengers being determined as the upper relationship having the highest level of intimacy, or in response to the video content provided in the personal space of each of the passengers being determined to be the same regardless of the relationship between the passengers, combine the personal spaces of each of the passengers to generate a shared space; and
extend and provide one video content and audio related to the one video content in the shared space, such that each of the passengers uses the extended video content and the audio together.

8. The apparatus of claim 1 , wherein the processor is configured to:
set a certain range as a personal space of a passenger seated in a seat, based on the initially designated position of the seat;
check whether there is a risk of theft of any item, in response to determining that a second passenger other than a first passenger who is the owner of the item is approaching within a predetermined distance of the item or in response to determining that the item is outside of the personal space of the first passenger, based on the information on the at least one of the video of the inside of the vehicle or the sound of the inside of the vehicle, which is collected at the predetermined cycles; and
in response to checking that there is a risk of theft of the item, provide a risk warning message for the item to a mobile terminal corresponding to the first passenger.

9. The apparatus of claim 8 , wherein the processor is configured to check whether there is a risk of theft of the item based on the relationship between the first and second passengers and a predetermined item allowance criterion for each relationship.

10. The apparatus of claim 1 , wherein the processor is configured to:
determine a movement allowance range for at least one of the position of the seat of the passenger, rotation of the seat, adjustment of the backrest of the seat, or adjustment of the armrest of the seat, based on the determined relationship between the passengers; and
control the interior of the vehicle by adjusting at least one of the seat, the backrest of the seat, or the armrest of the seat based on the movement allowance range.

11. The apparatus of claim 1 , wherein the processor is configured to determine a degree of change in the seat arrangement of the passengers in the vehicle based on the information on the at least one of the video of the inside of the vehicle or the sound of the inside of the vehicle, which is collected at the predetermined cycles, and re-determine the relationship between the passengers in the vehicle according to the degree of change in the seat arrangement.

12. The apparatus of claim 1 , wherein the processor is configured to check a reservation time and seat number of the passengers for the vehicle in a subscriber list, and based on the check result, determine the relationship between the passengers in the vehicle.

13. A method for controlling an interior of a vehicle, comprising:
receiving, at predetermined cycles, information on at least one of video of an inside of the vehicle collected by a camera installed in the vehicle or sound of the inside of the vehicle collected from a microphone installed in the vehicle;
determining a relationship between passengers in the vehicle based on the information received at the predetermined cycles, and controlling the interior of the vehicle based on the relationship between the passengers; and
de

Assignees

• Lg Electronics Inc

Inventors

• Lee Chul Hee
• Jung Jun Young
• Jeong Sangkyeong
• Song Kibong
• Kim Hyunkyu

Classifications

• B60N2/14 Primary

  rotatable, e.g. to permit easy access (B60N2/10 {, B60N2/2869} take precedence) · CPC title

• B60N2/0273

  taking into account user data, e.g. knee height or physical state · CPC title

• B60N2/0244 Primary

  with logic circuits · CPC title

• B60N2/763

  adjustable · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title