Apparatus and method of providing service according to status of passenger in aircraft during autonomous flight

Patent metadata

Field	Value
Publication number	US-12347565-B2
Application number	US-202318138897-A
Country	US
Kind code	B2
Filing date	Apr 25, 2023
Priority date	Dec 12, 2022
Publication date	Jul 1, 2025
Grant date	Jul 1, 2025

Abstract

A service providing apparatus of providing a service according to a state of a passenger in aircraft during autonomous flight includes a processor; and a storage medium recording one or more programs configured to be executable by the processor, wherein the one or more programs include instructions for an input/output (I/O) module receiving biometric data and a captured image of a passenger in real time, a determination module determining a state of the passenger based on the biometric data and the captured image of a passenger, and a control module performing a preset process according to the state of the passenger, wherein the preset process includes: a first process for adjusting a flight path of the aircraft when the state of the passenger is an emergency state or a second process for relieving tension when the state of the passenger is a state of tension.

First claim

What is claimed is:

1. A service providing apparatus of providing a service according to a state of a passenger in aircraft during autonomous flight, the apparatus comprising:
a processor; and
a storage medium recording one or more programs configured to be executable by the processor,
wherein the one or more programs include instructions for:
an input/output (I/O) module receiving biometric data and a captured image of the passenger in real time;
a determination module determining the state of the passenger based on the biometric data and the captured image of the passenger; and
a control module performing a preset process according to the state of the passenger,
wherein the preset process includes:
a first process for adjusting a flight path of the aircraft when the state of the passenger is an emergency state or a second process for relieving tension when the state of the passenger is a state of tension,
wherein the biometric data includes at least one of a heart rate, a sweat output, a grip strength, a blood pressure, a body temperature, and a respiration rate of the passenger, and
wherein the determination module determines whether the state of the passenger is the emergency state based on a combination of classification result from the machine learning model to which the captured image of the passenger is input and comparison result between at least one of the heart rate, the sweat output, the grip strength, the blood pressure, and the respiration rate of the passenger with each preset reference value, and
a type of biometric data and a classification result used in determining the emergency state are different for each disease.

2. The service providing apparatus of claim 1 , wherein
the machine learning model includes:
a first machine learning model for classifying a facial expression of the passenger in the captured image of the passenger; and
a second machine learning model for classifying a body posture of the passenger in the captured image of the passenger.

3. The service providing apparatus of claim 2 , wherein the first machine learning model classifies the facial expression of the passenger as pain, fear, tension, disgust, stuffiness, discomfort, and others.

4. The service providing apparatus of claim 2 , wherein the second machine learning model classifies the body posture of the passenger as one of a posture with eyes closed, a posture with hands put on the head, a posture in which a face is lowered down forwardly and is not visible, and others.

5. The service providing apparatus of claim 1 , wherein
the determination module estimates the disease based on the combination of classification result from the machine learning model to which the captured image of the passenger is input and comparison result between at least one of the heart rate, the sweat output, the grip strength, the blood pressure, and the respiration rate of the passenger with each preset reference value, and
when there are multiple estimated diseases, the determination module considers a disease according to an underlying disease of the passenger or family history as a priority among the multiple estimated diseases.

6. The service providing apparatus of claim 2 , wherein
the determination module determines whether the state of the passenger is a state of tension based on a classification result from the machine learning model to which the captured image of the passenger is input or comparison result between at least one of the heart rate, the sweat output, the grip strength, the blood pressure, and the respiration rate of the passenger with each preset reference value, and
the classification result from the first machine learning model is tension.

7. The service providing apparatus of claim 1 , wherein
the first process is a process of controlling a communication module to transmit an ID of the aircraft, a flight path of the aircraft, current location information of the aircraft, information of the passenger, biometric data of the passenger, and the captured image of the passenger to a control device and to receive landing location information of a landing point of an adjacent hospital based on the current location information from the control device, and
controlling a driving module to land the aircraft at the landing point by adjusting the flight path according to the received landing location information.

8. The service providing apparatus of claim 1 , wherein
the first process is a process of controlling a communication module to receive a first aid process from a control device, and
controlling the I/O module to broadcast information on the emergency state, the first aid process, and a location of medical supplies.

9. The service providing apparatus of claim 1 , wherein the first process further includes a process of controlling a driving module to ascend or descend an altitude of the aircraft.

10. The service providing apparatus of claim 1 , wherein
the second process includes a process of controlling the I/O module to provide music or video for relieving the tension and
a process of controlling a convenience device for inducing a comfortable posture of the passenger.

11. A method of providing a service according to a state of a passenger in aircraft during autonomous flight, the method comprising:
a first operation in which an input/output (I/O) module receives biometric data and a captured image of the passenger in real time;
a second operation in which a determination module determines the state of the passenger based on the biometric data and the captured image of the passenger; and
a third operation in which a control module performs a preset process according to the state of the passenger,
wherein the preset process includes:
a first process for adjusting a flight path of the aircraft when the state of the passenger is an emergency state or a second process for relieving tension when the state of the passenger is a state of tension,
wherein the biometric data includes at least one of a heart rate, a sweat output, a grip strength, a blood pressure, a body temperature, and a respiration rate of the passenger, and
in the second operation,
determining whether the state of the passenger is an emergency state based on a combination of classification result from the machine learning model to which the captured image of the passenger is input and comparison result between at least one of the heart rate, the sweat output, the grip strength, the blood pressure, and the respiration rate of the passenger with each preset reference value, and
a type of biometric data and a classification result used in determining the emergency state are different for each disease.

12. The method of claim 11 , wherein
the machine learning model includes:
a first machine learning model for classifying a facial expression of the passenger in the captured image of the passenger; and
a second machine learning model for classifying a body posture of the passenger in the captured image of the passenger.

13. The method of claim 12 , wherein the first machine learning model classifies the facial expression of the passenger as pain, fear, tension, disgust, stuffiness, discomfort, and others.

14. The method of claim 12 , wherein the second machine learning model classifies the body posture of the passenger as one of a posture with eyes closed, a posture with hands put on the head, a posture in which a face is lowered down forwardly and is not visible, and others.

15. The method of claim 11 , wherein the second operation further includes estimating the disease based on a combination of classification result fro

Assignees

• Hyundai Motor Co Ltd
• Kia Corp

Inventors

• Kim Kye Yoon

Classifications

• G05D1/6546

  Emergency landing · CPC title

• G05D2109/20

  Aircraft, e.g. drones · CPC title

• G16H50/20

  for computer-aided diagnosis, e.g. based on medical expert systems · CPC title

• G06V10/00

  Arrangements for image or video recognition or understanding (character recognition in images or video G06V30/10) · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title