Apparatus and method for detecting falling object

Patent metadata

Field	Value
Publication number	US-11062157-B2
Application number	US-201916575320-A
Country	US
Kind code	B2
Filing date	Sep 18, 2019
Priority date	Aug 20, 2019
Publication date	Jul 13, 2021
Grant date	Jul 13, 2021

Abstract

Disclosed are an apparatus and a method for detecting a fallen object which adjust a passenger's seat to easily pick up the fallen object in the vehicle. A fallen object detecting apparatus according to one embodiment of the present disclosure is an apparatus for detecting a fallen object in a vehicle which includes a camera configured to generate at least one image of an inside of the vehicle; an image identifier configured to identify at least one passenger and at least one object from the image, and to determine a location of the fallen object in response to a falling of the object in the vehicle; and a controller configured to provide the determined location of the fallen object via at least one component located in the vehicle and adjust a seat of the passenger based on the location of the fallen object and a condition of the passenger.

First claim

What is claimed is:

1. An apparatus for detecting a fallen object in a vehicle, the apparatus comprising:
a camera configured to generate at least one image of an inside of the vehicle;
an image identifier configured to identify at least one passenger and at least one object from the image, and to determine a location of the fallen object in response to a falling of the object in the vehicle; and
a controller configured to provide the determined location of the fallen object via at least one component located in the vehicle and adjust a seat of the passenger based on the location of the fallen object and a condition of the passenger.

2. The fallen object detecting apparatus according to claim 1 , wherein the controller is configured to set a range in which the passenger is capable of picking up the object based on the physical condition of the passenger including at least one of arm length, leg length, or hand length, and in response to the location of the object being out of the set range, adjust at least one of position, height, or angle of the seat of the passenger.

3. The fallen object detecting apparatus according to claim 1 , wherein the image identifier is configured to track a movement of the object from the image generated periodically by the camera and determine, as a tracking result, that the object has fallen based on the object dropping by a predetermined distance or more or dropping to a floor of the vehicle.

4. The fallen object detecting apparatus according to claim 1 , wherein the controller is configured to restrict an adjusting range of the seat of the passenger within a range in which the object does not become damaged, based on the location of the fallen object.

5. The fallen object detecting apparatus according to claim 1 , wherein the image identifier is configured to track the movement of the object from the image generated periodically by the camera, and
wherein in response to the tracked object being predicted to fall based on at least one of movement or shape of the tracked object, the controller is configured to suggest moving the object to a safe area, where it is determined that the tracked object will not fall, before the tracked object falls.

6. The fallen object detecting apparatus according to claim 1 , wherein the controller is configured to suggest picking up the fallen object based on the vehicle stopping or being in a sector where the vehicle drives at a predetermined speed or lower, or
the controller is configured to suggest immediately picking up the fallen object based on the vehicle being expected to turn sharply or drive on an irregular road within a predetermined time or based on the vehicle being expected to rapidly decelerate or accelerate, based on navigation information of the vehicle.

7. The fallen object detecting apparatus according to claim 1 , wherein
based on the fallen object being a communication device capable of being connected to the component in the vehicle via short range wireless communication, the controller is
configured to determine whether the object is connected to the component in the vehicle, and as a determination result, in response to the object being connected to the component, temporarily stop providing services of the fallen object and the component, and
in response to the object being picked up by the passenger, configured to control the object and the component to continuously provide the service.

8. The fallen object detecting apparatus according to claim 1 , wherein the image identifier is configured to identify whether the passenger picks up the fallen object from the image periodically generated by the camera, and
wherein the controller is configured to restrict an adjusting of the seat and suggest picking up the fallen object in response to a command for adjusting the seat to a range in which the fallen object becomes damaged being input in a state where it is identified that the fallen object is not picked up by the passenger.

9. The fallen object detecting apparatus according to claim 1 , wherein in response to the location of the fallen object being out of the range in which the passenger is capable of picking up the object, the controller is configured to move the fallen object to a pick up range by using a tool installed in the vehicle or change a driving direction or speed of the vehicle to move the fallen object to the pick up range.

10. The fallen object detecting apparatus according to claim 1 , wherein the image identifier is configured to determine, from the image of the fallen object, whether it is necessary to pick up the fallen object by applying a deep neural network model trained for determining whether the object is necessary for the passenger, and
the controller is configured to restrict the providing of the location of the object and the adjusting of the seat of the passenger based on a determination that it is not necessary to pick up the fallen object.

11. The fallen object detecting apparatus according to claim 1 , further comprising:
a display configured to display a location of the fallen object.

12. The fallen object detecting apparatus according to claim 10 , wherein based on the determination that it is not necessary to pick up the fallen object, the controller is configured to remove the object which is not necessary to pick up using a tool in the vehicle.

13. A method for detecting a fallen object in a vehicle, the method comprising:
generating at least one image of an inside of the vehicle;
identifying at least one passenger and at least one object from the image, and determining a location of the fallen object in response to a falling of the object in the vehicle; and
providing the location of the fallen object via at least one component in the vehicle and adjusting a seat of the passenger based on the location of the fallen object and a condition of the passenger.

14. The fallen object detecting method according to claim 13 , wherein the adjusting of the seat of the passenger comprises:
setting a range in which the passenger is capable of picking up the object based on the physical condition of the passenger including at least one of arm length, leg length, or hand length of the passenger, and in response to the location of the object being out of the set range, adjusting at least one of position, height, or angle of the seat of the passenger.

15. The fallen object detecting method according to claim 13 , further comprising:
tracking a movement of the object from the periodically generated image and determining, as a tracking result, that the object has fallen based on the object dropping by a predetermined distance or more or dropping to a floor of the vehicle.

16. The fallen object detecting method according to claim 13 , further comprising:
tracking the movement of the object from the periodically generated image; and
in response to the tracked object being predicted to fall based on at least one of movement or shape of the tracked object, suggesting to move the tracked object to a safe area, where it is determined that the tracked object will not fall, before the tracked object falls.

17. The fallen object detecting method according to claim 13 , further comprising:
suggesting to pick up the fallen object based on the vehicle stopping or being in a sector where the vehicle drives at a predetermined speed or lower; or
suggesting to immediately pick up the fallen object based on the vehicle being expected to turn sharply or drive on an irregular road within a predetermined time or based on the vehicle being expected to rapidly decelerate or a

Assignees

• Lg Electronics Inc

Inventors

• Lee Chul Hee
• Kim Hyun Kyu
• Song Ki Bong
• Jeong Sang Kyeong
• Jung Jun Young

Classifications

• G06V20/59 Primary

  inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions · CPC title

• B60N2/0268

  using sensors or detectors for adapting the seat or seat part, e.g. to the position of an occupant · CPC title

• B60N2220/20

  using a deterministic algorithm · CPC title

• B60N2/0239

  using movement detection or gesture recognition · CPC title

• B60N2210/24

  Cameras · CPC title