Intelligent mobility aid device and method of navigating and providing assistance to a user thereof

Patent metadata

Field	Value
Publication number	US-9316502-B2
Application number	US-201414338305-A
Country	US
Kind code	B2
Filing date	Jul 22, 2014
Priority date	Jul 22, 2014
Publication date	Apr 19, 2016
Grant date	Apr 19, 2016

Abstract

An intelligent navigation device is provided for actively collecting data about a user and the surrounding environment, drawing helpful inferences, and actively aiding the user in navigation, environmental awareness, and social interaction. The intelligent navigation device may include cameras for detecting image data regarding the surrounding environment. The intelligent navigation device may include a GPS unit, an IMU, and a memory for storing previously determined user data. The intelligent navigation device may include a processor for determining a desirable action or event based on the image data, data detected by the GPS unit or the IMU, or a recognized object in the surrounding environment. The processor may further determine a destination and provide navigation assistance to the user for reaching the destination. The intelligent navigation device may convey output data using a display, a speaker, a vibration unit, a mechanical feedback unit, or an electrical stimulation unit.

First claim

What is claimed is:

1. An intelligent guidance device, comprising:
a plurality of wheels for travelling on a ground surface;
a platform coupled to the plurality of wheels;
an inertial measurement unit (IMU) coupled to the platform and configured to detect inertial measurement data corresponding to a positioning, a velocity, or an acceleration of the intelligent navigation device;
a global position system (GPS) unit configured to detect location data corresponding to a location of the intelligent navigation device;
a plurality of cameras coupled to the platform for detecting image data corresponding to a surrounding environment of the intelligent guidance device;
a memory storing object data regarding previously determined objects and storing previously determined user data regarding a user of the intelligent guidance device;
a processor connected to the IMU, the GPS unit, and the plurality of cameras, and configured to:
proactively recognize an object in the surrounding environment by analyzing the image data based on the stored object data and at least one of the inertial measurement data or the location data,
proactively determine a desirable event or action based on at least one of the recognized object, the previously determined user data, or a current time or day,
proactively determine a destination based on the determined desirable event or action,
proactively determine a navigation path for navigating the intelligent guidance device based on the determined destination, the image data, and at least one of the inertial measurement data or the location data, and
proactively determine output data corresponding to the determined navigation path;

a display for proactively displaying a first image based on the output data to provide visual information to the user based on the navigation path and proactively displaying a second image based on the determined desirable event or action; and
a speaker for proactively providing first audio information to the user based on the output data and second audio information based on at least one of the recognized object or the determined desirable event or action.

2. The intelligent guidance device of claim 1 , further comprising:
a left bar coupled and having a left grip;
a right bar coupled to the platform and having a right grip;
a first vibratory motor or actuator coupled to the left grip for providing first haptic information to the user based on the determined output data; and
a second vibratory motor or actuator coupled to the right grip for providing second haptic information to the user based on the determined output data.

3. The intelligent navigation device of claim 1 , wherein the memory is configured to store map data, and the processor is configured to determine the navigation path based on the image data, the map data, and at least one of the location data or the inertial measurement data.

4. The intelligent navigation device of claim 1 , further comprising:
a motor for driving the plurality of wheels, wherein the processor is further configured to control the motor to drive the plurality of wheels based on the determined navigation path.

5. The intelligent navigation device of claim 1 , further comprising:
a left bar coupled to the platform and having a left grip;
a right bar coupled to the platform and having a right grip;
a first mechanical feedback unit having a rotatable or movable element proximal to the left bar,
a second mechanical feedback unit having a rotatable or movable element proximal to the right bar, wherein the processor is further configured to:
control the first mechanical feedback unit to provide a first feedback output based on the determined output data, and
control the second mechanical feedback unit to provide a second feedback output based on the determined output data, the second feedback output being different than the first feedback output.

6. The intelligent navigation device of claim 1 , further comprising:
a wireless communication antenna for establishing an audio or video communication with another portable electronic device or computer used by another person,
wherein the processor is further configured to establish the audio or video communication based on the determined desirable event or action.

7. The intelligent navigation device of claim 1 , further comprising a microphone for detecting a speech of the user or another person, wherein the processor is further configured to:
parse a conversation of the user or the another person into speech elements,
analyze the parsed speech elements based on the previously determined user data, and
determine the desirable event or action further based on the analyzed speech elements.

8. The intelligent navigation device of claim 1 , further comprising a battery coupled to the platform and capable of being wirelessly charged.

9. The intelligent navigation device of claim 1 , further comprising:
a left bar coupled to the platform and having a left grip;
a right bar coupled to the platform and having a right grip;
a first electrical stimulation feedback unit positioned proximal to the left grip of the left bar; and
a second electrical stimulation feedback unit positioned proximal to the right grip of the right bar, wherein the processor is further configured to:
control the first electrical stimulation feedback unit to provide a first feedback output based on the determined output data, and
control the second electrical stimulation feedback unit to provide a second feedback output based on the determined output data, the second feedback output being different than the first feedback output.

10. The intelligent navigation device of claim 1 , wherein the processor is further configured to determine terrain data regarding the surrounding environment based on the image data, the recognized object, and at least one of the inertial measurement data or the location data.

11. The intelligent navigation device of claim 10 , wherein the processer is further configured to:
determine a maneuverability condition or a non-traversable area of the surrounding environment based on the image data, the recognized object, and at least one of the inertial measurement data or the location data, and
determine the navigation path based on the maneuverability condition or the non-traversable area of the surrounding environment for navigating the intelligent navigation device on safely traversable areas of the surrounding environment towards the determined destination.

12. The intelligent navigation device of claim 1 , further comprising an encoder connected to the plurality of wheels and configured to detect data regarding a distance travelled by the intelligent navigation device or a travelling speed of the intelligent navigation device, and
wherein the processor is further configured to determine the location of the intelligent navigation device further based on the distance travelled by the intelligent navigation device or the travelling speed of the intelligent navigation device.

13. The intelligent navigation device of claim 1 , wherein the plurality of cameras include:
a pair of stereo cameras for detecting depth information regarding the surrounding environment.

14. The intelligent navigation device of claim 1 , wherein the plurality of cameras include an omnidirectional data for detecting the image data regarding the surrounding environment.

15. An intelligent guidance device controlled and operated by a user, comprising:
a plurality of wheels for travelling on a ground surface;
a platform coupled to the plurality of wheels;
a left bar having a lef

Assignees

• Toyota Eng & Mfg North America

Inventors

• Moore Douglas A
• Djugash Joseph M A
• Ota Yasuhiro

Classifications

• G01C21/1656 Primary

  with passive imaging devices, e.g. cameras · CPC title

• G01S19/49

  whereby the further system is an inertial position system, e.g. loosely-coupled · CPC title

• H04N13/239

  using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) · CPC title

• G06T2207/10032

  Satellite or aerial image; Remote sensing · CPC title

• G06T7/0046

  Physics · mapped topic