Systems and methods for detecting an environment external to a personal mobile vehicle in a fleet management system

Patent metadata

Field	Value
Publication number	US-12065158-B2
Application number	US-202217859881-A
Country	US
Kind code	B2
Filing date	Jul 7, 2022
Priority date	Aug 9, 2021
Publication date	Aug 20, 2024
Grant date	Aug 20, 2024

Abstract

Commercial personal mobile vehicles (PMVs) managed by a fleet management system are sometimes equipped with a radar sensor to detect objects in an environment external to the PMVs. Specifically, the PMV may be equipped with a variety of sensors, such as a radar, a sonar sensor, a (optional) camera, an inertia measurement unit (IMU), and/or the like. The combination of a radar reflection signal and a sonar signal may provide measurements of characteristics such as a Doppler velocity and height information of a nearby object, which may be input to a machine learning classifier to determine the probability that the nearby object is a VRU. For another example, the reflection pattern from radar and ultrasonic may be used to input to a machine learning classifier to determine a type of the road surface, e.g., an asphalt road surface, a concrete sidewalk surface, a wet-grass lawn surface, and/or the like.

First claim

What is claimed is:

1. A personal mobile vehicle (PMV) for riding on a road surface and managed by a fleet management system, the PMV comprising:
a frame carrying a wheel;
a radar sensor carried by the frame of the PMV, wherein the radar sensor is configured to (i) emit an electromagnetic wave signal towards an environment external to the PMV, and (ii) receive a radar reflection signal indicating a characteristic of the road surface of the environment external to the PMV; and
a processing unit communicatively coupled to the radar sensor, the processing unit being configured to:
determine one or more characteristic factors of the road surface of the environment external to the PMV according to the radar reflection signal, wherein the one or more characteristics factors include at least a signal mean power or signal attenuation of the radar reflection signal;
generate a road surface type classification distribution by using a first machine learning classifier to process the one or more characteristic factors of the road surface of the environment, wherein the road surface type classification distribution represents probabilities of the road surface belonging to one or more road surface types;
generate a vulnerable road user (VRU) classification probability distribution by using a second machine learning classifier, wherein the VRU classification probability distribution indicates whether a combination of the one or more characteristic factors is associated with the VRU;
determine, based on one or more of the road surface type classification distribution or the VRU classification probability distribution, that the road surface is associated with a predetermined road surface type of concern or the VRU;
in response to a determination that the road surface is associated with the predetermined road surface type of concern or the VRU, determine a recommended speed limit for the PMV to operate on the predetermined road surface type of concern; and
present an alert message indicating the recommended speed limit to a display associated with the PMV.

2. The PMV of claim 1 , further comprising at least one of a sonar sensor, a camera, or an inertia measurement unit (IMU).

3. The PMV of claim 1 , further comprising:
a second sensor configured to capture a second sensing signal that is of a different type from the radar reflection signal.

4. The PMV of claim 1 , wherein the one or more characteristic factors relates to an object on the road surface in the environment, including any combination of:
a Doppler velocity of the object;
height information of the object;
visual information of the object; and
an acceleration rate of the object.

5. The PMV of claim 1 , wherein the one or more characteristic factors of the road surface are determined based on a combination of the radar reflection signal and a sonar signal from the road surface.

6. The PMV of claim 1 , wherein the road surface type classification distribution represents a first probability of the road surface being an asphalt road surface and a second probability of the road surface being a concrete sidewalk surface.

7. The PMV of claim 1 , wherein the processing unit is further configured to:
generate the alert message including the recommended speed limit based on a corresponding likelihood that the environment external to the PMV includes at least one of the VRU or the predetermined road surface type of concern.

8. The PMV of claim 1 , wherein the processing unit is further configured to:
transmit, to a control mechanism that controls the wheel, a control signal that limits a speed of the PMV based on a corresponding likelihood that the environment includes at least one of the VRU or the predetermined road surface type of concern.

9. The PMV of claim 1 , further comprising:
a positioning unit configured to capture first location information of the PMV; and
a transceiver configured to:
transmit, to a remote server, determined information on whether the environment includes at least one of the VRU or the predetermined road surface type of concern and the first location information of the PMV; and
receive, from the remote server, second location information and an indication on whether a corresponding location of the second location information contains at least one of another VRU or another predetermined road surface type of concern,
wherein the second location information and the indication are uploadable to the remote server by another PMV.

10. The PMV of claim 9 , wherein the transceiver is further configured to receive, from the remote server, a control message indicating a suggested speed limit based on the determined information on whether the environment includes at least one of the VRU or the predetermined road surface type of concern and the first location information of the PMV.

11. A computer-implemented method of receiving road condition information from personal mobile vehicles (PMVs), the method comprising, by a fleet management system:
receiving, by the fleet management system from a first PMV, a first signal indicating whether a first surrounding of the first PMV includes at least one of i) a first vulnerable road user (VRU) or ii) a first predetermined road surface type of concern and a first recommended speed limit associated with the first predetermined road surface type of concern, wherein:
the first predetermined road surface type of concern is determined based on a road surface type classification distribution generated by using a first machine learning classifier, and
the first VRU is determined based on a VRU classification probability distribution generated by using a second machine learning classifier;

obtaining, by the fleet management system, first location information of a first user associated with the first PMV;
receiving, by the fleet management system from a second PMV, a second signal indicating whether a second surrounding of the second PMV includes at least one of iii) a second VRU or iv) a second predetermined road surface type of concern and a second recommended speed limit associated with the second predetermined road surface type of concern;
obtaining, by the fleet management system, second location information of a second user associated with the second PMV; and
transmitting, by the fleet management system to the first PMV, road condition information received from the second PMV that is located in vicinity to the first PMV, the road condition information received from the second PMV comprising information on whether the second surrounding of the second PMV includes the second location information and at least one of iii) the second VRU or iv) the second predetermined road surface type of concern and the second recommended speed limit associated with the second predetermined road surface type of concern, wherein the road condition information received from the second PMV is used to ascertain whether the first surrounding of the first PMV includes at least one of the first VRU or the first predetermined road surface type of concern.

12. The computer-implemented method of claim 11 , further comprising:
determining, for the first PMV, the first recommended speed limit depending on whether the first surrounding of the first PMV includes at least one of the first VRU or the first predetermined road surface type of concern and the first location information; and
transmitting, to the first PMV or a user device of the user, an alert message comprising the first recommended speed limit.

13. The computer-implemented method of claim 11 , further comprising:
determining, for the first PMV, an alternate route when the first surrounding of the first

Assignees

• Lyft Inc

Inventors

• Cooper Ashley John
• Thurber Alison Marie
• El Naga Eahab Nagi
• Cull Christy Fernandez
• Zaidi Abdullah Ahsan

Classifications

• B60W2420/408

  Radar; Laser, e.g. lidar · CPC title

• B60W2420/403

  Image sensing, e.g. optical camera · CPC title

• G01S13/86

  Combinations of radar systems with non-radar systems, e.g. sonar, direction finder · CPC title

• B60W2420/54

  Audio sensitive means, e.g. ultrasound · CPC title

• B60W2050/146

  Display means · CPC title