Safety-assured remote driving for autonomous vehicles

Patent metadata

Field	Value
Publication number	US-11274936-B2
Application number	US-201916683685-A
Country	US
Kind code	B2
Filing date	Nov 14, 2019
Priority date	Nov 14, 2019
Publication date	Mar 15, 2022
Grant date	Mar 15, 2022

Abstract

An apparatus for traveling through a transportation network performs a method including generating a display that includes a geographical area about a starting location for route assistance through the transportation network and a virtual vehicle at the starting location, and forming a virtual path for the route assistance using the virtual vehicle. The virtual path includes a first portion obtained from advancing the virtual vehicle from the starting location while an autonomous vehicle (AV) is at the starting location and a second portion obtained from, after the virtual vehicle departs from the starting location, extrapolating from the virtual vehicle through the geographical area to a stopping location or an ending location of the route assistance. Points along the virtual path are continually transmitted to a trajectory planner of the AV while the virtual vehicle advances through the geographical area, and the trajectory planner generates a route for the AV conforming to the virtual path.

First claim

What is claimed is:

1. An apparatus for traveling through a transportation network, comprising:
a communication device; and
a processor in connection with the communication device and configured to perform a method comprising:
generating a display that includes a geographical area about a starting location for route assistance for an autonomous vehicle (AV) through the transportation network, and a virtual vehicle at the starting location;
forming a virtual path for the route assistance using the virtual vehicle, comprising:
forming a first portion of the virtual path by advancing the virtual vehicle from the starting location while the AV remains at the starting location within the display; and
forming a second portion of the virtual path by, after the virtual vehicle departs from the starting location, extrapolating from the virtual vehicle through the geographical area to a stopping location or an ending location of the route assistance; and

continually transmitting, via the communication device, points along the virtual path to a trajectory planner of the AV while the virtual vehicle advances through the geographical area, wherein the trajectory planner generates a route for the AV conforming to the virtual path, and wherein forming the second portion occurs, at least in part, while the AV advances along the route behind the virtual vehicle relative to a direction of travel of the virtual vehicle.

2. The apparatus of claim 1 , wherein the method comprises:
receiving, for the AV, a request for the route assistance through the transportation network, wherein the request includes the starting location.

3. The apparatus of claim 1 , wherein:
the virtual path comprises a left border and a right border forming a tunnel, and wherein the route for the AV is constrained to remain within the tunnel.

4. The apparatus of claim 1 , wherein:
the route includes a deviation from the virtual path that is homotopic between two points of the virtual path.

5. The apparatus of claim 1 , wherein the method further comprises:
transmitting, via the communication device, instructions to the trajectory planner regarding how to generate the route using the virtual path.

6. The apparatus of claim 1 , wherein the method further comprises:
transmitting, via the communication device, instructions to the AV to disregard one or more sensors of the AV when following the route.

7. The apparatus of claim 1 , wherein generating the display comprises:
generating a representation of the AV at the starting location, wherein the virtual vehicle has a same pose as the AV at the starting location;
continually updating representations of the virtual vehicle and the virtual path extending from the AV to a current location of the virtual vehicle, and extending from the virtual vehicle to an end of the virtual path; and
responsive to the virtual path having a defined distance from the starting location:
continually updating the representations of the AV while the AV advances along the route behind the virtual vehicle relative to a direction of travel of the virtual vehicle.

8. The apparatus of claim 1 , further comprising:
a simulator including an interface configured to receive input from a user defining the virtual path, the simulator providing output corresponding to the virtual path to the processor for generating the display.

9. The apparatus of claim 1 , wherein the method further comprises:
responsive to the AV falling behind the virtual vehicle within the geographical area by a defined amount while advancing the virtual vehicle through the geographical area, defining a current location of the AV as a new starting location of the route assistance.

10. The apparatus of claim 1 , wherein the method further comprises:
responsive to the AV stopping while the virtual vehicle advances through the geographical area, defining a current location of the AV as a new starting location of the route assistance.

11. The apparatus of claim 1 , wherein the generating the display comprises:
receiving images from the AV; and
including, in the display, objects and drivable areas derived from the images.

12. A system for traveling through a transportation network, comprising:
a communication device;
a processor in connection with the communication device and configured to perform a method comprising:
transmitting, via the communication device, a request for route assistance through the transportation network for an autonomous vehicle (AV); and
continually receiving points along a virtual path for the route assistance, the virtual path created by:
generating a display including a geographical area about a starting location of the route assistance and a virtual vehicle at the starting location; and
forming the virtual path using the virtual vehicle, comprising:
forming a first portion of the virtual path by advancing the virtual vehicle from the starting location while the AV is remains at the starting location within the display; and
forming a second portion of the virtual path by, after the virtual vehicle departs from the starting location, extrapolating from the virtual vehicle through the geographical area to a stopping location or an ending location of the route assistance, wherein continually receiving the points comprises continually receiving, via the communication device, the points along the virtual path while the virtual vehicle advances through the geographical area;

a trajectory planner that generates a route for the AV conforming to the virtual path using the points; and
a trajectory follower that generates control inputs for the AV to follow the route while the virtual vehicle advances through the geographical area, wherein forming the second portion occurs, at least in part, while the AV advances along the route behind the virtual vehicle relative to a direction of travel of the virtual vehicle.

13. The system of claim 12 , wherein the method further comprises:
receiving, via the communication device, rules regarding at least one of how to generate the route or how to generate the control inputs.

14. The system of claim 13 , wherein the virtual path comprises a left border and a right border forming a tunnel, and wherein the rules require the trajectory planner to generate the route constraining the AV to the tunnel.

15. The system of claim 13 , wherein the rules require the trajectory follower to generate the control inputs while ignoring at least one sensor of the AV.

16. The system of claim 13 , wherein the rules require the trajectory planner to generate the route to deviate to either a left or a right of the virtual path between two points of the virtual path to avoid an object responsive to the virtual vehicle interfering with the object while forming the virtual path, a deviation in the route being homotopic with respect to the virtual path.

17. The system of claim 12 , wherein the method further comprises:
transmitting, via the communication device, images from the AV regarding environment about the AV, wherein the images are used for generating the display.

18. The system of claim 12 , wherein:
the trajectory follower generates the control inputs for the AV to follow the route substantially in real-time while the second portion of the virtual path is formed.

19. The system of claim 18 , wherein:
the control inputs cause the AV to follow the route at substantially a same speed as the second portion of the virtual path is formed.

20. An autonomous vehicle (AV) for traveling through a trans

Assignees

• Nissan North America Inc
• Renault Sas

Inventors

• Ostafew Christopher
• Aguiar Thiago
• Kobashi Atsuhide
• Ilstrup David
• Hagaribommanahalli Sachin
• Tam Qizhan
• Cypher-Plissart Therese

Classifications

• G05D1/227

  Handing over between remote control and on-board control; Handing over between remote control arrangements · CPC title

• G05D1/2245

  providing the operator with a purely computer-generated representation of the environment of the vehicle, e.g. virtual reality · CPC title

• G05D2107/13

  Spaces reserved for vehicle traffic, e.g. roads, regulated airspace or regulated waters · CPC title

• G05D2105/22

  of humans · CPC title

• G05D2109/10

  Land vehicles · CPC title