Systems and methods to apply surface treatments

Patent metadata

Field	Value
Publication number	US-12559894-B2
Application number	US-202318094041-A
Country	US
Kind code	B2
Filing date	Jan 6, 2023
Priority date	Aug 31, 2017
Publication date	Feb 24, 2026
Grant date	Feb 24, 2026

Abstract

An example method includes storing surface treatment data to specify at least one selected surface treatment to apply at a target location along a vehicle path of travel, the surface treatment data including a machine-readable description and a reference coordinate frame for the selected surface treatment. The method also includes generating task plan data to apply the selected surface treatment based on the surface treatment data and at least one parameter of an application tool. The method also includes determining a location and orientation of the application tool with respect to the vehicle path of travel based on location data representing a current location of a vehicle carrying the application tool. The method also includes computing a joint-space trajectory to enable the application tool to apply the selected surface treatment at the target location based on the task plan data and the determined location of the application tool.

First claim

What is claimed is:

1 . A system to apply surface treatments to a surface on which a vehicle is adapted to travel, the system comprising:
an application tool having a joint space, the application tool being carried by the vehicle and including a robot having a dispensing tool adapted to apply a surface treatment; one or more non-transitory machine-readable media to store instructions and surface treatment data, the surface treatment data describing at least one selected surface treatment to apply at a target location on the surface, the surface treatment data including a reference frame for the selected surface treatment that is to be applied; a processor configured to execute the instructions to cause the processor to at least:
generate task plan data based on the surface treatment data and at least one parameter of the application tool, the task plan data including instructions describing a process of the application tool applying the selected surface treatment independent of the target location and a pose of the application tool;
determine a pose of the reference frame with respect to the application tool;
define the target location and an orientation of the selected surface treatment on the surface in response to a substantially real-time user input instruction entered through a graphical user interface at runtime when applying the selected surface treatment; and
responsive to the user input instruction:
compute a joint-space trajectory to enable the application tool to apply the selected surface treatment at the target location with the orientation based on the task plan data, the pose of the reference frame and the pose of the application tool; and
control the joint space of the application tool and the dispensing tool to apply the selected surface treatment at the target location with the orientation based on the computed joint-space trajectory.

2 . The system of claim 1 , wherein the instructions are further programmed to:
generate guidance to inform a vehicle operator whether or not the application tool is within a marking zone that defines a spatial region from which the application tool has sufficient reachability to apply at least a substantial portion of the selected surface treatment at the target location, wherein the guidance includes a graphical representation of the selected surface treatment superimposed onto an image of the surface that includes the target location based on the target location and the orientation of the selected surface treatment.

3 . The system of claim 1 , wherein the instructions are further programmed to:
enable the tool controller to execute the joint-space trajectory in response to the user input instruction confirming application of the selected surface treatment at the target location; and disable the tool controller from executing the joint-space trajectory in response to the user input instruction rejecting application of the selected surface treatment at the target location.

4 . The system of claim 1 , wherein the instructions are further programmed to:
generate the graphical user interface (GUI) to provide a visualization of the selected surface treatment at the target location, in which the visualization includes a graphical representation of the selected surface treatment superimposed onto a real-time image that includes the target location; in response to the user input instruction via the GUI rejecting application of the selected surface treatment at the target location shown in the image, receive another user input instruction via the GUI to adjust the target location and orientation to a modified target location and orientation; and responsive to the user input instruction to adjust the target location to a modified target location, compute a modified joint-space trajectory to enable the application tool to apply the selected surface treatment at the modified target location and having the modified orientation based on the pose of the reference frame, the task plan data and the pose of the application tool, wherein the tool controller is configured to control the application tool to apply the selected surface treatment at the modified target location based on the modified joint-space trajectory.

5 . The system of claim 1 , further comprising:
a global positioning system device to provide geospatial coordinates of the vehicle, the vehicle carrying the application tool, the processor and the tool controller; and at least one other sensor configured to sense fiducials, the processor further programmed to determine location data from the geospatial coordinates and the sensed fiducials, the location data being representative of an absolute spatial location of the vehicle, wherein the processor is further programmed to determine the pose of the application tool based on the location data.

6 . The system of claim 1 , wherein the selected surface treatment is a selected marking and the instructions are further programmed to:
generate the graphical user interface to display a graphical template of the selected marking superimposed on a real-time image of a marking zone, which represents an area on the surface that includes the target location; and adjust a position and/or orientation of the graphical template in the image in response to the user input instruction, wherein the target location and the surface treatment data, including the reference frame, for the selected marking are determined based on the position and/or orientation of the graphical template in the image.

7 . The system of claim 6 , further comprising a camera configured to provide the image of the marking zone, wherein the image of the marking zone is one of a stored image or a real time image.

8 . The system of claim 6 , wherein the user input instruction is a first user input instruction, and the instructions are further programmed to:
generate the graphical template of the selected marking in response to a selection user input instruction received via the graphical user interface to choose the selected marking from a plurality of different markings.

9 . The system of claim 1 , wherein the user input instruction is a first user input instruction, the selected surface treatment is a selected marking, and the instructions are further programmed to at least:
display the graphical user interface; and in response to a second user input instruction received at runtime via the graphical user interface to choose a new marking, store new marking data in the non-transitory machine-readable media, the new marking data including a marking identifier to uniquely identify the new marking in a data store of available markings, a heading of the new marking and spatial coordinates specifying a user-selected target location and orientation on the surface where the new marking is to be applied.

10 . The system of claim 9 , wherein, in response to another user input instruction received via the GUI to apply the new marking, the instructions are further programmed to:
determine a current pose of the application tool based on the current pose of the vehicle carrying the application tool; and compute a corresponding joint-space trajectory to enable the application tool to apply the new marking at the user-selected target location on the surface based on the new marking data and the current pose of the application tool, wherein the tool controller is configured to control the application tool to apply the new marking at the user-selected target location based on the computed corresponding joint-space trajectory.

11 . The system of claim 1 , wherein the application tool is configured to apply the selected surface treatment by at least one of adding a volu

Assignees

• Univ Case Western Reserve

Inventors

• Newman Wyatt S
• Bell Samuel A

Classifications

• G01C21/28

  with correlation of data from several navigational instruments · CPC title

• G06F9/451

  Execution arrangements for user interfaces · CPC title

• G06Q10/06316 Primary

  Sequencing of tasks or work · CPC title

• G06F3/04845

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title

• E01F9/518

  formed in situ, e.g. by painting, by casting into the road surface or by deforming the road surface · CPC title