Vehicle based external environment augmentation for operator alertness

Patent metadata

Field	Value
Publication number	US-12313420-B2
Application number	US-202117529481-A
Country	US
Kind code	B2
Filing date	Nov 18, 2021
Priority date	Nov 18, 2021
Publication date	May 27, 2025
Grant date	May 27, 2025

Abstract

A route from a first physical location to a second physical location is divided into a set of segments using a route optimization engine. Using a user response analysis model, a response to a physical environment associated with a segment in the set of segments is scored, the scoring resulting in a score. Using a content generation model and the score, the physical environment is augmented, the augmenting combining the physical environment and a generated environment.

First claim

What is claimed is:

1. A computer-implemented method comprising:
dividing, into a set of segments using a route optimization engine, a route from a first physical location to a second physical location;
scoring, using a user response analysis model, an alertness response to a physical environment external to and adjacent to a vehicle travelling within a segment in the set of segments, the scoring resulting in an alertness score ensuring that a vehicle operator is alert enough to respond quickly when necessary; and
augmenting, using a content generation model and the alertness score, the physical environment external to and adjacent to the vehicle, the augmenting combining image data of the physical environment external to and adjacent to the vehicle with image data of a generated environment, the image data of the physical environment external to and adjacent to the vehicle collected while the vehicle is within the segment, the augmenting performed while the vehicle is within the segment and the alertness score is below a threshold, wherein augmenting the physical environment further comprises:
generating, using an image content generation model, the alertness score, a preset generated environment image which the content generation model determined improved the alertness response, and the image data, an image depicting the generated environment; and
depicting, on a display visible to the vehicle operator, the generated environment.

2. The computer-implemented method of claim 1 , wherein the generated environment comprises a depiction of a route of travel.

3. The computer-implemented method of claim 1 , wherein the generated environment comprises a depiction of terrain associated with a route of travel.

4. The computer-implemented method of claim 1 , wherein the generated environment comprises a depiction of an atmospheric condition associated with a route of travel.

5. The computer-implemented method of claim 1 , wherein augmenting the physical environment comprises:
generating, using an audio content generation model, the alertness score, and the generated environment, the generated environment comprising audio data; and
outputting, using an audio output device audible to the vehicle operator, the generated environment.

6. The computer-implemented method of claim 1 , further comprising:
scoring, using the user response analysis model, a response to the generated environment, the scoring resulting in a scored response;
adjusting, based on the scored response, the generated environment; and
augmenting, using the content generation model, the scored response, and the physical environment, the augmenting combining the physical environment and the adjusted generated environment.

7. A computer program product for vehicle based external environment augmentation, the computer program product comprising:
one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the stored program instructions comprising:
program instructions to divide, into a set of segments using a route optimization engine, a route from a first physical location to a second physical location;
program instructions to score, using a user response analysis model, an alertness response to a physical environment external to and adjacent to a vehicle travelling within a segment in the set of segments, the scoring resulting in an alertness score ensuring that a vehicle operator is alert enough to respond quickly when necessary; and
program instructions to augment, using a content generation model and the alertness score, the physical environment external to and adjacent to the vehicle, the augmenting combining image data of the physical environment external to and adjacent to the vehicle with image data of a generated environment, the image data of the physical environment external to and adjacent to the vehicle collected while the vehicle is within the segment, the augmenting performed while the vehicle is within the segment and the alertness score is below a threshold, wherein augmenting the physical environment further comprises:
program instructions to generate, using an image content generation model, the alertness score, a preset generated environment image which the content generation model determined improved the alertness response, and the image data, an image depicting the generated environment; and
program instructions to depict, on a display visible to the vehicle operator, the generated environment.

8. The computer program product of claim 7 , wherein the generated environment comprises a depiction of a route of travel.

9. The computer program product of claim 7 , wherein the generated environment comprises a depiction of terrain associated with a route of travel.

10. The computer program product of claim 7 , wherein the generated environment comprises a depiction of an atmospheric condition associated with a route of travel.

11. The computer program product of claim 7 , wherein augmenting the physical environment comprises:
program instructions to generate, using an audio content generation model, the alertness score, and the generated environment, the generated environment comprising audio data; and
program instructions to output, using an audio output device audible to the vehicle operator, the generated environment.

12. The computer program product of claim 7 , the stored program instructions further comprising:
program instructions to score, using the user response analysis model, a response to the generated environment, the scoring resulting in a scored response;
program instructions to adjust, based on the scored response, the generated environment; and

program instructions to augment, using the content generation model, the scored response, and the physical environment, the augmenting combining the physical environment and the adjusted generated environment.

13. The computer program product of claim 7 , wherein the stored program instructions are stored in the at least one of the one or more storage media of a local data processing system, and wherein the stored program instructions are transferred over a network from a remote data processing system.

14. The computer program product of claim 7 , wherein the stored program instructions are stored in the at least one of the one or more storage media of a server data processing system, and wherein the stored program instructions are downloaded over a network to a remote data processing system for use in a computer readable storage device associated with the remote data processing system.

15. The computer program product of claim 7 , wherein the computer program product is provided as a service in a cloud environment.

16. A computer system comprising one or more processors, one or more computer-readable memories, and one or more computer-readable storage media, and program instructions stored on at least one of the one or more storage media for execution by at least one of the one or more processors via at least one of the one or more memories, the stored program instructions comprising:
program instructions to divide, into a set of segments using a route optimization engine, a route from a first physical location to a second physical location;
program instructions to score, using a user response analysis model, an alertness response to a physical environment external to and adjacent to a vehicle travelling within a segment in the set of segments, the scoring resulting in an alertness score ensuring that a vehicle operator is alert enough to respond quickly when necessary;

Assignees

• IBM

Inventors

• Baughman Aaron K
• Kwatra Shikhar
• Pandey Diwesh
• Bose Tathagato

Classifications

• G01C21/3617

  using user history, behaviour, conditions or preferences, e.g. predicted or inferred from previous use or current movement · CPC title

• G01C21/3629

  Guidance using speech or audio output, e.g. text-to-speech (text to speech systems per se G10L13/00) · CPC title

• G01C21/3638

  including 3D objects and buildings (three dimensional [3D] modelling, e.g. data description of 3D objects G06T17/00; geographic models G06T17/05) · CPC title

• G01C21/3691

  Retrieval, searching and output of information related to real-time traffic, weather, or environmental conditions (arrangements for giving variable traffic instructions G08G1/09) · CPC title

• G01C21/3484

  Personalized, e.g. from learned user behaviour or user-defined profiles · CPC title