Interactive indoor navigation framework

Patent metadata

Field	Value
Publication number	US-11353327-B2
Application number	US-202015929966-A
Country	US
Kind code	B2
Filing date	May 31, 2020
Priority date	May 31, 2020
Publication date	Jun 7, 2022
Grant date	Jun 7, 2022

Abstract

According to an aspect of an embodiment, operations include receiving user information comprising motion information associated with a group of users in a built environment and user preference information associated with a group of events hosted in the built environment. The operations further include receiving layout information associated with the built environment and schedule information associated with the group of events. The operations further include predicting event-specific action associated with each user based on the user information, and the layout and schedule information. The operations further include predicting a path for navigation towards an event location associated with one of the group of events based on the predicted event-specific action and historical movement information of the group of users. The operations further include selecting, from the predicted paths, one or paths for navigation and generating navigation suggestions based on the selected paths. The navigation suggestions are displayed on display devices.

First claim

What is claimed is:

1. A method, comprising:
receiving user information comprising motion information associated with a group of users in a built environment and user preference information associated with a group of events hosted in the built environment;
receiving layout information associated with the built environment;
receiving schedule information associated with the group of events;
predicting an event-specific action associated with each user of the group of users in the built environment based on the received user information, the received layout information, the received schedule information;
predicting, for each user of the group of users, a path for navigation towards a first event location associated with one of the group of events based on the predicted event-specific action of each user of the group of users and historical movement information of the group of users inside the built environment;
selecting, from the predicted paths for the group of users, one or more paths as one or more optimal options for navigation towards the first event location;
generating one or more navigation suggestions based on the selected one or more paths; and
controlling one or more display devices to display the generated one or more navigation suggestions.

2. The method according to claim 1 , further comprising controlling a plurality of motion trackers at a corresponding plurality of locations inside the built environment to collect the motion information associated with the group of users in the built environment.

3. The method according to the claim 2 , further comprising receiving, from the plurality of motion trackers at the corresponding plurality of locations inside the built environment, a plurality of images as the motion information of the group of users.

4. The method according to claim 1 , wherein the layout information comprises a static physical layout of the built environment and a dynamic layout indicative of user density at a plurality of locations in the built environment.

5. The method according to claim 1 , wherein the event-specific action for each user of the group of users indicates a maximum likelihood of movement of a corresponding user towards a location in the built environment.

6. The method according to claim 1 , wherein the received schedule information comprises an event name for each event of the group of events, an event location of each event of the group of events in the built environment, a seating capacity associated with each event of the group of events, and a time or duration associated with each event of the group of events.

7. The method according to claim 1 , further comprising:
providing the received user information, the received layout information, the received schedule information as input to a graph-structured recurrent neural network, wherein
the graph-structured recurrent neural network is a pre-trained neural network in which nodes are configured for user detection and edges are configured to learn spatial or temporal user-to-user interactions and user-to-object interactions in the built environment, and
the graph-structured recurrent neural network is configured to select, for each user of the group of users, an action label having a maximum likelihood from among a plurality of action labels of an action space based on the input; and

predicting the event-specific action of each user of the group of users in the built environment based on selected action label.

8. The method according to claim 1 , further comprising:
providing the predicted event-specific actions of the group of users and the historical movement information of the group of users as input to an imitative decision learning framework,
wherein the imitative decision learning framework is configured to:
extracting a distribution of latent decisions associated with the group of users based on the input; and
generating a policy for predicting the path for the navigation towards the first event location for each user of the group of users based on the input and extracted distribution of latent decisions; and

predicting, for each user of the group of users, the path for the navigation towards the first event location associated with one of the group of events based on the generated policy.

9. The method according to claim 1 , further comprising:
determining whether a first path of the predicted paths for a first user of the group of users coincides with a second path of the predicted paths for a second user of the group of users; and
selecting the first path as an optimal option of the one or more optimal options for both the first user and the second user for the navigation towards the first event location.

10. The method according to claim 1 , further comprising:
extracting constraint information comprising seating capacity information associated with the first event location in the built environment based on the received layout information associated with the built environment;
formulating an objective function of a constrained convex optimization problem for route optimization based on the extracted constraint information;
determining, from the group of users, a first cluster of users who are moving towards the first event location based on the predicted event-specific actions for the group of users and the received user preference information; and
selecting, from the predicted paths for the group of users, the one or more paths as the one or more optimal options for the first cluster of users to navigate towards the first event location by minimizing the formulation objective function.

11. A non-transitory computer-readable storage medium configured to store instructions that, in response to being executed, causes a system to perform operations, the operations comprising:
receiving user information comprising motion information associated with a group of users in a built environment and user preference information associated with a group of events hosted in the built environment;
receiving layout information associated with the built environment;
receiving schedule information associated with the group of events;
predicting an event-specific action associated with each user of the group of users in the built environment based on the received user information, the received layout information, the received schedule information;
predicting, for each user of the group of users, a path for navigation towards a first event location associated with one of the group of events based on the predicted event-specific action of each user of the group of users and historical movement information of the group of users inside the built environment;
selecting, from the predicted paths for the group of users, one or more paths as one or more optimal options for navigation towards the first event location;
generating one or more navigation suggestions based on the selected one or more paths; and
controlling one or more display devices to display the generated one or more navigation suggestions.

12. The non-transitory computer-readable storage medium according to claim 11 , wherein the operations further comprises controlling a plurality of motion trackers at a corresponding plurality of locations inside the built environment to collect the motion information associated with the group of users in the built environment.

13. The non-transitory computer-readable storage medium according to claim 12 , wherein the operations further comprises receiving, from the plurality of motion trackers at the corresponding plurality of locations inside the built environment, a plurality of images as the motion information of the group of users.

Assignees

• Fujitsu Ltd

Inventors

• Chander Ajay
• Srinivasan Ramya Malur

Classifications

• G01C21/3667

  Display of a road map (G01C21/3614 takes precedence; guidance using 3D or perspective road maps G01C21/3635) · CPC title

• G06V10/82

  using neural networks · CPC title

• G01C21/206 Primary

  specially adapted for indoor navigation · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• H04N7/181

  for receiving images from a plurality of remote sources · CPC title