Method and data presenting device for assisting a remote user to provide instructions

Patent metadata

Field	Value
Publication number	US-9829873-B2
Application number	US-201414893271-A
Country	US
Kind code	B2
Filing date	May 9, 2014
Priority date	Jun 27, 2013
Publication date	Nov 28, 2017
Grant date	Nov 28, 2017

Abstract

A method, data presenting device and computer program product are provided for assisting a remote user. The data presenting device includes a communication interface, a camera having a field of view in which images are captured, a projector having a presentation area in which presentation items may be presented, and a control unit which receives field of view control commands a command to project a presentation item in the presentation area, controls the field of view of the camera and controls the projector to project the presentation item in the presentation area, wherein the control of the projection in the presentation area is made independently of the control of the field of view.

First claim

The invention claimed is:

1. A method for assisting a remote user to provide instructions to a location in an industrial site where a process control system is operated, the method being performed by a data presenting device when being involved in a communication session with a device of the remote user, the data presenting device being moveable from location to location comprising a camera and a projector, the method comprising:
retrieving or creating a three-dimensional model of the location;
associating a presentation item to a position in the three-dimensional model, where the presentation item is a remote user generated presentation item comprising instructions and visual indicators;
receiving camera control commands from the remote user, said camera control commands comprising field of view control commands;
controlling the camera using the camera control commands comprising controlling the field of view of the camera;
receiving projector control commands from the remote user, said projector control commands comprising a command to project the presentation item in a presentation area of the projector; and
controlling the projector using the projector control commands comprising controlling the projector to project the presentation item in the presentation area,
wherein the controlling of the presentation in the presentation area is performed independently of the controlling of the field of view of the camera and comprises controlling the projector to project the presentation item onto a real world position corresponding to the position in the three-dimensional model.

2. The method according to claim 1 , wherein the camera control commands comprise commands controlling the orientation of the camera and the projector control commands comprise commands controlling the orientation of the projector and the controlling of the orientation of the projector is performed independently of the controlling of the orientation of the camera.

3. The method according to claim 2 , wherein the presentation area may differ from the field of view.

4. The method according to claim 2 , further comprising receiving the presentation item from the remote user.

5. The method according to claim 2 , wherein the real world position is retained if the presentation area is moved.

6. The method according to claim 2 , further comprising receiving a selection of a process control object, fetching data about the process control object from the process control system and transferring said data to the remote user.

7. The method according to claim 1 , wherein the presentation area may differ from the field of view.

8. The method according to claim 7 , further comprising receiving the presentation item from the remote user.

9. The method according to claim 7 , wherein the real world position is retained if the presentation area is moved.

10. The method according to claim 7 , further comprising receiving a selection of a process control object, fetching data about the process control object from the process control system and transferring said data to the remote user.

11. The method according to claim 1 , further comprising receiving the presentation item from the remote user.

12. The method according to claim 11 , wherein the real world position is retained if the presentation area is moved.

13. The method according to claim 1 , wherein the real world position is retained if the presentation area is moved.

14. The method according to claim 1 , further comprising receiving a selection of a process control object, fetching data about the process control object from the process control system and transferring said data to the remote user.

15. A data presenting device for assisting a remote user in providing instructions to a location in an industrial site where a process control system is operated, the data presenting device being moveable from location to location and comprising:
a communication interface for providing a communication session with a device of the remote user;
a camera having a field of view in which images are captured;
a projector having a presentation area in which presentation items may be presented; and
a control unit configured to retrieve or create a three-dimensional model of the location, associate a presentation item to a position in the three-dimensional model, where the presentation item is a remote user generated presentation item comprising instructions and visual indicators, receive, from the remote user, camera control commands comprising field of view control commands and projector control commands comprising a command to project the presentation item in the presentation area, control the camera using the camera control commands comprising controlling the field of view of the camera and control the projector using the projector control commands comprising controlling the projector to project the presentation item in the presentation area,
wherein the control unit is configured to control the presentation in the presentation area independently of the control of the field of view of the camera and control the projector to project the presentation item onto a real world position corresponding to the position in the three-dimensional model.

16. The data presenting device according to claim 15 , wherein the camera control commands comprise commands controlling the orientation of the camera and the projector control commands comprise commands controlling the orientation of the projector and the control unit is further configured to control the orientation of the projector independently of the control of the orientation of the camera.

17. The data presenting device according to claim 15 , wherein the presentation area may differ from the field of view.

18. The data presenting device according to claim 15 , wherein the control unit is further configured to receive the presentation item from the remote user.

19. The data presenting device according to claim 15 , wherein the real world position is retained if the presentation area is moved.

20. A computer program product for assisting a remote user to provide instructions to a location in an industrial site where a process control system is operated, said computer program product being provided on a non-transitory data carrier comprising computer program code configured to cause a data presenting device being moveable from location to location and comprising a camera and a projector to, when the data presenting device is involved in a communication session with a device of the remote user,
retrieve or create a three-dimensional model (3DM) of the location;
associate a presentation item to a position in the three-dimensional model, where the presentation item is a remote user generated presentation item comprising instructions and visual indicators;
receive camera control commands from the remote user, said camera control commands comprising field of view control commands;
control the camera using the camera control commands comprising controlling the field of view of the camera;
receive projector control commands from the remote user, said projector control commands comprising a command to project a presentation item in a presentation area of the projector; and
control the projector using the projector control commands comprising controlling the projector to project the presentation item in the presentation area,
wherein the control of the presentation in the presentation area is performed independently of the control of the field of view of the camera and comprises con

Assignees

• Abb Technology Ltd
• Abb Schweiz Ag

Inventors

• Alfredsson Fredrik
• Vartiainen Elina
• Brönmark Jonas
• Tsai Tony
• Tait Matthew
• Billinghurst Mark

Classifications

• G05B2219/23258

  GUI graphical user interface, icon, function bloc editor, labview · CPC title

• G05B23/0216

  Human interface functionality, e.g. monitoring system providing help to the user in the selection of tests or in its configuration · CPC title

• G05B19/0425 Primary

  Safety, monitoring · CPC title

• G05B2219/13

  Plc programming · CPC title

• G05B2219/35494

  Online documentation, manual, procedures, operator, user guidance, assistance · CPC title