Providing uninterrupted remote control of a production device via virtual reality devices

Patent metadata

Field	Value
Publication number	US-12586012-B2
Application number	US-202318356275-A
Country	US
Kind code	B2
Filing date	Jul 21, 2023
Priority date	Jul 21, 2023
Publication date	Mar 24, 2026
Grant date	Mar 24, 2026

Abstract

Computer-implemented methods for providing uninterrupted remote control of a production device are provided. Aspects include obtaining a production task to be performed using the production device, identifying a remote worker, from a remote worker database, to perform the production task via a virtual reality device, and assigning the production task to the remote worker. Aspects also include monitoring a remote worker environment of the remote worker during performance of the production task and determining that the production task needs to be reassigned. Aspects further include identifying a replacement remote worker from the remote worker database and reassigning the production task to the replacement remote worker.

First claim

What is claimed is:

1 . A method for providing uninterrupted remote control of a production device comprising:
obtaining a production task to be performed using the production device; identifying a remote worker, from a remote worker database, to perform the production task via a virtual reality device; assigning the production task to the remote worker, wherein the virtual reality device generates realistic images and sounds to simulate the presence of the remote worker in a production environment, presents a digital twin of the production device in the production environment, and captures interaction between the remote worker and the digital twin of the production device, wherein the virtual reality device transmits the captured interaction data between the remote worker and the digital twin of the production device to the production device such that the remote worker causes one or more sensors to physically control the production device by interacting with the digital twin of the production device using the virtual reality device, wherein the virtual reality device generates, in real time, control signals corresponding to operational parameters of the production device, including at least movement path, operational speed, and operation direction, and to transmit the control signals to the production device for execution, thereby effecting a transformation of the production device; monitoring a remote worker environment of the remote worker during performance of the production task, wherein one or more sensors monitor a location of the remote worker in the remote worker environment, spacing between the remote worker and objects in the remote worker environment, movement of the remote worker in the remote worker environment, a power level of the virtual reality device, and a quality of a network connection between the virtual reality device and the production environment; determining that the production task needs to be reassigned, wherein determining that the production task needs to be reassigned comprises obtaining multi-channel time-series data from the one or more sensors, inputting the time-series data into a trained artificial neural network selected from the group consisting of a convolutional neural network and a recurrent neural network, generating, by the artificial neural network, a quantitative risk score representing a likelihood of an interruption event occurring in the remote worker environment, and comparing the risk score to a threshold to determine whether the production task needs to be reassigned, wherein the interruption event includes one of a projected impact between the remote worker and an object in the remote worker environment, a projected communications disruption between the virtual reality device and the production device, and a projected loss of power to the virtual reality device; identifying a replacement remote worker from the remote worker database; and reassigning the production task to the replacement remote worker, wherein reassigning the production task to the replacement remote worker includes performing a handoff of the production task from the remote worker to the replacement remote worker, wherein a voice communications channel is established between the remote worker and the replacement remote
worker during the handoff, wherein both the remote worker and the replacement remote worker cause one or more sensors to physically control the operation of the production device during the handoff period, wherein a duration of the handoff is determined at least in part based on an expected amount of time between the reassignment and the expected interruption event, and wherein the reassignment of the production task to the replacement remote worker includes instructing the virtual reality device of the replacement remote worker to provide a simulation of the production environment and a digital twin of the production device for the replacement remote worker to interact with,
wherein a scheduling system coordinates the handoff, wherein, during the handoff, the virtual reality device of the replacement remote worker displays a simulation of the production environment including the assigned remote worker, and wherein the reassignment is performed prior to any interruption in the production task to ensure continuous operation of the production device by one of the remote worker and the replacement remote worker.

2 . The method of claim 1 , wherein the remote worker database includes a plurality of remote workers, characteristics of the remote worker environment of each of the plurality of remote workers, one or more qualifications of each of the plurality of remote workers, and an availability of each the plurality of remote workers.

3 . The method of claim 2 , wherein the production task includes one or more required remote worker qualifications, an expected duration of the production task, and one or more special requirements and wherein identification of the remote worker is based on a comparison of the characteristics of the remote worker environment of each of the plurality of remote workers to the one or more special requirements, the one or more qualifications of each of the plurality of remote workers to the one or more required remote worker qualifications, and the expected duration of the production task to the availability of each the plurality of remote workers.

4 . The method of claim 1 , wherein one or more sensors monitor the remote worker environment during performance of the production task and wherein determining that the production task needs to be reassigned comprises determining that a likelihood of an interruption event occurring in the remote worker environment of the remote worker is greater than a threshold level based on monitored changes in available space for movement of the remote worker in the remote worker environment.

5 . The method of claim 4 , wherein the interruption event includes an expected impact between the remote worker and an object in the remote worker environment.

6 . The method of claim 5 , further comprising providing a warning to the remote worker regarding the expected impact and deactivating the virtual reality device of the remote worker.

7 . The method of claim 1 , wherein the virtual reality device provides the remote worker with a digital twin simulation of a production environment including the production device and operational parameters of the production device comprising a movement path, an operational speed, and an operation direction.

8 . A computing system having a memory having computer readable instructions and one or more processors for executing the computer readable instructions, the computer readable instructions controlling the one or more processors to perform operations comprising:
obtaining a production task to be performed using a production device; identifying a remote worker, from a remote worker database, to perform the production task via a virtual reality device; assigning the production task to the remote worker, wherein the virtual reality device generates realistic images and sounds to simulate the presence of the remote worker in a production environment, presents a digital twin of the production device in the production environment, and captures interaction between the remote worker and the digital twin of the production device, wherein the virtual reality device transmits the captured interaction data between the remote worker and the digital twin of the production device to the production device such that the remote worker causes one or more sensors to physically control the production device by interacting with the digital twin of the production device using the virtual reality device, wherein the virtual reality device generates,

Assignees

• IBM

Inventors

• Rakshit Sarbajit K
• Kairali Sudheesh S
• Thomas Binoy
• Jakkula Satyam

Classifications

• G06T17/00

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06Q10/063112

  Skill-based matching of a person or a group to a task · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06Q10/063116 Primary

  Schedule adjustment for a person or group · CPC title