Teleoperation assist device, teleoperation assist method, and storage medium

Patent metadata

Field	Value
Publication number	US-12570002-B2
Application number	US-202218080797-A
Country	US
Kind code	B2
Filing date	Dec 14, 2022
Priority date	Jan 24, 2022
Publication date	Mar 10, 2026
Grant date	Mar 10, 2026

Abstract

A teleoperation assist device includes: a motion acquiring unit configured to acquire information on a motion of an operator who operates an end effector; an intention estimating unit configured to estimate a target object which is a target operated by the end effector and a task which is an operation method of the target object using the information acquired by the motion acquiring unit; an environmental status determining unit configured to acquire environment information of an environment in which the end effector is operated; a parameter setting unit configured to acquire information from the intention estimating unit and the environmental status determining unit and to set parameters of an operation type of the end effector from the acquired information; and an end effector operation determining unit configured to determine an amount of operation of the end effector on the basis of taxonomy which is the set parameters and the information acquired by the motion acquiring unit.

First claim

What is claimed is:

1 . A teleoperation assist device that remotely operates at least an end effector representative of a human hand, the teleoperation assist device comprising:
a moving means which moves the end effector; and a processor configured to: acquire information on a motion of an operator who operates the teleoperation assist device to cause, via the moving means, at least the end effector to move, the information on the motion of the operator including information of a position of a finger of the operator and a movement of the finger of the operator; acquire environment information with respect to a first environment in which the end effector is operated and a second environment in which the teleoperation assist device is operated; estimate a target object which is a target associated with what the operator intends to operate by the end effector and a task which is an operation method of the target object related to how the operator intends to operate, by inputting the information associated with the motion of the operator and the environment to a first model, the target object and the operator method changing according to the environment information; set a first parameter associated with an operation type of the end effector and a second parameter associated with an operation of the moving means by inputting the estimate of the target object and the task which is the operation method of the target object to a second model; when a same object as the target object is not included in the second model, set the first parameter and the second parameter based on information of a similar object being similar to the target object in the second model; determine an amount associated with an operation of the end effector based on a grasp taxonomy of human grasp type representative of the first parameter and the information associated with the motion of the operator; determine an amount of operation associated with the moving means based on the second parameter, and the information associated with the motion of the operator.

2 . The teleoperation assist device according to claim 1 , wherein the first parameter and the second parameter comprise the taxonomy, object shape information of the target object, information indicating a task which is performed with a single hand, information indicating a task which is performed with two hands, margin information for interference avoidance, and a target torque.

3 . The teleoperation assist device according to claim 1 , wherein the processor is configured to:
perform control such that the moving means is able to maintain a restricted motion along a surface of the target object using a wrist command value associated with a position and movement of a wrist of the operator, object shape information of the target object which is one of the first parameter and the second parameter, and information indicating a task which is performed with a single hand or information indicating a task performed with two hands which is the parameter; provide a target position and a target posture of the moving means using a command value and margin information for interference avoidance and generate a joint angle command value for the moving means; and generate a joint target torque command value using the joint angle command value and a target torque and control a torque of the moving means based on the joint target torque command value.

4 . The teleoperation assist device according to claim 3 , wherein the processor is configured to output an instruction to invalidate a process when a shape of the target object is atypical.

5 . The teleoperation assist device according to claim 3 , wherein a number of moving means and a number of end effectors are two or more,
wherein the processor is configured to perform control such that a plurality of end effectors operate in cooperation, and wherein the processor is configured to perform control such that a plurality of moving means operate in cooperation.

6 . The teleoperation assist device according to claim 1 , wherein the information on the motion of the operator further includes information of a gaze of the operator, a position of a wrist of the operator, and movement of a wrist of the operator.

7 . A teleoperation assist method that is performed by a teleoperation assist device remotely operating at least an end effector representative of a human hand, the teleoperation assist method comprising:
acquiring information on a motion of an operator who operates the teleoperation assist device to cause at least the end effector to move, via a moving means, wherein the information on the motion of the operator includes information of a position of a finger of the operator and a movement of the finger of the operator; acquiring environment information of with respect to a first environment in which the end effector is operated and a second environment in which the teleoperation assist device is operated; estimating a target object which is a target associated with what the operator intends to operate by the end effector and a task which is an operation method of the target object related to how the operator intends to operate, by inputting the information associated with the motion of the operator and the environment information to a first model, the target object and the operator method changes according to the environment information; setting a first parameter associated with an operation type of the end effector and a second parameter associated with an operation of the moving means by inputting an estimate of the target object and the task which is the operation method of the target object to a second model; when a same object as the target object is not included in the second model, set the first parameter and the second parameter based on information of a similar object being similar to the target object in the second model; determining an amount associated with an operation of the end effector based on a grasp taxonomy of human grasp type representative of the first parameter and the information associated with the motion of the operator; determining an amount of operation associated with the moving means based on the second parameter, and the information associated with the motion of the operator.

8 . A non-transitory computer-readable storage medium storing a program causing a computer of a teleoperation assist device remotely operating at least an end effector representative of a human hand to perform:
acquiring information on a motion of an operator who operates the teleoperation assist device to cause, via a moving means, at least the end effector and the moving means to move, wherein the information on the motion of the operator includes information of a position of a finger of the operator and a movement of the finger of the operator; acquiring environment information of with respect to a first environment in which the end effector is operated and a second environment in which the teleoperation assist device is operated; estimating a target object which is a target associated with what the operator intends to operate by the end effector and a task which is an operation method of the target object related to how the operator intends to operate, by inputting information associated with the motion of the operator and the environment information to a first model, the target object and the operator method changes according to the environment information; setting a first parameter associated with an operation type of the end effector and a second parameter associated with an operation of the moving means by inputting an estimate of the target object and the task which is the operation method of the target object to a second model; when a same object as the tar

Assignees

• Honda Motor Co Ltd

Inventors

• Dong Yili
• Mizutani Akira
• Chaki Tomohiro
• Watabe Tomoki

Classifications

• G06F3/014

  Hand-worn input/output arrangements, e.g. data gloves · CPC title

• G05B19/4155

  characterised by program execution, i.e. part program or machine function execution, e.g. selection of a program · CPC title

• B25J9/1682

  Dual arm manipulator; Coordination of several manipulators · CPC title

• B25J9/1633

  compliant, force, torque control, e.g. combined with position control · CPC title

• G05B2219/40184

  Compliant teleoperation, operator controls motion, system controls contact, force · CPC title