Task execution method and apparatus, device, and computer medium

Patent metadata

Field	Value
Publication number	US-2025242496-A1
Application number	US-202418945322-A
Country	US
Kind code	A1
Filing date	Nov 12, 2024
Priority date	Jan 30, 2024
Publication date	Jul 31, 2025
Grant date	—

Abstract

The present disclosure discloses a task execution method and apparatus, a device, and a computer medium. The method includes: displaying acquired environment image information; and controlling a holding apparatus to execute a task corresponding to an operation of a user on the environment image information in response to the operation of the user on the environment image information.

First claim

1 . A task execution method, applicable to an intelligent robot, the method comprising:
displaying acquired environment image information; and controlling a holding apparatus to execute a task corresponding to an operation of a user on the environment image information, in response to the operation of the user on the environment image information.

2 . The method according to claim 1 , wherein the method further comprises:
performing instance segmentation on the environment image information to obtain at least one first instance corresponding to at least one object comprised in the environment image information; displaying the at least one first instance; in response to an operation of the user on any first instance in the at least one first instance, determining a target instance that is operated on and a task to be executed for the target instance; and the controlling the holding apparatus to execute the task corresponding to the operation comprises: controlling, based on the operation, the holding apparatus to execute the task to be executed for the target instance.

3 . The method according to claim 2 , wherein the method further comprises:
for any object in the at least one object, obtaining an instruction from the user for performing instance segmentation on the object, and performing instance segmentation on the object to obtain at least one second instance corresponding to the object; and using the at least one second instance as a first instance corresponding to the object.

4 . The method according to claim 2 , wherein the operation of the user on the target instance comprises any one or more of the following operations:
an operation of the user to move the target instance; an operation of the user to rotate the target instance; and an operation of the user on the target instance itself.

5 . The method according to claim 2 , wherein the operation of the user on the target instance is implemented in any one or more of the following manners:
clicking the target instance, and sliding the target instance.

6 . The method according to claim 2 , wherein the method further comprises:
determining, for each first instance in the at least one first instance, a three-dimensional model corresponding to the first instance; and the displaying the at least one first instance comprises: displaying at least one three-dimensional model corresponding to the at least one first instance.

7 . The method according to claim 2 , wherein the controlling, based on the operation, the holding apparatus to execute the task to be executed for the target instance comprises:
determining, from the environment image information, an object to be held corresponding to the target instance; determining, based on the operation, movement parameter information of the object to be held; and controlling the holding apparatus to execute the task to be executed for the target instance based on the movement parameter information.

8 . The method according to claim 7 , wherein the determining, based on the operation, the movement parameter information of the object to be held comprises:
when the operation is an operation of the user to move the target instance, obtaining an initial position of the target instance; obtaining a target position to which the target instance is moved; and determining the movement parameter information of the object to be held based on the initial position and the target position.

9 . The method according to claim 8 , wherein the determining the movement parameter information of the object to be held based on the initial position and the target position comprises:
determining a movement trajectory of the holding apparatus based on the initial position and the target position; and determining the movement parameter information of the object to be held based on the movement trajectory.

10 . An electronic device, comprising:
a processor; and a memory, configured to store executable instructions for the processor, wherein the processor is configured to execute the executable instructions to perform: displaying acquired environment image information; and controlling a holding apparatus to execute a task corresponding to an operation of a user on the environment image information, in response to the operation of the user on the environment image information.

11 . The electronic device according to claim 10 , wherein the processor is configured to execute the executable instructions to further perform:
performing instance segmentation on the environment image information to obtain at least one first instance corresponding to at least one object comprised in the environment image information; displaying the at least one first instance; in response to an operation of the user on any first instance in the at least one first instance, determining a target instance that is operated on and a task to be executed for the target instance; and the controlling the holding apparatus to execute the task corresponding to the operation comprises: controlling, based on the operation, the holding apparatus to execute the task to be executed for the target instance.

12 . The electronic device according to claim 11 , wherein the processor is configured to execute the executable instructions to further perform:
for any object in the at least one object, obtaining an instruction from the user for performing instance segmentation on the object, and performing instance segmentation on the object to obtain at least one second instance corresponding to the object; and using the at least one second instance as a first instance corresponding to the object.

13 . The electronic device according to claim 11 , wherein the processor is configured to execute the executable instructions to further perform:
determining, for each first instance in the at least one first instance, a three-dimensional model corresponding to the first instance; and the displaying the at least one first instance comprises: displaying at least one three-dimensional model corresponding to the at least one first instance.

14 . The electronic device according to claim 11 , wherein the controlling, based on the operation, the holding apparatus to execute the task to be executed for the target instance comprises:
determining, from the environment image information, an object to be held corresponding to the target instance; determining, based on the operation, movement parameter information of the object to be held; and controlling the holding apparatus to execute the task to be executed for the target instance based on the movement parameter information.

15 . The electronic device according to claim 14 , wherein the determining, based on the operation, the movement parameter information of the object to be held comprises:
when the operation is an operation of the user to move the target instance, obtaining an initial position of the target instance; obtaining a target position to which the target instance is moved; and determining the movement parameter information of the object to be held based on the initial position and the target position.

16 . A non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, causes implementation of:
displaying acquired environment image information; and controlling a holding apparatus to execute a task corresponding to an operation of a user on the environment image information, in response to the operation of the user on the environment image information.

Assignees

• Beijing Youzhuju Network Tech Co Ltd

Inventors

• Zhang Hanbo
• Xu Jie
• Li Yifeng
• Kong Tao

Classifications

• B25J9/16

  Program controls (total factory control, i.e. centrally controlling a plurality of machines, G05B19/418) · CPC title

• B25J9/1697

  Vision controlled systems · CPC title

• B25J9/1664

  characterised by motion, path, trajectory planning · CPC title

• B25J9/1669 Primary

  characterised by special application, e.g. multi-arm co-operation, assembly, grasping · CPC title

• G06T17/00

  Three-dimensional [3D] modelling for computer graphics · CPC title