Training and deploying pose regressions in neural networks in autonomous machines

Patent metadata

Field	Value
Publication number	US-11810318-B2
Application number	US-201616326005-A
Country	US
Kind code	B2
Filing date	Sep 9, 2016
Priority date	Sep 9, 2016
Publication date	Nov 7, 2023
Grant date	Nov 7, 2023

Abstract

A mechanism is described for facilitating training and deploying of pose regression in neural networks in autonomous machines. A method, as described herein, includes facilitating capturing, by an image capturing device of a computing device, one or more images of one or more objects, where the one or more images include one or more training images associated with a neural network. The method may further include continuously estimating, in real-time, a present orientation of the computing device, where estimating includes continuously detecting a real-time view field as viewed by the image capturing device and based on the one or more images. The method may further include applying pose regression relating to the image capturing device using the real-time view field.

First claim

What is claimed is:

1. An apparatus comprising:
one or more processors coupled to a memory, the one or more processors to:
continuously estimate, in real-time, a present orientation of the apparatus, wherein estimating includes continuously detecting, based on one or more images, a real-time view field as viewed by an image capturing device of the apparatus and communicatively coupled to the one or more processors, wherein the one or more images are captured by the image capturing device and include one or more training images, wherein estimating further includes estimating, in real-time, a difference between two consecutive rotations, wherein the difference is regarded as a prediction error; and
apply pose regression relating to the image capturing device using the real-time view field, and dynamically estimate, in real time, a future orientation of the apparatus based on the pose regression, wherein to apply pose regression further includes to apply one or more prediction errors to the pose regression such that the pose regression and the future orientation are adjusted based on the one or more prediction errors.

2. The apparatus of claim 1 , wherein the view field to provide at least one of translations representing global coordinates and rotations representing movements of the image capturing device along its axes, wherein applying pose regression includes adjusting the present orientation of the apparatus to facilitate accurate capturing of input data and offering of output results associated with workings of a neural network.

3. The apparatus of claim 1 , wherein the one or more processors are further to:
form at least one of rotation matrix and rotation quaternion corresponding to rotation representations of the one or more images; and
transition the rotation matrix or the rotation quaternion to decomposed angle representation using the angle estimator.

4. The apparatus of claim 3 , wherein decomposed angle representations include a plurality of angles associated with the movements of the image capturing device, wherein the plurality of angles are presented as one or more of cos(yaw), sin(yaw), cos(pitch), sin(pitch), cos(roll), and sin(roll).

5. The apparatus of claim 1 , wherein the input capturing device comprises at least one of one or more cameras, one or more robot eyes, one or more microphones, and one or more sensors, wherein the apparatus comprises an autonomous machine or an artificially intelligent agent, wherein the autonomous machine includes at least one of one or more robots, one or more self-driving vehicles, and one or more self-operating equipment.

6. A method comprising:
capturing, by an image capturing device of a computing device, one or more images of one or more objects;
continuously estimating, in real-time, a present orientation of the computing device, wherein estimating includes continuously detecting, based on one or more images, a real-time view field as viewed by the image capturing device of the computing device and communicatively coupled to one or more processors of the computing device, wherein the one or more images include one or more training images, wherein estimating further includes estimating, in real-time, a difference between two consecutive rotations, wherein the difference is regarded as a prediction error; and
applying pose regression relating to the image capturing device using the real-time view field, and dynamically estimating, in real-time, a future orientation of the computing device based on the pose regression, wherein applying pose regression further includes applying one or more prediction errors to the pose regression such that the pose regression and the future orientation are adjusted based on the one or more prediction errors.

7. The method of claim 6 , wherein the view field to provide at least one of translations representing global coordinates and rotations representing movements of the image capturing device along its axes, wherein applying pose regression includes adjusting the present orientation of the computing device to facilitate accurate capturing of input data and offering of output results associated with workings of the neural network.

8. The method of claim 6 , further comprising:
forming at least one of rotation matrix and rotation quaternion corresponding to rotation representations of the one or more images; and
transitioning the rotation matrix or the rotation quaternion to decomposed angle representation using the angle estimator.

9. The method of claim 8 , wherein decomposed angle representations include a plurality of angles associated with the movements of the image capturing device, wherein the plurality of angles are presented as one or more of cos(yaw), sin(yaw), cos(pitch), sin(pitch), cos(roll), and sin(roll).

10. The method of claim 6 , wherein the input capturing device comprises at least one of one or more cameras, one or more robot eyes, one or more microphones, and one or more sensors, wherein the computing device comprises an autonomous machine or an artificially intelligent agent, wherein the autonomous machine includes at least one of one or more robots, one or more self-driving vehicles, and one or more self-operating equipment.

11. At least one non-transitory computer readable medium having stored thereon instructions which, when executed, cause a computing device to perform operations comprising:
capturing, by an image capturing device of the computing device, one or more images of one or more objects;
continuously estimating, in real-time, a present orientation of the computing device, wherein estimating includes continuously detecting, based on one or more images, a real-time view field as viewed by the image capturing device of the computing device and communicatively coupled to one or more processors of the computing device, wherein the one or more images include one or more training images, wherein estimating further includes estimating, in real-time, a difference between two consecutive rotations, wherein the difference is regarded as a prediction error; and
applying pose regression relating to the image capturing device using the real-time view field, and dynamically estimating, in real-time, a future orientation of the computing device based on the pose regression, wherein applying pose regression further includes applying one or more prediction errors to the pose regression such that the pose regression and the future orientation are adjusted based on the one or more prediction errors.

12. The non-transitory computer readable medium of claim 11 , wherein the view field to provide at least one of translations representing global coordinates and rotations representing movements of the image capturing device along its axes, wherein applying pose regression includes adjusting the present orientation of the computing device to facilitate accurate capturing of input data and offering of output results associated with workings of the neural network.

13. The non-transitory computer-readable medium of claim 11 , wherein the operations further comprise:
forming at least one of rotation matrix and rotation quaternion corresponding to rotation representations of the one or more images; and

transition the rotation matrix or the rotation quaternion to decomposed angle representation using the angle estimator.

14. The non-transitory computer-readable medium of claim 13 , wherein decomposed angle representations include a plurality of angles associated with the movements of the image capturing device, wherein the plurality of angles are presented as one or more of cos(yaw), sin(yaw), cos(pitch), sin(pitch), cos(roll), and sin(roll

Assignees

• Intel Corp

Inventors

• Ma Liwei

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06T7/73 Primary

  using feature-based methods · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06T2207/20081

  Training; Learning · CPC title