Methods and apparatus for discriminative semantic transfer and physics-inspired optimization of features in deep learning

Patent metadata

Field	Value
Publication number	US-12079713-B2
Application number	US-202318142997-A
Country	US
Kind code	B2
Filing date	May 3, 2023
Priority date	May 23, 2017
Publication date	Sep 3, 2024
Grant date	Sep 3, 2024

Abstract

Methods and apparatus for discrimitive semantic transfer and physics-inspired optimization in deep learning are disclosed. A computation training method for a convolutional neural network (CNN) includes receiving a sequence of training images in the CNN of a first stage to describe objects of a cluttered scene as a semantic segmentation mask. The semantic segmentation mask is received in a semantic segmentation network of a second stage to produce semantic features. Using weights from the first stage as feature extractors and weights from the second stage as classifiers, edges of the cluttered scene are identified using the semantic features.

First claim

What is claimed is:

1. At least one non-transitory machine-readable medium comprising instructions stored thereon, that if executed by one or more circuitry, cause the one or more circuitry to:
perform a first stage configured to receive a sequence of training images in a convolutional neural network (CNN) to describe objects of a cluttered scene as a semantic segmentation mask;
perform a second stage configured to receive the semantic segmentation mask in a semantic segmentation network and to produce semantic features; and
perform a third stage configured to use weights from the first stage as feature extractors and weights from the second stage as classifiers in order to identify at least one partially occluded edge of the cluttered scene using the semantic features.

2. The at least one machine-readable medium of claim 1 , wherein performance of the third stage is to adjust the CNN that receives the sequence of training images.

3. The at least one machine-readable non-transitory medium of claim 1 , wherein performance of the second stage is to apply a softmax operation in the semantic segmentation network.

4. The at least one machine-readable non-transitory medium of claim 1 , wherein the objects of the cluttered scene are part of a room layout dataset.

5. The at least one machine-readable non-transitory medium of claim 4 , wherein
the room layout dataset comprises pixels and
the second stage is configured to process pixels of the room layout dataset as a sample in a connected layer of the semantic segmentation network.

6. The at least one machine-readable non-transitory medium of claim 4 , wherein performance of the second stage is further to model relationships between the room layout dataset and the objects in the CNN.

7. The at least one machine-readable non-transitory medium of claim 1 , wherein performance of the third stage is further to label edges of the cluttered scene.

8. An apparatus comprising:
a memory device and
at least one processor, coupled to the memory device, the at least one processor configured to:
perform a first stage configured to receive a sequence of training images in a convolutional neural network (CNN) to describe objects of a cluttered scene as a semantic segmentation mask;
perform a second stage configured to receive the semantic segmentation mask in a semantic segmentation network and to produce semantic features; and
perform a third stage configured to use weights from the first stage as feature extractors and weights from the second stage as classifiers in order to identify at least one partially occluded edge of the cluttered scene using the semantic features.

9. The apparatus of claim 8 , wherein to perform the third stage, the at least one processor is to adjust the CNN that receives the sequence of training images.

10. The apparatus of claim 8 , wherein to perform the second stage, the at least one processor is to use a softmax operation in the semantic segmentation network.

11. The apparatus of claim 8 , wherein the objects of the cluttered scene are part of a room layout dataset.

12. The apparatus of claim 11 , wherein
the room layout dataset comprises pixels and
the second stage is configured to process pixels of the room layout dataset as a sample in a connected layer of the semantic segmentation network.

13. The apparatus of claim 11 , wherein to perform the second stage, the at least one processor is to model relationships between the room layout dataset and the objects in the CNN.

14. The apparatus of claim 8 , wherein to perform the third stage, the at least one processor is to label edges of the cluttered scene.

15. A computer-implemented method comprising:
performing a first stage configured to receive a sequence of training images in a convolutional neural network (CNN) to describe objects of a cluttered scene as a semantic segmentation mask;
performing a second stage configured to receive the semantic segmentation mask in a semantic segmentation network and to produce semantic features; and
performing a third stage configured to use weights from the first stage as feature extractors and weights from the second stage as classifiers in order to identify at least one partially occluded edge of the cluttered scene using the semantic features.

16. The method of claim 15 , wherein the performing the third stage comprising adjusting the CNN that receives the sequence of training images.

17. The method of claim 15 , wherein the performing the second stage applies a softmax operation in the semantic segmentation network.

18. The method of claim 15 , wherein
the objects of the cluttered scene are part of a room layout dataset,
the room layout dataset comprises pixels, and
the second stage is configured to process pixels of the room layout dataset as a sample in a connected layer of the semantic segmentation network.

19. The method of claim 18 , wherein the performing the second stage comprises modeling relationships between the room layout dataset and the objects in the CNN.

20. The method of claim 15 , wherein the performing the third stage comprises labelling edges of the cluttered scene.

Assignees

• Intel Corp

Inventors

• Yao Anbang
• Zhao Hao
• Lu Ming
• Guo Yiwen
• Chen Yurong

Classifications

• G06N3/096

  Transfer learning · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title