Counterfactual context-aware texture learning for camouflaged object detection

Patent metadata

Field	Value
Publication number	US-12136261-B2
Application number	US-202418644727-A
Country	US
Kind code	B2
Filing date	Apr 24, 2024
Priority date	Mar 13, 2023
Publication date	Nov 5, 2024
Grant date	Nov 5, 2024

Abstract

A counterfactual context-aware texture learning network system, including: a camera configured to capture an input image; a processor configured to perform camouflaged object detection on the input image; and a memory configured to store a texture-aware refinement module (TRM), a context-aware fused module (CFM), and a counterfactual intervention module (CIM); wherein the processor is configured to execute program instructions of the TRM, the CFM, and the CIM; the TRM is configured to extract dimension features from the input image; the CFM is configured to infuse multi-scale contextual features; the CIM is configured to identify a camouflaged object with counterfactual intervention via the processor; the TRM includes: a receptive field block (RFB) configured to expand a receptive field and extract texture features; and a position attention module (PAM) and a channel attention module (CAM) configured to further refine texture-aware features and obtain discriminant feature representation.

First claim

What is claimed is:

1. A counterfactual context-aware texture learning network system, comprising:
a camera configured to capture an input image;
a processor configured to perform camouflaged object detection on the input image; and
a memory configured to store a texture-aware refinement module (TRM), a context-aware fused module (CFM), and a counterfactual intervention module (CIM);
wherein the processor is configured to execute program instructions of the TRM, the CFM, and the CIM;
the TRM is configured to extract dimension features from the input image;
the CFM is configured to infuse multi-scale contextual features;
the CIM is configured to identify a camouflaged object with counterfactual intervention via the processor;
the TRM comprises:
a receptive field block (RFB) configured to expand a receptive field and extract texture features; and
a position attention module (PAM) and a channel attention module (CAM) configured to further refine texture-aware features and obtain discriminant feature representation;

the RFB comprises five branches b k , (k=1,2,3,4,5), each branch of the five branches comprising a 1×1 convolution operation to reduce a channel size to 64;
each branch where k>2 further comprises a 1×(2i−1) convolutional layer, a (2i−1)×1 convolutional layer, and a (2i−1)×(2i−1) convolutional layer, with a dilation rate of (2i−1), where i=k−1;
each branch where k>1 is concatenated, input into a second 1×1 convolution operation, and added with a branch of the five branches where k=1;
a result of the RFB is input into a Rectified Linear Unit (ReLU) activation function to obtain an output feature f i ′∈ C×H×W , where C, H and W represent a channel number, a channel height, and a channel width, respectively;
the output feature f′ is input into the PAM and the CAM,
the PAM is configured to:
obtain three feature maps B, C, and D through three convolution layers, where {B, C, and D}∈ C×H×W , and the three feature maps are reshaped to C×N ; and
multiply the transpose of B by C, and perform a softmax layer to calculate the spatial attention map sa∈ N×N :

s
⁢

a

i
⁢
j

=

exp
⁡
(

B
i

·

C
j

)

∑

i
=
1

N

⁢

exp
⁡
(

B
i

·

C
j

)

(
1
)

where sa ij denotes the j th position's impact on the i th position;
a loss function L=L BCE W +L IoU W is used to train the counterfactual context-aware texture learning network system to learn effective textures, where L BCE W is the weighted binary cross entropy (BCE) loss which restricts each pixel, and Lou is a weighted intersection-over-union (IoU) loss that focuses on a global structure; and
a total loss is formulated as:
L total =L ( Y,y )+λ L ( Y effect ,y )  (2)

where y is a ground truth, λ=0:1, L(Y, y) are main clues which learn general texture features, Y is a prediction of the main clues, and λL(Y effect , y) is a counterfactual term that penalizes a wrong prediction affected by contextual biases;
thereby performing the camouflaged object detection in the input image with enhanced accuracy.

2. The counterfactual context-aware texture learning network system of claim 1 , wherein the PAM is configured to:
multiply the transpose of sa by a matrix of the D feature map and reshape an aggregated attentive features result to C×H×W ;
multiply the aggregated attentive features result by a scale parameter η and apply an element-wise sum operation with the output feature f′ to obtain spatial feature maps f p ∈ C×H×W :

f
i
p

=

η
⁢

∑

j
=
1

N

(

s
⁢

a

i
⁢
j

⁢

D
j

)

+

f
′

(
3
)

where η is initialized as 0 and gradually learns more weight, and f p is a weighted sum at each position which enhances a semantic representation of the feature.

3. The counterfactual context-aware texture learning network system of claim 2 , wherein the CAM is configured to reshape the f i ′ to C×N , multiply a transpose of f′ by the f′ matrix, and apply a softmax layer to obtain channel attention maps ca∈ C×C :

Assignees

• National Univ Of Defense Technology

Inventors

• Li Shuohao
• Li Xiaofei
• Zhang Jun
• Huang Kuihua
• Chen Chao
• Sun Boliang
• Lei Jun
• Yu Miaomiao

Classifications

• G06V10/54 Primary

  relating to texture · CPC title

• G06V10/768

  using context analysis, e.g. recognition aided by known co-occurring patterns · CPC title

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title