Method for constructing design concept generation network (dcgn) and method for automatically generating conceptual scheme

Patent metadata

Field	Value
Publication number	US-2024005130-A1
Application number	US-202318120434-A
Country	US
Kind code	A1
Filing date	Mar 13, 2023
Priority date	Jul 4, 2022
Publication date	Jan 4, 2024
Grant date	—

Abstract

A method for constructing a design concept generation network (DCGN) and a method for automatically generating a conceptual scheme are provided. A DCGN includes a Transformer encoder, a Transformer decoder, an importance constraint matrix generation module, an importance constraint embedding layer, a cross-attention (CA) layer, and an optimization module. A word importance constraint is ingeniously introduced based on an attention mechanism of a Transformer to record input word constraint information contained in a generated text sequence. This can effectively ensure the reliability and effectiveness of a generated conceptual scheme and is conducive to capturing potential semantic importance information and implementing semantic knowledge reasoning.

First claim

What is claimed is:

1 . A method for constructing a design concept generation network (DCGN), wherein the DCGN comprises a Transformer encoder, a Transformer decoder, an importance constraint matrix generation module, an importance constraint embedding layer, a cross-attention (CA) layer, and an optimization module; and the method comprises the following steps:
S1: obtaining, by the Transformer encoder, a feature of a hidden layer of the Transformer encoder based on input words in a sample; S2: obtaining, by the Transformer decoder, a feature of a hidden layer of the Transformer decoder based on a target sequence in the sample; S3: obtaining, by the importance constraint matrix generation module, an importance constraint matrix based on the input words and the target sequence in the sample; S4: mapping, by the importance constraint embedding layer, the importance constraint matrix to a distributed vector space to obtain two input word importance embedding features; S5: obtaining, by the CA layer, a generated sequence based on the feature of the hidden layer of the Transformer encoder, the feature of the hidden layer of the Transformer decoder, and the two input word importance embedding features; and S6: constructing a loss function based on the generated sequence and the target sequence, and adjusting, by the optimization module, network parameters based on the loss function; and repeating S1 to S6 until the loss function meets a specified requirement to obtain the DCGN.

2 . The method according to claim 1 , wherein in S1, the Transformer encoder obtains the feature h e of the hidden layer of the Transformer encoder by using the following formula:
h e =SA( W e K x,W e V x,W e Q x )  (1),
wherein x represents the input words; SA( ) represents a spatial attention; and W e K , W e V , and W e Q represent weight matrices of a self-attention layer of the Transformer encoder.

3 . The method according to claim 1 , wherein in S2, the Transformer decoder maps a target sequence y :t-1 =[y 0 ,y 1 , . . . , y t-1 ] at a moment t−1 to a distributed feature representation through a self-attention layer to obtain the feature h d t of the hidden layer of the Transformer decoder:
h d t =SA( W d K y :t-1 ,W d V y :t-1 ,W d Q y :t-1 )  (2),

wherein SA( ) represents a spatial attention; and W d K , W d V , and W d Q represent weight matrices of the self-attention layer of the Transformer decoder.

4 . The method according to claim 3 , wherein in S3, f(x, w , y :t ) represents an input word importance constraint vector C :t contained in the target sequence Y :t ;
f(x, w ,y :t ) is calculated as follows:
f ( x, w ,y :t )= w ·c t   (4),

wherein ⋅· represents a dot product operation of a vector or a matrix; and w =[ w 1 , w 2 , . . . , w i , . . . , w m ]∈ m represents a relative importance vector of the input words x in the target sequence y :t and is calculated as follows:

w
¯

i

=

[

w
i

-

w
min

w
max

-

w
min

×

(

M
-
1

)

]

,

∀

i
∈

{

1
,
2
,
⋯
,
m

}

,

(
5
)

wherein w i represents a relative importance of an i th input word in the target sequence y :t ; w i represents an absolute importance of the i th input word in the target sequence y :t ; w min represents a minimum absolute importance of the input word in the target sequence y :t ; w max represents a maximum absolute importance of the input word in the target sequence y :t ; [ ] represents a rounding operation; and M≥m>1 and M represents a maximum number of input words contained in samples in an entire training sample set; and
c t ∈ m represents an input word constraint contained in the target sequence y :t ; when the target sequence y :t contains the i th input word, an i th element in the vector c t is 1, and the vector is calculated as follows:

c
t

=

(

c
t
i

)

i
=
1

m

=

{

c
t
i

=
0

,

if
⁢

Assignees

• Univ Sichuan

Inventors

• Zhao Wu
• Yu Miao
• Guo Xin
• Zhang Kai
• Zhao Qian
• Yu Hui
• Li Jun
• Lai Bing
• Jiang Chong
• Jiang Yiwei
• Wu Bo
• Chen Xingyu

Classifications

• G06N3/0455 Primary

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/08

  Learning methods · CPC title

• G06F17/16 Primary

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title

• G06N3/0475

  Generative networks · CPC title

• G06N3/047

  Probabilistic or stochastic networks · CPC title