Big GCVAE: decision-making with adaptive transformer model for failure root cause analysis in semiconductor industry - FAYOL / DEMO : Décision en Entreprise : Modélisation, Optimisation Access content directly
Journal Articles Journal of Intelligent Manufacturing Year : 2024

Big GCVAE: decision-making with adaptive transformer model for failure root cause analysis in semiconductor industry

Abstract

Pre-trained large language models (LLMs) have gained significant attention in the field of natural language processing (NLP), especially for the task of text summarization, generation, and question answering. The success of LMs can be attributed to the attention mechanism introduced in Transformer models, which have outperformed traditional recurrent neural network models (e.g., LSTM) in modeling sequential data. In this paper, we leverage pre-trained causal language models for the downstream task of failure analysis triplet generation (FATG), which involves generating a sequence of failure analysis decision steps for identifying failure root causes in the semiconductor industry. In particular, we conduct extensive comparative analysis of various transformer models for the FATG task and find that the BERT-GPT-2 Transformer (Big GCVAE), fine-tuned on a proposed Generalized-Controllable Variational AutoEncoder loss (GCVAE), exhibits superior performance in generating informative latent space by promoting disentanglement of latent factors. Specifically, we observe that fine-tuning the Transformer style BERT-GPT2 on the GCVAE loss yields optimal representation by reducing the trade-off between reconstruction loss and KL-divergence, promoting meaningful, diverse and coherent FATs similar to expert expectations.
Fichier principal
Vignette du fichier
Big_GCVAE soumis.pdf (2.26 Mo) Télécharger le fichier
Origin Files produced by the author(s)
licence

Dates and versions

emse-04530213 , version 1 (04-04-2024)
emse-04530213 , version 2 (22-04-2024)

Licence

Identifiers

Cite

Kenneth Ezukwoke, Anis Hoayek, Mireille Batton-Hubert, Xavier Boucher, Pascal Gounet, et al.. Big GCVAE: decision-making with adaptive transformer model for failure root cause analysis in semiconductor industry. Journal of Intelligent Manufacturing, 2024, ⟨10.1007/s10845-024-02346-x⟩. ⟨emse-04530213v1⟩
40 View
11 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More