TRAINING AND EVALUATING THE USE OF LARGE LANGUAGE MODELS (LLMS) IN THE DOMAIN OF CANADIAN NUCLEAR INDUSTRY

Abstract

This thesis addresses the challenges of accuracy, reliability, data privacy, and resource constraints in applying Large Language Models (LLMs) to the Canadian nuclear industry. It presents a multi-faceted approach by evaluating existing models, developing synthetic data generation techniques, and training a secure, domain-specific LLM from scratch.

The research first demonstrates that while general-purpose LLMs are prone to factual inaccuracies on nuclear-specific topics, their reliability is significantly improved by integrating a Retrieval-Augmented Generation (RAG) framework. This approach enhances factual accuracy by grounding responses in verified, domain-specific documents.

To overcome data scarcity and confidentiality barriers, the thesis pioneers a methodology for generating synthetic, structured question-and-answer pairs from unstructured nuclear texts using LLMs. This scalable and privacy-preserving approach creates valuable, model-ready datasets for training and evaluation without exposing sensitive information.

Furthermore, the work validates the feasibility of developing a secure, private LLM from scratch. By training a compact model on a single GPU using the "Essential CANDU" textbook, it demonstrates a practical path for creating in-house models that mitigate cybersecurity risks and can learn specialized terminology within a resource-constrained and secure environment.

Collectively, this research provides a comprehensive framework for integrating LLM technology safely and effectively into the nuclear industry, establishing a foundation for advanced AI tools that enhance knowledge management and operational support.