FINE-TUNING LIGHTWEIGHT LLM FOR SELF-CORRECTING TEXT-TO-SQL THROUGH A DYNAMIC FEEDBACK DATASET

This thesis explores fine-tuning methodologies for Lightweight Large Language Models (LLMs) to significantly enhance their Text-to-SQL generation capabilities, specifically by equipping them with an autonomous self-correction mechanism. The core of this research is an innovative approach centered on the creation and utilization of a unique self-correction dataset. This dataset is dynamically generated through an automated critique-correction loop and used to fine-tune models via Low-Rank Adaptation (LoRA). The methodology involves exposing the LLM to initially flawed SQL generations, followed by specific error feedback and the corresponding corrected SQL queries. This structured dataset teaches the model to interpret diagnostic feedback and apply targeted fixes. Unlike large-scale state-of-the-art systems such as OmniSQL, which rely on millions of synthetic training examples, our approach achieves competitive performance using a substantially smaller dataset. This demonstrates that high efficiency and strong generalization can be obtained without extensive data or heavy computational resources. Furthermore, we introduce novel prompt engineering techniques that incorporate special tags to guide the model's correction process more effectively. The ultimate goal is to develop LLMs capable of autonomous error remediation and optimized operational efficiency, paving the way for more reliable and self-improving Text-to-SQL systems.