Authors - Hiruni Samarage, Pumudu A. Fernando Abstract - Tertiary education institutes increasingly face challenges in managing high volumes of student inquiries related to admissions, courses, fees, and scholarships. Traditional inquiry-handling mechanisms and rule-based chatbots often struggle with scalability, delayed responses, and limited understanding of complex or unstructured queries. While recent advances in large language models (LLMs) offer promising opportunities, many existing academic chatbot implementations continue to lack semantic retrieval, session continuity, and personalization. This paper presents the design, implementation, and evaluation of an AI-based semantic chatbot prototype tailored for tertiary education environments. The prototype integrates retrieval-augmented generation with a large language model to enable context-aware responses across multiple institutional knowledge domains through semantic retrieval of structured knowledge representations. The system was evaluated using accuracy, precision, recall, robustness to query length variations, and retrieval effectiveness metrics. Experimental results demonstrate an overall accuracy of 86%, with precision and recall values of 89% and 91%, respectively. Robustness testing shows consistent performance across paraphrased and variable-length queries, while response times remained within acceptable limits for real-time academic support. User testing further indicated positive usability and response relevance outcomes. These results confirm the feasibility and effectiveness of applying semantic retrieval and LLM-based reasoning to scalable inquiry management in tertiary education contexts.
Open Zoom