Authors - Vanderson dos Santos Araujo, Eliane Tamara Lima Oliveira, Pedro Manoel Hermínio Alves, Andre Luiz Firmino Alves, Claudio de Souza Baptista Abstract - Auditing public tenders requires analyzing lengthy documents to verify compliance with tender notices, a time-consuming task prone to human error. This article empirically evaluates the use of Large Language Models (LLMs) to assist with auditing tender notices. A total of 50 official tender notices and 736 audit instances were analyzed, comparing three context-provisioning strategies: expanded context windows, integrated file retrieval, and a custom Retrieval-Augmented Generation (RAG) pipeline. The results show that no single approach is superior across all scenarios. Models with long windows performed better at confirming explicit conformities, whereas retrieval-based strategies demonstrated greater sensitivity to potential non-conformities due to omissions. The analysis also indicates that the type of question strongly influences performance, especially for interpretive questions or those that rely on the absence of documentary evidence. As a key contribution, the study demonstrates that the effectiveness of AI-assisted auditing depends on the combination of the contextualization strategy, the quality of the retrieved context, and the formulation of the questions, reinforcing the role of LLMs as tools to support the auditor.
Open Zoom