Educational institutions can leverage large language models (LLMs) to streamline the processing and interpretation of complex policy manuals. In this study, we benchmarked two locally hosted, offline LLMs against official university bylaws using a standard desktop computer. Across ten representative administrative queries, Qwen2.5:7B demonstrated clearer analytical precision, scoring an average of 4.80 out of 5 versus Mistral:7B’s 4.30. By systematically accounting for conditional rules and policy nuances, Qwen2.5:7B achieved a perfect score on 80% of queries, whereas Mistral:7B reached 50%. On the other hand, Mistral:7B responded slightly faster—averaging 1.567 seconds per query compared to Qwen2.5:7B’s 1.782 seconds. Crucially, neither model generated severe hallucinations or total answer failures. These results confirm that offline LLMs offer institutions a privacy-safe, cost-effective workflow, where administrators can select Mistral:7B for rapid retrieval or Qwen2.5:7B for high-precision policy extraction.

暂无