LLMASP A Framework for Mitigating Hallucinations and Supporting Symbolic Reasoning in Large Language Models

Abstract

Answer Set Programming (ASP) and Large Language Models (LLMs)
are two influential branches of artificial intelligence, offering complementary
strengths in knowledge representation and natural-language understanding.
This thesis investigates how they can be integrated to build structured representations of complex information expressed in natural language. The proposed workflow combines the syntactic structures produced by an LLM with
the semantic elements stored in a knowledge base, unifying them in an ASP formalism. A YAML configuration file containing prompt templates and domainspecific background knowledge governs the interaction between the LLM and
the ASP solver. The LLM first converts user input into an ASP atom structure; the ASP solver then reasons over this structure and derives additional
atoms, which are sent back to the LLM for explanation before being delivered to the user. The results presented in this thesis show how the reasoning
power increases when using LLMASP, illustrating different ways of achieving
this by employing various GBNF grammar rules for generating ASP atoms,
such as CSV, JSON, and direct ASP atom grammar. The differences between
these approaches are analyzed, highlighting how each grammar impacts the
precision of the extraction process. It also shows how to extract the ASP code
logic from the problem description by using GBNF grammar to make sure it
generates correct executable syntax.

Date of Award	2025
Original language	English
Supervisor	Stephan Dreiseitl (Supervisor) & Mario Alviano (Supervisor)