Vulnerability Hunting Using Concolic-Guided Fuzzing

Abstract

Automatic detection of vulnerabilities in programs is a hard task in its own right, made
even harder when the source code of the program is not available for analysis, since
information about data structures and high-level context is missing after compilation.
Furthermore, modern compilers may heavily optimize or even obfuscate the resulting
binary program, removing debugging information, which makes the task of logically
analyzing the resulting binary even harder.
The usage of fuzz testing has been the de facto standard for discovering vulnerabilities within binary code for many years, which has sparked the development of many
open-source fuzzing tools. Prominent companies such as Microsoft and Google utilize
it extensively in the discovery of security flaws in their products. One major shortcoming of fuzzing is the inherent weakness against sanity checks and expected input
structures, which would require deeper semantic understanding of the program under
test to overcome.
Concolic execution attempts to solve this problem by examining the program with
concrete inputs and determining which inputs would traverse previously unevaluated
paths of the program. Concolic-guided fuzzing is a concolic execution engine and a
fuzzing engine working together in order to achieve better code coverage and enhance
the fuzzer’s ability to find security flaws.
In this master’s thesis, different approaches to vulnerability detection using fuzzing,
symbolic and concolic execution, and concolic-guided fuzzing of binary programs are
examined, including how binary analysis using concolic execution and vulnerability detection with fuzzing may be performed, and what the limitations of these technologies
are. Furthermore, the CFTracer custom concolic-guided fuzzing framework has been designed and implemented with an emphasis on modularity and scalability. The CFTracer
framework was evaluated using the FuzzBench fuzzing benchmark framework to test
whether concolic-guided fuzzing is viable for improving fuzzing performance.
CFTracer was evaluated using 22 test cases from the FuzzBench benchmarking
framework while using AFL++ as the fuzzing backend and SymQEMU as the concolic execution engine. The results of CFTracer were compared to a standalone instance
of AFL++, evaluating the FuzzBench results of the concolic-guided fuzzer against an
unassisted instance of AFL++. The results showed that, while CFTracer was able to
outperform the standalone fuzzer on 16 benchmarks with an average median coverage
increase of 1.34%, the performance increase provided by concolic execution was dependent on whether SymQEMU was able to contribute novel inputs from analyzing the
target binary.

Date of Award	2025
Original language	English
Supervisor	Harald Lampesberger (Supervisor)