Title: Random Testing for Security: Blackbox vs. Whitebox Fuzzing
Speaker: Patrice Godefroid, Microsoft Research, Redmond, USA
Abstract
Fuzz testing is an effective technique for finding security vulnerabilities
in software. Fuzz testing is a form of blackbox
random testing which randomly mutates well-formed inputs
and tests the program on the resulting data. In some cases,
grammars are used to randomly generate the well-formed
inputs. This also allows the tester to encode application-specific
knowledge (such as corner cases of particular interest)
as part of the grammar, and to specify test heuristics
by assigning probabilistic weights to production rules. Although
fuzz testing can be remarkably effective, the limitations
of blackbox random testing are well-known. For
instance, the then branch of the conditional statement if
(x==10) thenhas only one in 232 chances of being exercised
if x is a randomly chosen 32-bit input value. This intuitively
explains why random testing usually provides low code coverage.
Recently, we have proposed an alternative approach of
whitebox fuzz testing [4], building upon recent advances in
dynamic symbolic execution and test generation [2]. Starting
with a well-formed input, our approach symbolically
executes the program dynamically and gathers constraints
on inputs from conditional statements encountered along
the way. The collected constraints are then systematically
negated and solved with a constraint solver, yielding new inputs
that exercise different execution paths in the program.
This process is repeated using a novel search algorithm with
a coverage-maximizing heuristic designed to find defects as
fast as possible in large search spaces. For example, symbolic
execution of the above code fragment on the input x = 0 generates the constraint x != 10. Once this constraint is
negated and solved, it yields x = 10, which gives us a new
input that causes the program to follow the then branch of
the given conditional statement.
We have implemented this approach in SAGE (Scalable,
Automated, Guided Execution), a tool based on x86 instruction-level
tracing and emulation for whitebox fuzzing of file-reading
Windows applications. While still in an early stage of development
and deployment, SAGE has already discovered
more than 30 new bugs in large shipped Windows applications
including image processors, media players and file
decoders. Several of these bugs are potentially exploitable
memory access violations.
In this talk, I will briefly review blackbox fuzzing for security
testing. Then, I will present an overview of our recent
work on whitebox fuzzing [4] (joint work with Michael
Y. Levin and David Molnar), with an emphasis on the key
algorithms and techniques needed to make this approach effective
and scalable (see also [1, 3]).
About the speaker
Patrice Godefroid is a Principal Researcher at Microsoft Research. He
received the B.S. degree in Electrical Engineering (Computer Science
elective) and the Ph.D. degree in Computer Science from the University
of Liege, Belgium, in 1989 and 1994 respectively. From 1994 to 2006,
he worked at Bell Laboratories (part of Lucent Technologies), where he
was promoted to "distinguished member of technical staff" in 2001. His
research interests include program (mostly software) specification,
analysis, testing and verification.
|