As the importance of computer systems in modern-day societies grows, so
does the damage that malicious software causes. This led the security industry
to engage in an arms race against malware authors to create better
systems to detect malware and prevent it fromspreading. On their side, to
cope with the advances in the field of malware analysis, malware authors
sharpened their tools with the objective of thwarting the analysis and defeating
countermeasures. In this arms race, in fact, all wrong assumptions
(no matter how subtle) may allow malware to circumvent detection systems,
effectively running unopposed for a long period of time.
This thesis focuses on two aspects of modern malware analysis techniques
that are often overlooked, namely the use of API-level information
for encoding malicious behavior and the reimplementation of parsing
routines for executable file formats in security-oriented tools. This thesis
shows that taking advantage of these practices is possible on a large and
automated scale. By reviewing recent evidence brought to light by security
researchers and hunting malware in the wild, we also demonstrate that
malware authors show increasing interest in exploiting these practices.
Lastly,we study the feasibility of fixing these problems at their roots, measuring
the difficulties that anti-malware architects may encounter and
providing strategies to solve them.
© EURECOM. Personal use of this material is permitted. The definitive version of this paper was published in Thesis and is available at :