Self-modification and dynamic behavior analysis

In this talk, I will present some works in progress on computer virology which are made in the team Carte at Nancy University thanks to the High Security Lab (http://lhs.loria.fr/). Nowadays, all malware are obfuscated by using for example home-made packers. In other words, malware are self-modifying programs. As a result it is difficult to analyze them. I will present a simple formal representation of self-modifying programs and a dynamical analysis of them which reconstruct a partial representation. Then, I will talk about a work on behavior abstraction in order to analyze malware, which was presented at Runtime Verification conference in 2010. Our technique consists in abstracting program traces, by rewriting given subtraces into abstract symbols representing their functionality. Traces are captured dynamically by code instrumentation, which allows us to handle packed or self-modifying malware.