Praca Feliksa Schustera.
[T]his dissertation explores the topic of application software security in three modern adversarial settings: (i) the classic setting, (ii) the backdoor setting in which the attacker may additionally add a backdoor to a component of a software, and (iii) the cloud setting in which the attacker largely controls hardware and software and may at
different places read or manipulate a cloud applications code and data.
In this dissertation, we begin with an evaluation of existing defensive approaches (e. g.,
control-flow integrity) in the classic setting. Thereto, we present various advanced codereuse
attacks. Our attacks break with commonly held assumptions on the nature of
code-reuse attacks and as such bypass many existing academic and commercial defenses.
Among others, our results here show that purely intuitive arguments or limited empirical
evidence are no sufficient criteria for the security of a defense.
We discuss the challenges of the backdoor and the cloud adversarial settings and propose
and evaluate novel ways to tackle them. We present a dynamic analysis approach for
the detection and dismantling of backdoors in binary server applications across different
processor architectures. Among others, we demonstrate how our approach can disarm realworld
backdoors (e. g., in a malicious version of OpenSSH) in a fully automated manner.
Furthermore, we describe the first end-to-end secure system for the execution of distributed
(MapReduce) applications in the untrusted cloud. The security of this system
founds on two novel cryptographic protocols—for which we provide proof—and Intel’s
SGX technology as hardware-rooted trusted computing base (TCB)