Abstract
The widespread use of embedded systems (ES) in critical infrastructures, including industrial control systems and medical devices, necessitates the implementation of robust security mechanisms. These systems typically rely on Real-Time Operating Systems (RTOS) to ensure predictable responses, deterministic performance, and multitasking capabilities. However, their stringent real-time constraints and resource limitations present unique security challenges that limit the applicability of conventional defense mechanisms. By examining recent security practices within a continually evolving threat landscape, this study aims to utilize Capability Hardware Enhanced RISC Instructions (CHERI) in RTOS, emphasizing enhanced security. CHERI seeks to enforce spatial memory safety through hardware-based compartmentalization, effectively isolating vulnerabilities and reducing the attack surface. Nevertheless, the fail-stop exception handling model of CHERI-based RTOS, which immediately terminates tasks upon capability violations, presents significant availability risks in domains that require high uptime and graceful degradation. To address this limitation, this dissertation introduces a dynamic privilege adjustment framework that proactively analyzes triggered exceptions to detect security breaches and adaptively restricts permissions based on anomalous patterns. This approach integrates real-time exception monitoring with automated capability revocation to mitigate threats while preserving system functionality.
Additionally, while the CHERI compartmentalization model effectively reduces attack surfaces, its reliance on traditional inter-compartment communication mechanisms, such as message queues and shared libraries, introduces considerable overhead. These methods often necessitate multiple memory copies and frequent scheduler involvement, increasing latency and consuming valuable processing cycles. This challenge is particularly pronounced in high-frequency producer-consumer scenarios and large data transfers across compartments. To address this inefficiency, this dissertation also proposes a zero-copy cross-compartment communication model that utilizes futex-based synchronization primitives.
Experimental validation on the CHERIoT RTOS demonstrates that the proposed framework effectively prevents attacks across fault injection scenarios while preserving system functionality, in contrast to CHERIoT's default task termination. Additionally, the zero-copy communication model significantly diminishes communication latency and computational overhead compared to traditional methods that rely on multiple memory copies and scheduler involvement.