The operating system is the software component that provides abstraction of physical hardware, and provides a generalized model for application software to execute without the need for specific hardware knowledge. Questions on the topic of operating system security should use this tag; you may also like to use one of the [windows], [linux] or [macos] tags.
Operating systems are critical to security because they provide the fundamental mechanisms for control of processes, separation of data, control of access to data, and control of access to system resources.
Questions you may have / recommended reading
- Hardening Linux Server
- Are passwords in memory?
- Secure Linux Desktop
- Windows hardening
- Security of passwords remembered by Windows
- Can viruses actually inject their code in other processes?
- Mac OS X from an IT Security Practitioners' Perspective?
Operating system architecture
Operating systems are built according, simply speaking, to two pure philosophies:
- Monolithic Kernels - these are huge binary blobs containing the entire compiled code of the operating system (hence "monolithic"). Such code bases benefit from being able to directly alter (with locks) variables belonging to other kernel threads and using locks as concurrency guards; as such these types of kernels are often very quick.
- Microkernels, by contrast, often contain a small core and push the majority of operating system features out into "servers" i.e. processes which send messages back to the core component and to other components to get work done. Due to the need for message queues and message processing, these kernels often perform more slowly.
A simple comparison is available which roughly explains the difference. The security implications of the design of each kernel lie in the use of the CPU's security features; microkernels are able to run much more of the operating system as unprivileged processes.
In reality, modern operating systems borrow features from both types of kernel design.
Access Control
A number of access control systems are in use / considered in operating systems:
- Discretionary Access Control, a method of access control that allows child processes to inherit the privileges of the context (i.e. user or parent process) from which they were launched.
- Mandatory Access Control - a method of access control through which each subject (process, user) is given explicit permissions on an object (resource) regardless of the parent's rights.
- Role-based Access Control is a third and increasingly more prevalent method of access control through which more complicated organisational structures are used to determine privilege.
Currently, there are also a number of research-area access control methods including:
- multi-level/multi-category security. These types of access control are often implemented on top of MAC and add labels and/or category requirements to further isolate data.
- capability-based security is an entirely different concept to the use of access control lists (ACLs) and requires processes pass/inherit/ask for capabilities and possessing these capabilities gives them the right to perform actions. Capability-based security is strongly associated with microkernel-based design.
Process/Memory Isolation
Most operating systems provide some form of process isolation capability and utilise hardware to do so. By far the most common commodity chip is the x86 and descendant families of processor, which provide several abilities:
- CPU "rings" or privilege levels. Labelled 0, 1, 2 and 3, these determine what an executing process can do to the system.
- Memory segmentation is common in older x86 systems and was a method of dividing up the address space.
- Memory paging - the ability to swap memory to and from disk as required, to keep virtual memory free.
- Virtual memory - using the concept that memory can be paged out to increase free space, most operating systems take advantage of the ability to swap out entire processes when not in use and thus provide memory isolation (in addition to the protections provided by privilege levels). In addition, it is possible to design the operating system such that the OS and userland do not even share an address space.
- No Execute bits, W^X etc implement software/hardware concepts of the idea that writeable memory for user processes should not also be executable, to prevent stack-based vulnerabilities.
Virtualization
Virtualization is the process of making an operating system run on emulated hardware, or co-exist on common hardware such that it is unaware it does not have complete control of the system. For more information, have a look at the closely-related virtualization tag.
Resources
