1. The Invisible Arbitrator
At its core, an operating system (OS) is a resource allocator. It serves as the vital intermediary, the arbitrator, between the visible applications you interact with and the invisible, complex metal and silicon of the hardware.
If you were to anthropomorphize a computer, the CPU is the brain, but the OS is the conscious mind that directs focus. It decides which application gets access to the CPU, how long it stays there, and which chunk of memory it occupies. Without this arbitration, two programs trying to write to the same memory address simultaneously would result in digital chaos.
A Brief History: The GM-NAA I/O
To understand where we are, we must look at where we started. The lineage of the modern OS traces back to 1956 with the creation of GM-NAA I/O.
Developed by General Motors for the IBM 704 mainframe, this is widely cited as the first "real" operating system. Before GM-NAA I/O, computing was a manual, laborious process. Programmers effectively had to book time slots on the machine, manually load their punch cards, and wait for execution. If a program crashed, the machine sat idle until the human operator noticed.
GM-NAA I/O introduced batch processing. It allowed the computer to move automatically from one job to the next without human intervention. While other systems like GM-OS and GM-RS existed, GM-NAA I/O is historically significant because it was the first to successfully automate job execution, setting the stage for the Unix, Linux, and Windows systems we use today.
2. The Case for Customization: Why Reinvent the Wheel?
We live in a world dominated by General Purpose Operating Systems (GPOS) like Linux and Windows. These are "Swiss Army Knives", designed to do everything reasonably well. However, in extreme engineering, "reasonably well" is often a bottleneck.
Many high-performance industries bypass standard commercial operating systems in favor of proprietary or heavily customized kernels:
3. The Anatomy of a Minimal OS
What constitutes a "minimal" operating system? It depends on the mission. A bare-bones kernel must, at the absolute minimum, handle three things:
Bootstrapping: Getting the hardware running.
Memory Management: Preventing programs from overwriting each other.
Execution: Running instructions.
However, to build a functional environment, we generally look at the following seven pillars. Below is a deep dive into building these components, using Linux architecture principles as our North Star.
I. A Minimal File System (RAMFS)
The Concept:
A filesystem organizes data into a hierarchy. For a minimal OS, interacting with a physical hard drive (SATA/NVMe) is complex due to driver requirements. The solution is RAMFS (Random Access Memory File System).
How to Build It:
Instead of writing to a disk, you write a driver that treats a chunk of your RAM as if it were a disk.
VFS Layer: You need a Virtual File System abstraction. This allows your OS to use standard commands (like open, read, write) without caring that the "disk" is actually just a memory array.
Implementation: In Linux, ramfs simply utilizes the kernel's existing disk caching mechanisms. For your custom OS, you will allocate a dynamic list of memory blocks. When a user "saves a file," you are essentially memcpying data into that block and creating a pointer (an inode) that names it.
II. A Minimal Scheduler
The Concept:
The scheduler is the traffic cop. It decides which process runs on the CPU and for how long.
How to Build It:
The Structure: You need a data structure (like the task_struct in Linux) to hold the state of a program (its registers, stack pointer, and status).
The Algorithm: Start with Round Robin.
- Set a timer interrupt (e.g., every 10ms).
- When the timer fires, the CPU pauses the current task.
- Save the current task's registers to memory (Context Save).
- Load the next task's registers from memory (Context Restore).
- Resume execution.
III. Minimal Paging (Memory Management)
The Concept:
Paging creates the illusion that every program has access to the entire memory of the computer (Virtual Memory), while physically keeping them safe and separate.
How to Build It:
Hardware Support: On x86 processors, you must interact with the MMU (Memory Management Unit).
Page Tables: You will create a hierarchical table structure (Page Directory → Page Table → Physical Address).
CR3 Register: You load the physical address of your Page Directory into the CPU's CR3 register. Once enabled, any memory address a program tries to access is intercepted by the hardware and translated according to your tables. This allows you to implement "swapping" (moving idle data to disk) later on.
IV. A Minimal Network Driver
The Concept:
Networking is essentially moving packets from a wire into memory.
How to Build It:
The Ring Buffer: Most network cards (NICs) use a Circular Ring Buffer (RX for receiving, TX for transmitting).
DMA (Direct Memory Access): You shouldn't make the CPU copy every byte. Instead, you configure the NIC to write incoming packets directly into a specific area of RAM via DMA.
Interrupts: When the buffer has data, the NIC sends an interrupt to the CPU. Your driver catches this interrupt and processes the packet.
Optimization: In high-performance custom OS designs (like the HFT example), you might disable interrupts and use Polling (constantly checking the card) to avoid the overhead of pausing the CPU.
V. A Minimal Scheduler/Task Class
The Concept:
Linux uses "Scheduling Classes" to treat different types of processes differently. Real-time processes need different rules than background downloads.
How to Build It:
Modularity: Don't hardcode your scheduling logic. Create an interface (a struct of function pointers) called sched_class.
Methods: This interface should define methods like enqueue_task, dequeue_task, and pick_next_task.
The Benefit: This allows you to have a "Real-Time Class" (FIFO) and a "Normal Class" (Round Robin) running side-by-side in your OS.
VI. Minimal I/O (Input/Output)
The Concept:
I/O is how the CPU talks to peripherals (Keyboard, Mouse, Screen).
How to Build It:
Ports vs. Memory Mapped: Legacy hardware often uses I/O ports (communicated via inb and outb instructions). Modern hardware uses Memory Mapped I/O (MMIO), where you write to a specific "memory address" that is actually a configuration register on a device.
The Interrupt Descriptor Table (IDT): You must set up an IDT. When a key is pressed, the keyboard controller sends a signal. The CPU looks up the IDT to find your "Keyboard Handler" function, which then reads the keystroke.
VII. Minimal System Calls
The Concept:
Applications live in "User Mode" (Ring 3) and cannot touch hardware directly. They must ask the Kernel (Ring 0) to do it for them. This request is a System Call.
How to Build It:
The Crossing: You need a mechanism to switch security rings safely.
The Instruction: On modern x86_64, you use the syscall instruction (or int 0x80 on older legacy systems).
The Table: You build a table where each number corresponds to a function (e.g., 0 = read, 1 = write, 2 = open).
The Flow:
- User program puts the System Call ID in the RAX register.
- User program executes
syscall. - CPU switches to Kernel Mode and jumps to your
syscall_handler. - Kernel executes the function and returns the result.
The True Minimum Requirements
Even though we talked about the 7 core parts of an operating system modeled after Linux, the presence of all 7 is not that required. The following is just enough for a minimal OS:
- Boot
- Interrupts
- Task struct
- Scheduler
- Context switch
- Page/frame allocator
- Basic paging
- Syscalls
- Console output (not strictly required)