Do you speak binary? Can you comprehend machine code? If I gave you a sheet full of 1s and 0s, could you tell me what it means/does? If you were to go to a country you’ve never been to that speaks a language you’ve never heard, or maybe you’ve heard of it but don’t actually speak it, what would you need while there to help you communicate with the locals?

You would need a translator. Your operating system functions as that translator in your PC. It converts those 1s and 0s, yes/no, and on/off values into a readable language that you will understand. It does all of this in a streamlined graphical user interface, or GUI, that you can move around with a mouse, click things, move them, and see them happening before your eyes.

While the extent and depth of knowledge can be questioned, knowing more than the fundamentals can be critical to how well your program runs and even to its structure and flow.

Why? When you write a program and it runs too slowly, but you see nothing wrong with your code, where else will you look for a solution? How will you be able to debug the problem if you don’t know how the operating system works? Are you accessing too many files? Running out of memory and swap is in high usage? But you don’t even know what swap is! Or is I/O blocking?

And you want to communicate with another machine. How do you do that locally or over the internet? And what’s the difference? Why do some programmers prefer one OS over another?

In an attempt to be a serious developer, I recently took Georgia Tech’s course “Introduction to Operating Systems.” It teaches the basic OS abstractions, mechanisms, and their implementations. The core of the course contains concurrent programming (threads and synchronization), inter-process communication, and an introduction to distributed OSs.

I want to use this post to share my takeaways from the course, that is the 10 critical operating system concepts that you need to learn if you want to get good at developing software.

What is an Operating System?

But first, let’s define what an operating system is. An Operating System (OS) is a collection of software that manages computer hardware and provides services for programs. Specifically, it hides hardware complexity, manages computational resources, and provides isolation and protection. Most importantly, it directly has privilege access to the underlying hardware.

Major components of an OS are the file system, scheduler, and device driver. You probably have used both Desktop (Windows, Mac, Linux) and Embedded (Android, iOS) operating systems before.

There are three key elements of an operating system, which are: (1) Abstractions (process, thread, file, socket, memory), (2) Mechanisms (create, schedule, open, write, allocate), and (3) Policies (LRU, EDF).

There are two operating system design principles, which are: (1) Separation of mechanism and policy by implementing flexible mechanisms to support policies, and (2) Optimization for common case: Where will the OS be used? What will the user want to execute on that machine? What are the workload requirements?

There are three types of Operating Systems commonly used nowadays. The first is Monolithic OS, where the entire OS is working in kernel space and is alone in supervisor mode. The second is Modular OS, in which some part of the system core will be located in independent files called modules that can be added to the system at run time. And the third is Micro OS, where the kernel is broken down into separate processes, known as servers. Some of the servers run in kernel space and some run in user-space.

Now let’s get into those major concepts you need to understand in more detail.

1: Processes and Process Management

A process is basically a program in execution. The execution of a process must progress in a sequential fashion. To put it in simple terms, we write our computer programs in a text file, and when we execute this program, it becomes a process which performs all the tasks mentioned in the program.

When a program is loaded into the memory and it becomes a process, it can be divided into four sections ─ stack, heap, text, and data. The following image shows a simplified layout of a process inside main memory

Stack: The process Stack contains the temporary data, such as method/function parameters, return address, and local variables.

The process Stack contains the temporary data, such as method/function parameters, return address, and local variables. Heap: This is dynamically allocated memory to a process during its run time.

This is dynamically allocated memory to a process during its run time. Text: This includes the current activity represented by the value of Program Counter and the contents of the processor’s registers.

This includes the current activity represented by the value of Program Counter and the contents of the processor’s registers. Data: This section contains the global and static variables.

When a process executes, it passes through different states. These stages may differ in different operating systems, and the names of these states are also not standardized. In general, a process can have one of the following five states at a time:

Start: The initial state when a process is first started/created.

The initial state when a process is first started/created. Ready: The process is waiting to be assigned to a processor. Ready processes are waiting to have the processor allocated to them by the operating system so that they can run. A process may come into this state after the Start state, or while running it by but getting interrupted by the scheduler to assign CPU to some other process.

The process is waiting to be assigned to a processor. Ready processes are waiting to have the processor allocated to them by the operating system so that they can run. A process may come into this state after the state, or while running it by but getting interrupted by the scheduler to assign CPU to some other process. Running: Once the process has been assigned to a processor by the OS scheduler, the process state is set to running and the processor executes its instructions.

Once the process has been assigned to a processor by the OS scheduler, the process state is set to running and the processor executes its instructions. Waiting: the process moves into the waiting state if it needs to wait for a resource, such as waiting for user input, or waiting for a file to become available.

the process moves into the waiting state if it needs to wait for a resource, such as waiting for user input, or waiting for a file to become available. Terminated or Exit: Once the process finishes its execution, or it is terminated by the operating system, it is moved to the terminated state where it waits to be removed from main memory.

A Process Control Block is a data structure maintained by the Operating System for every process. The PCB is identified by an integer process ID (PID). A PCB keeps all the information needed to keep track of a process as listed below:

Process State: The current state of the process — whether it is ready, running, waiting, or whatever.

The current state of the process — whether it is ready, running, waiting, or whatever. Process Privileges: This is required to allow/disallow access to system resources.

This is required to allow/disallow access to system resources. Process ID: Unique identification for each of the processes in the operating system.

Unique identification for each of the processes in the operating system. Pointer: A pointer to the parent process.

A pointer to the parent process. Program Counter: Program Counter is a pointer to the address of the next instruction to be executed for this process.

Program Counter is a pointer to the address of the next instruction to be executed for this process. CPU Registers: Various CPU registers where processes need to be stored for execution for running state.

Various CPU registers where processes need to be stored for execution for running state. CPU Scheduling Information: Process priority and other scheduling information which is required to schedule the process.

Process priority and other scheduling information which is required to schedule the process. Memory Management Information: This includes the information of page table, memory limits, and segment table, depending on the memory used by the operating system.

This includes the information of page table, memory limits, and segment table, depending on the memory used by the operating system. Accounting Information: This includes the amount of CPU used for process execution, time limits, execution ID, and so on.

This includes the amount of CPU used for process execution, time limits, execution ID, and so on. IO Status Information: This includes a list of I/O devices allocated to the process.

2: Threads and Concurrency

A thread is a flow of execution through the process code. It has its own program counter that keeps track of which instruction to execute next. It also has system registers which hold its current working variables, and a stack which contains the execution history.

A thread shares with its peer threads various information like code segment, data segment, and open files. When one thread alters a code segment memory item, all other threads see that.

A thread is also called a lightweight process. Threads provide a way to improve application performance through parallelism. Threads represent a software approach to improving the performance of operating systems by reducing the overhead. A thread is equivalent to a classical process.

Each thread belongs to exactly one process, and no thread can exist outside a process. Each thread represents a separate flow of control. Threads have been successfully used in implementing network servers and web servers. They also provide a suitable foundation for parallel execution of applications on shared memory multiprocessors.

Advantages of threads:

They minimize the context switching time.

Using them provides concurrency within a process.

They provide efficient communication.

It is more economical to create and context switch threads.

Threads allow utilization of multiprocessor architectures to a greater scale and efficiency.

Threads are implemented in the following two ways:

User Level Threads: User-managed threads.

User-managed threads. Kernel Level Threads: Operating System-managed threads acting on a kernel, an operating system core.

User Level Threads

In this case, the thread management kernel is not aware of the existence of threads. The thread library contains code for creating and destroying threads, for passing messages and data between threads, for scheduling thread execution, and for saving and restoring thread contexts. The application starts with a single thread.

Advantages:

Thread switching does not require Kernel mode privileges.

User level thread can run on any operating system.

Scheduling can be application-specific in the user level thread.

User level threads are fast to create and manage.

Disadvantages:

In a typical operating system, most system calls are blocking.

Multithreaded application cannot take advantage of multiprocessing.

Kernel Level Threads

In this case, thread management is done by the Kernel. There is no thread management code in the application area. Kernel threads are supported directly by the operating system. Any application can be programmed to be multithreaded. All of the threads within an application are supported within a single process.

The Kernel maintains context information for the process as a whole and for individuals threads within the process. Scheduling by the Kernel is done on a thread basis. The Kernel performs thread creation, scheduling, and management in Kernel space. Kernel threads are generally slower to create and manage than the user threads.

Advantages

The Kernel can simultaneously schedule multiple threads from the same process on multiple processes.

If one thread in a process is blocked, the Kernel can schedule another thread of the same process.

Kernel routines themselves can be multithreaded.

Disadvantages

Kernel threads are generally slower to create and manage than the user threads.

Transfer of control from one thread to another within the same process requires a mode switch to the Kernel.

3: Scheduling

The process of scheduling is the responsibility of the process manager that handles the removal of the running process from the CPU and the selection of another process on the basis of a particular strategy.

Process scheduling is an essential part of a Multiprogramming operating system. These operating systems allow more than one process to be loaded into the executable memory at a time, and the loaded process shares the CPU using time multiplexing.

The OS maintains all Process Control Blocks (PCBs) in Process Scheduling Queues. The OS maintains a separate queue for each of the process states, and PCBs of all processes in the same execution state are placed in the same queue. When the state of a process is changed, its PCB is unlinked from its current queue and moved to its new state queue.

The Operating System maintains the following important process scheduling queues:

Job queue: This queue keeps all the processes in the system.

This queue keeps all the processes in the system. Ready queue: This queue keeps a set of all processes residing in the main memory, ready and waiting to execute. A new process is always put in this queue.

This queue keeps a set of all processes residing in the main memory, ready and waiting to execute. A new process is always put in this queue. Device queues: The processes which are blocked due to unavailability of an I/O device constitute this queue.

The OS can use different policies to manage each queue (FIFO, Round Robin, Priority, etc.). The OS scheduler determines how to move processes between the ready and run queues which can only have one entry per processor core on the system. In the above diagram, it has been merged with the CPU.

Two-state process models refer to running and non-running states:

Running: When a new process is created, it enters into the system in the running state.

When a new process is created, it enters into the system in the running state. Not Running: Processes that are not running are kept in queue, waiting for their turn to execute. Each entry in the queue is a pointer to a particular process. Queue is implemented by using a linked list. The use of dispatcher is as follows: when a process is interrupted, that process is transferred in the waiting queue. If the process has completed or aborted, the process is discarded. In either case, the dispatcher then selects a process from the queue to execute.

A context switch is the mechanism that stores and restores the state or context of a CPU in the Process Control block. It allows a process execution to be resumed from the same point at a later time. Using this technique, a context switcher enables multiple processes to share a single CPU. Context switching is an essential feature of a multitasking operating system.

When the scheduler switches the CPU from executing one process to another, the state from the current running process is stored into the process control block. After this, the state for the next process is loaded from its own PCB and used to set the PC, registers, etc. At that point, the second process can start executing.

Context switches are computationally intensive, since register and memory state must be saved and restored. To avoid the amount of context switching time, some hardware systems employ two or more sets of processor registers.

When the process is switched, the following information is stored for later use: Program Counter, Scheduling Information, Base and Limit Register Value, Currently Used Register, Changed State, I/O State Information, and Accounting Information.

4: Memory Management

Memory management is the functionality of an operating system which handles or manages primary memory. It moves processes back and forth between the main memory and the disk during execution.

Memory management keeps track of each and every memory location, regardless of whether it is allocated to some process or free. It checks how much memory is to be allocated to processes. It decides which process will get memory at what time. And it tracks whenever memory gets freed up or unallocated, and correspondingly updates the status.