The Machine and the OS Underneath

The morning the certificates stopped

Devika had been the backend engineer for the state government's citizen-services portal in Bhopal for about a year when the scholarship deadline arrived. Every student applying for the state's post-matric scholarship needed an income certificate, generated and digitally signed through the portal, uploaded before midnight. The scheme office had said so in a circular sent to every government school in the state. By nine in the morning, the certificate-generation queue had over ten thousand requests waiting in it, and the portal, which had handled a few hundred a day without complaint for six months, was struggling. Each certificate took a little longer to generate than the one before it. By eleven, requests were timing out outright. By noon, the server had stopped responding to anything at all, including Devika's own attempts to log in and see what was wrong.

Her manager wanted a fix by evening. Devika did not yet know that the answer was sitting quietly inside a machine she used every day and had never really looked at closely.

This chapter is about what Devika would spend the rest of that day learning to see.

1. The layers nobody showed you

Devika's server did not fail because of one bad line of code. It failed several layers below any line of code she had written, in a part of the machine most engineers never consciously look at. Before we get to what she found, it helps to see the layers on something smaller and calmer.

A student opens a laptop, opens a code editor, types print("hello"), and presses the run button. The word hello appears on the screen. The student moves on to the next exercise.

Stop for a moment. Something extraordinary just happened, and it is worth looking at properly, because most of us stop noticing it after the first week of learning to code.

Between the keypress and the pixel on the screen, your computer did, very approximately, the following. The keyboard sent an electrical signal to the operating system. The operating system woke up your code editor. Your code editor wrote the new character to a file. When you pressed run, the editor asked the operating system to start a new program. The operating system found your Python interpreter, loaded it from disk into memory, asked the CPU to start executing it, and gave it a small, protected box of memory to live in. The interpreter read your file, parsed the word print, looked up what print means, called the function, which asked the operating system, very politely, to please put the letters h, e, l, l, o, on a particular region of the screen. The operating system asked the graphics driver. The graphics driver asked the actual hardware. Pixels lit up.

That is the short version. The long version takes a semester. There are at least eight layers between your code and the pixel, and most students go their entire degree without consciously meeting any of them except the top one.

This is fine for the first three months of learning to code. It is not fine for the rest of your career. Every bug you will ever fix, every performance problem you will ever solve, every "it works on my machine" you will ever curse, traces back to one of these layers. You do not need to be an operating systems engineer. You need to know enough that when something breaks, you know which floor of the building to walk down to.

That is the only goal of this chapter. Not to make you an expert. To make you not afraid of the layers below.

2. What a CPU actually does

The CPU, the central processing unit, is the part of the computer that does work. Everything else is in service of it.

A CPU does one stupid thing very fast, billions of times per second. The stupid thing is: fetch an instruction from memory, figure out what it means, do it, move on. Fetch, decode, execute. That is the whole loop. An instruction might be "add these two numbers," or "compare these two values," or "jump to a different point in the program if this is true." There are a few hundred such instructions in a typical CPU's vocabulary. None of them, individually, can do anything interesting. Together, billions of times per second, they run YouTube and Photoshop and Linux and the entire internet.

The CPU has a tiny amount of extremely fast scratch space inside it called registers. A few dozen slots, each big enough to hold a single number. Every instruction either reads from registers, writes to registers, or moves data between registers and the outside world. The register is the smallest, fastest, most precious storage in your computer.

You will hear the phrase "this CPU runs at 3 GHz." This means the CPU's internal clock ticks three billion times per second, and at each tick it can advance through some part of the fetch-decode-execute loop. You should be careful with this number. A 3 GHz chip from 2024 will run circles around a 3 GHz chip from 2008, even though the clock speeds are identical. The new chip does more work per tick, predicts the future better, fetches more data in parallel, and has more clever tricks than its older cousin. Clock speed is a cricket-batting average, not a player.

The other word you will hear is cores. A modern laptop CPU has somewhere between four and sixteen cores. A core is, roughly, a complete CPU on its own. A four-core CPU can do four things at exactly the same instant. This is important, and it is also less important than you think, because most code is not written to use multiple cores at the same time, and getting it to do so correctly is one of the hardest problems in our field. We will return to this in Section 4.

You may also have noticed that newer laptops talk about performance cores and efficiency cores. The chip designers gave up trying to make every core both fast and power-saving, and put two kinds of cores on the same chip. The fast ones get used for heavy work. The slow ones handle background tasks and save your battery. Apple started this trend on phones, and now everyone does it.

That is enough about the CPU for this book. We have not said a word about instruction sets or assembly or pipelines, and we are not going to. The mental model you need is: the CPU is the thing that does the work, it has a few dozen tiny scratch slots called registers, it has multiple cores that can do things at the same time, and the speed at which it ticks is not the same as the speed at which it actually gets things done.

3. The memory hierarchy, and why it explains half of all performance bugs

Your computer has many places to put data. They are not equal.

The fastest place is the CPU's registers, which we just met. The next fastest is something called the cache, which lives on the CPU itself, very close to the registers. The cache comes in layers, called L1, L2, and L3, getting larger and slower as the numbers go up. After the cache comes RAM, which is the main memory of your computer, the eight or sixteen or thirty-two gigabytes you bought when you got your laptop. After RAM comes your SSD or hard drive. After your disk comes the network, which means anything not on your machine, including the entire internet.

Each of these layers is roughly ten to a hundred times slower than the one above it. Look at the actual numbers, because they will change how you think about everything.

A register access takes less than one nanosecond. A nanosecond is one billionth of a second. The L1 cache takes about one nanosecond. The L2 cache, about three. The L3 cache, about ten. RAM, about one hundred nanoseconds. An SSD, about one hundred thousand nanoseconds, which is one hundred microseconds. A spinning hard disk, about ten million nanoseconds, which is ten milliseconds. A network round trip from India to a server in Europe, about one hundred million nanoseconds, which is one hundred milliseconds.

Read those numbers again. The difference between the fastest and the slowest is a factor of one hundred million. If a register access took one second, a network round trip would take three years.

Here is one of the most important things a junior engineer can internalise about performance. A great deal of slow code is slow because the data is in the wrong layer — not because the algorithm is bad, not because the language is slow, but because the program keeps reaching for data that lives far away when it could have arranged for the data to be close. This is not the only cause of slowness, and it would be dishonest to say it is. A genuinely bad algorithm, checking every item against every other item when a smarter approach would not have to, can be slower than any amount of data locality can fix. Lock contention, where many threads queue up waiting for the same piece of memory, causes a different kind of slowdown that data placement does not touch. A slow external dependency, an API you call that takes three seconds to answer, will make you slow no matter how close your own data sits. Data locality is a frequent cause and a high-leverage one to check first, alongside these others, not a replacement for looking at the algorithm, the contention, and the dependencies too.

A small example, without code. Imagine you are looking up the marks of every student in your college, one by one, by walking to the records office for each lookup. That is what a program does when it goes to the disk for every piece of data. Now imagine you walk to the records office once, photocopy a hundred records at a time, bring them back to your desk, and look them up locally. That is what a program does when it reads data into memory in batches. The second version is faster not because you are smarter, but because the data is closer when you need it.

This single idea, bring the data closer to the work, is the heart of why caches exist, why databases have indexes, why CDNs exist, why your browser stores things locally, why everything in our field that touches performance touches some version of this idea. Once you see it, you see it everywhere.

4. Processes, threads, and what "running" really means

Your laptop, right now, is running dozens of programs. The browser. The code editor. The music app. The Wi-Fi service. The clock. The notification manager. Twenty more you do not know about.

But your CPU has, let us say, eight cores. Eight things at a time, maximum. How is the laptop running fifty things?

The honest answer is that it is not. The operating system is lying to all the programs, very fast, and all the programs believe the lie.

Here is how the lie works. The OS runs Program A on a core for a few milliseconds. Then it pauses Program A in the middle of whatever it was doing, saves everything Program A was thinking about (the values in its registers, what line of code it was on), and gives the core to Program B. A few milliseconds later, it pauses Program B and gives the core to Program C. After a tiny pause, it brings Program A back, restores everything it saved, and lets Program A continue exactly where it left off, as if no time had passed. This is called context switching, and your computer does it thousands of times per second, on every core, all day.

Each running program is called a process. A process has its own protected box of memory that no other process can see or touch. This is one of the most important inventions in the history of computing. Without it, every crashed program would take the whole machine down with it, and every program could read every other program's data. The process boundary is what makes modern computing safe enough to be useful.

This is also the layer Devika's frozen server was quietly struggling with, though she would not know it for a few more hours. Her portal was not making one process do more work than it could handle. It was making far too many processes at once, and every single one of them needed the operating system to remember it, schedule it, and set aside memory for it. A machine can survive a lot of processes. It cannot survive an unbounded number of them.

Sometimes a single process wants to do more than one thing at once. For example, your browser wants to download a file in the background while you keep scrolling. For this, the process can have multiple threads. Threads are like multiple workers inside the same process. They share the same box of memory, which is what makes them fast. They share the same box of memory, which is also what makes them dangerous. Two threads writing to the same piece of memory at the same time can corrupt it in ways that are very hard to debug. This is called a race condition, and it is the source of some of the most painful bugs in software.

Imagine two threads, both trying to add money to your bank balance. Thread A reads the current balance, which is one hundred rupees. Thread B reads the current balance, also sees one hundred rupees. Thread A adds fifty and writes one hundred and fifty. Thread B adds fifty and writes one hundred and fifty. You deposited a hundred rupees in total and your balance went up by fifty. The bank is short fifty rupees and nobody knows why.

This is a race condition. It is rare, it is non-deterministic, and it is the reason why "it worked when I tested it" is the most dangerous sentence a young engineer can say.

There is a useful distinction between two words that sound similar and mean different things. Concurrency is dealing with many things at once. Parallelism is doing many things at once. A single chef cooking three dishes by switching between them is concurrent. Three chefs cooking three dishes at the same time is parallel. Your operating system is concurrent on every machine. It is parallel only when you have multiple cores. Most of the time, you want concurrency. Sometimes you want parallelism. The two get confused constantly, including by people who should know better.

We have not written a line of multithreaded code, and we are not going to in this book. The mental model is what matters. Processes are isolated. Threads share memory and are dangerous. The OS is lying to all of them about how much CPU they have, very fast, and most of the time the lie is good for everyone.

5. User space, kernel space, and the system call

We need to talk about why your program cannot just do whatever it wants.

Your program, when it runs, lives in something called user space. Almost everything you write runs there. The operating system itself, the part of it that talks directly to the hardware, lives in something called kernel space. The two are separated by an invisible wall that the CPU itself helps enforce.

Why? Because if every program could talk directly to the hard disk, every program could overwrite every file. If every program could talk directly to the network card, every program could spy on every other program's traffic. The wall between user space and kernel space is the reason you can run untrusted software without it taking your whole machine hostage. Most of the time.

But your program needs to do real things in the world. It needs to read a file. It needs to send data over the network. It needs to draw on the screen. For each of these, it has to ask the kernel. The way it asks is called a system call.

Think of the kernel as a very busy bouncer at a very exclusive club. Your program walks up to the door and says, "Please read me bytes one to a thousand of the file called notes.txt." The bouncer checks if your program is allowed (do you own the file? are the permissions right?), then goes inside, fetches what you asked for, and hands it back. Your program waits during this exchange. The whole transaction is one system call.

This sounds like overhead because it is. Crossing the wall between user space and kernel space costs time. Not a lot of time, but enough that if your program is making millions of system calls per second, it will spend most of its life standing at the bouncer's door instead of doing actual work.

This one concept, the system call, explains a startling number of things. It explains why disk and network operations are slow even when the disk and network are fast. It explains why containers like Docker work (they fool programs into thinking they have their own kernel, while sharing the real one). It explains why security models work (every dangerous action goes through the bouncer). It explains why some "make it faster" tricks work and others do not. If your bottleneck is the kernel, no amount of cleverness in your code will help, and you have to find a way to make fewer trips to the bouncer.

Most students go years without learning what a system call is. You have learned it in two pages. You will see it referenced in documentation, in Stack Overflow answers, in error messages, in performance discussions, for the rest of your career. Now you know what they mean.

6. The filesystem, and the magic of ls -la

Your computer stores files in a tree. Not a list. A tree. There is a root, and folders inside it, and folders inside those, and files at the leaves. On Linux and macOS the root is called /. On Windows it is C:\. The shape of the tree is the same.

Each file is, at the lowest level, a sequence of bytes. The bytes might be text, an image, a video, a program, a database file, anything. The operating system does not really know what is in the file. It just stores the bytes and remembers some metadata about each one. Who owns the file. When it was created. When it was last modified. Who is allowed to read it, write to it, or run it as a program.

That last point matters more than students realise. On any Unix-like system, which is to say Linux, macOS, every cloud server you will ever touch, and every Docker container you will ever build, every file has a permissions triplet. Three bits for the owner of the file, three bits for the owner's group, three bits for everyone else. Read, write, execute. If you ever see a string like -rwxr-xr--, that is what it means. The owner can read, write, and execute. The group can read and execute. Everyone else can only read.

I want to flag something honestly. Most Indian engineering colleges teach Windows in the labs and never mention Linux except in passing. Then the student joins a job, gets handed an SSH key, and is told to log into a Linux server. The first day on the new job is spent quietly confused about why nothing looks the same. This is one of the most common silent struggles of the first six months of a career, and almost nobody talks about it. We will fix this together. The next chapter is about living in the terminal properly, including on Linux, including on a Mac, and including on Windows through a tool called WSL that makes your Windows machine pretend to be Linux when you need it to.

7. What Linux, macOS, and Windows actually share

You probably use Windows. Maybe macOS. You will work on Linux servers, even if you never install Linux on your laptop, because almost every cloud server in the world runs some flavour of it.

The good news is that everything in this chapter is true on all three operating systems. Processes exist on all of them. Threads exist on all of them. Memory works the same way. System calls exist (Windows calls them something different but the idea is identical). The filesystem is a tree on all of them. The mental model transfers.

The names and the details differ, sometimes annoyingly. Windows uses backslashes for file paths and the others use forward slashes. Windows permissions look different from Unix permissions. But the underlying ideas are shared.

Two practical notes for your reality. If you are on Windows and feeling left out of the Linux world, install something called WSL, the Windows Subsystem for Linux. One command, a few minutes, and your Windows machine grows a real Linux installation that lives alongside Windows and shares your files. Almost every "Linux" thing you read about online will work inside WSL. Many professional developers on Windows do all their work inside WSL and only use Windows for the browser and the meetings.

If you are on macOS, congratulations, you are already on a Unix system in disguise. The terminal on a Mac is a real Unix terminal. Most things that work on Linux also work on a Mac with the same commands. The few that do not, you will discover in their own time, and they will not bite you often.

9. Where this shows up

Devika found her answer using exactly the tool this chapter has been building toward: a look at the process list on the struggling server. Every certificate request was spawning a brand new process to render and sign the PDF, and under ten thousand simultaneous requests, thousands of these processes were alive at once, each holding its own share of memory, none of them being cleaned up fast enough to make room for the next. The machine was not slow. It was drowning in its own children. The fix that evening was to reuse a small pool of long-running processes instead of spawning a fresh one for every request. The certificates started flowing again by seven.

This layer shows up everywhere once you know to look for it. A fintech company's nightly settlement batch grinding to a halt is often a process or memory problem wearing a business costume. A hospital's on-premise records server slowing down every afternoon is frequently the same OS-level story: too many processes, not enough memory, nobody having looked below the application layer. An e-commerce warehouse system freezing during a flash sale is, often enough, this exact chapter, playing out again, in a different building, for different stakes.

10. A quick check before you move on

Three questions, no looking back until you have tried each one.

1. Why does "the CPU runs at 3 GHz" tell you less than it sounds like it tells you?
2. In your own words, what was actually wrong with Devika's server — not "it was slow," but which specific resource was running out?
3. What is the difference between concurrency and parallelism, and which one does your laptop have all the time, regardless of how many cores it has?

If the third one is fuzzy, reread Section 4 before the ritual. It is the idea most students think they understood on the first pass and did not.